mlpack::ann::FastLSTM< InputDataType, OutputDataType > Class Template Reference

An implementation of a faster version of the Fast LSTM network layer. More...

#include <fast_lstm.hpp>

Public Types
typedef InputDataType::elem_type	InputElemType
typedef OutputDataType::elem_type	ElemType

Public Member Functions
	FastLSTM ()
	Create the Fast LSTM object.
	FastLSTM (const FastLSTM &layer)
	Copy Constructor.
	FastLSTM (FastLSTM &&layer)
	Move Constructor.
FastLSTM &	operator= (const FastLSTM &layer)
	Copy assignment operator.
FastLSTM &	operator= (FastLSTM &&layer)
	Move assignment operator.
	FastLSTM (const size_t inSize, const size_t outSize, const size_t rho=std::numeric_limits< size_t >::max())
	Create the Fast LSTM layer object using the specified parameters. More...
template<typename InputType , typename OutputType >
void	Forward (const InputType &input, OutputType &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...
template<typename InputType , typename ErrorType , typename GradientType >
void	Backward (const InputType &input, const ErrorType &gy, GradientType &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...
void	Reset ()
void	ResetCell (const size_t size)
template<typename InputType , typename ErrorType , typename GradientType >
void	Gradient (const InputType &input, const ErrorType &error, GradientType &gradient)
size_t	Rho () const
	Get the maximum number of steps to backpropagate through time (BPTT).
size_t &	Rho ()
	Modify the maximum number of steps to backpropagate through time (BPTT).
OutputDataType const &	Parameters () const
	Get the parameters.
OutputDataType &	Parameters ()
	Modify the parameters.
OutputDataType const &	OutputParameter () const
	Get the output parameter.
OutputDataType &	OutputParameter ()
	Modify the output parameter.
OutputDataType const &	Delta () const
	Get the delta.
OutputDataType &	Delta ()
	Modify the delta.
OutputDataType const &	Gradient () const
	Get the gradient.
OutputDataType &	Gradient ()
	Modify the gradient.
size_t	InSize () const
	Get the number of input units.
size_t	OutSize () const
	Get the number of output units.
size_t	WeightSize () const
	Get the size of the weight matrix.
size_t	InputShape () const
	Get the shape of the input.
template<typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer.

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::FastLSTM< InputDataType, OutputDataType >

An implementation of a faster version of the Fast LSTM network layer.

Basically by combining the calculation of the input, forget, output gates and hidden state in a single step. The standard formula changes as follows:

\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \cdot c + i \cdot z \\ o &=& sigmoid(W \cdot x + W \cdot h + b) \\ h &=& o \cdot tanh(c) \end{eqnarray}

Note that FastLSTM network layer does not use peephole connections between the cell and gates.

Note also that if a FastLSTM layer is desired as the first layer of a neural network, an IdentityLayer should be added to the network as the first layer, and then the FastLSTM layer should be added.

For more information, see the following.

@article{Hochreiter1997,
  author  = {Hochreiter, Sepp and Schmidhuber, J\"{u}rgen},
  title   = {Long Short-term Memory},
  journal = {Neural Comput.},
  year    = {1997},
  url     = {https://www.bioinf.jku.at/publications/older/2604.pdf}
}

See also

LSTM for a standard implementation of the LSTM layer.

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Constructor & Destructor Documentation

FastLSTM()

template<typename InputDataType , typename OutputDataType >

mlpack::ann::FastLSTM< InputDataType, OutputDataType >::FastLSTM	(	const size_t	inSize,
		const size_t	outSize,
		const size_t	rho = std::numeric_limits<size_t>::max()
	)

Create the Fast LSTM layer object using the specified parameters.

Parameters

inSize	The number of input units.
outSize	The number of output units.
rho	Maximum number of steps to backpropagate through time (BPTT).

Member Function Documentation

Backward()

template<typename InputDataType , typename OutputDataType >

template<typename InputType , typename ErrorType , typename GradientType >

void mlpack::ann::FastLSTM< InputDataType, OutputDataType >::Backward	(	const InputType &	input,
		const ErrorType &	gy,
		GradientType &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Forward()

template<typename InputDataType , typename OutputDataType >

template<typename InputType , typename OutputType >

void mlpack::ann::FastLSTM< InputDataType, OutputDataType >::Forward	(	const InputType &	input,
		OutputType &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

---

The documentation for this class was generated from the following files:

• src/mlpack/methods/ann/layer/fast_lstm.hpp
• src/mlpack/methods/ann/layer/fast_lstm_impl.hpp