mlpack::ann::PositionalEncoding< InputDataType, OutputDataType > Class Template Reference

Positional Encoding injects some information about the relative or absolute position of the tokens in the sequence. More...

#include <positional_encoding.hpp>

Public Member Functions
	PositionalEncoding ()
	Create PositionalEncoding object.
	PositionalEncoding (const size_t embedDim, const size_t maxSequenceLength)
	Create the PositionalEncoding layer object using the specified parameters. More...
template<typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...
template<typename eT >
void	Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...
InputDataType const &	InputParameter () const
	Get the input parameter.
InputDataType &	InputParameter ()
	Modify the input parameter.
OutputDataType const &	OutputParameter () const
	Get the output parameter.
OutputDataType &	OutputParameter ()
	Modify the output parameter.
OutputDataType const &	Delta () const
	Get the delta.
OutputDataType &	Delta ()
	Modify the delta.
InputDataType const &	Encoding () const
	Get the positional encoding vector.
size_t	InputShape () const
template<typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer.

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::PositionalEncoding< InputDataType, OutputDataType >

Positional Encoding injects some information about the relative or absolute position of the tokens in the sequence.

The input and the output have the same shape: (embedDim * maxSequenceLength, batchSize). The embeddings are stored consequently.

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Constructor & Destructor Documentation

PositionalEncoding()

template<typename InputDataType , typename OutputDataType >

mlpack::ann::PositionalEncoding< InputDataType, OutputDataType >::PositionalEncoding	(	const size_t	embedDim,
		const size_t	maxSequenceLength
	)

Create the PositionalEncoding layer object using the specified parameters.

Parameters

embedDim	The length of the embedding vector.
maxSequenceLength	Number of tokens in each sequence.

Member Function Documentation

Backward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::PositionalEncoding< InputDataType, OutputDataType >::Backward	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

*	(input) The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Forward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::PositionalEncoding< InputDataType, OutputDataType >::Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

---

The documentation for this class was generated from the following files:

• src/mlpack/methods/ann/layer/positional_encoding.hpp
• src/mlpack/methods/ann/layer/positional_encoding_impl.hpp