mlpack::ann::AlphaDropout< InputDataType, OutputDataType > Class Template Reference

The alpha - dropout layer is a regularizer that randomly with probability 'ratio' sets input values to alphaDash. More...

#include <alpha_dropout.hpp>

Public Member Functions
	AlphaDropout (const double ratio=0.5, const double alphaDash=-alpha *lambda)
	Create the Alpha_Dropout object using the specified ratio. More...
template<typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Ordinary feed forward pass of the alpha_dropout layer. More...
template<typename eT >
void	Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Ordinary feed backward pass of the alpha_dropout layer. More...
OutputDataType const &	OutputParameter () const
	Get the output parameter.
OutputDataType &	OutputParameter ()
	Modify the output parameter.
OutputDataType const &	Delta () const
	Get the detla.
OutputDataType &	Delta ()
	Modify the delta.
bool	Deterministic () const
	The value of the deterministic parameter.
bool &	Deterministic ()
	Modify the value of the deterministic parameter.
double	Ratio () const
	The probability of setting a value to alphaDash.
double	A () const
	Value to be multiplied with x for affine transformation.
double	B () const
	Value to be added to a*x for affine transformation.
double	AlphaDash () const
	Value of alphaDash.
OutputDataType const &	Mask () const
	Get the mask.
void	Ratio (const double r)
	Modify the probability of setting a value to alphaDash. More...
template<typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer.

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::AlphaDropout< InputDataType, OutputDataType >

The alpha - dropout layer is a regularizer that randomly with probability 'ratio' sets input values to alphaDash.

The alpha - dropout layer is mostly used for SELU activation function where successive layers don't have same mean and variance.

For more information, see the following.

@article{Klambauer2017,
  author  = {Gunter Klambauer and Thomas Unterthiner and
             Andreas Mayr},
  title   = {Self-Normalizing Neural Networks},
  journal = {Advances in Neural Information Processing Systems},
  year    = {2017},
  url     = {https://arxiv.org/abs/1706.02515}
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Constructor & Destructor Documentation

AlphaDropout()

template<typename InputDataType , typename OutputDataType >

mlpack::ann::AlphaDropout< InputDataType, OutputDataType >::AlphaDropout	(	const double	ratio = 0.5,
		const double	alphaDash = -alpha * lambda
	)

Create the Alpha_Dropout object using the specified ratio.

Parameters

ratio	The probability of setting a value to alphaDash.
alphaDash	The dropout scaling parameter.

Member Function Documentation

Backward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::AlphaDropout< InputDataType, OutputDataType >::Backward	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Ordinary feed backward pass of the alpha_dropout layer.

Parameters

*	(input) The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Forward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::AlphaDropout< InputDataType, OutputDataType >::Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Ordinary feed forward pass of the alpha_dropout layer.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Ratio()

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

void mlpack::ann::AlphaDropout< InputDataType, OutputDataType >::Ratio ( const double  r )

inline

Modify the probability of setting a value to alphaDash.

As 'a' and 'b' depend on 'ratio', modify them as well.

---

The documentation for this class was generated from the following files:

• src/mlpack/methods/ann/layer/alpha_dropout.hpp
• src/mlpack/methods/ann/layer/alpha_dropout_impl.hpp