mlpack::ann::DropConnect< InputDataType, OutputDataType > Class Template Reference

The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio). More...

#include <dropconnect.hpp>

Public Member Functions
	DropConnect ()
	Create the DropConnect object.
	DropConnect (const size_t inSize, const size_t outSize, const double ratio=0.5)
	Creates the DropConnect Layer as a Linear Object that takes input size, output size and ratio as parameter. More...
template<typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Ordinary feed forward pass of the DropConnect layer. More...
template<typename eT >
void	Backward (const arma::Mat< eT > &input, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Ordinary feed backward pass of the DropConnect layer. More...
template<typename eT >
void	Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &)
	Calculate the gradient using the output delta and the input activation. More...
std::vector< LayerTypes<> > &	Model ()
	Get the model modules.
OutputDataType const &	Parameters () const
	Get the parameters.
OutputDataType &	Parameters ()
	Modify the parameters.
OutputDataType const &	OutputParameter () const
	Get the output parameter.
OutputDataType &	OutputParameter ()
	Modify the output parameter.
OutputDataType const &	Delta () const
	Get the delta.
OutputDataType &	Delta ()
	Modify the delta.
OutputDataType const &	Gradient () const
	Get the gradient.
OutputDataType &	Gradient ()
	Modify the gradient.
bool	Deterministic () const
	The value of the deterministic parameter.
bool &	Deterministic ()
	Modify the value of the deterministic parameter.
double	Ratio () const
	The probability of setting a value to zero.
void	Ratio (const double r)
	Modify the probability of setting a value to zero.
size_t	WeightSize () const
	Return the size of the weight matrix.
template<typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer.

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::DropConnect< InputDataType, OutputDataType >

The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio).

The output is scaled with 1 / (1 - p) when deterministic is false. In the deterministic mode(during testing), the layer just computes the output. The output is computed according to the input layer. If no input layer is given, it will take a linear layer as default.

Note: During training you should set deterministic to false and during testing you should set deterministic to true.

For more information, see the following.

@inproceedings{WanICML2013,
  title={Regularization of Neural Networks using DropConnect},
  booktitle = {Proceedings of the 30th International Conference on Machine
               Learning(ICML - 13)},
  author = {Li Wan and Matthew Zeiler and Sixin Zhang and Yann L. Cun and
            Rob Fergus},
  year = {2013},
  url  = {http://proceedings.mlr.press/v28/wan13.pdf}
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Constructor & Destructor Documentation

DropConnect()

template<typename InputDataType , typename OutputDataType >

mlpack::ann::DropConnect< InputDataType, OutputDataType >::DropConnect	(	const size_t	inSize,
		const size_t	outSize,
		const double	ratio = 0.5
	)

Creates the DropConnect Layer as a Linear Object that takes input size, output size and ratio as parameter.

Parameters

inSize	The number of input units.
outSize	The number of output units.
ratio	The probability of setting a value to zero.

Member Function Documentation

Backward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::DropConnect< InputDataType, OutputDataType >::Backward	(	const arma::Mat< eT > &	input,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Ordinary feed backward pass of the DropConnect layer.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Forward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::DropConnect< InputDataType, OutputDataType >::Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Ordinary feed forward pass of the DropConnect layer.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Gradient()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::DropConnect< InputDataType, OutputDataType >::Gradient	(	const arma::Mat< eT > &	input,
		const arma::Mat< eT > &	error,
		arma::Mat< eT > &
	)

Calculate the gradient using the output delta and the input activation.

Parameters

input	The propagated input.
error	The calculated error.
*	(gradient) The calculated gradient.

---

The documentation for this class was generated from the following files:

• src/mlpack/methods/ann/layer/dropconnect.hpp
• src/mlpack/methods/ann/layer/dropconnect_impl.hpp