mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType > Class Template Reference

Declaration of the VirtualBatchNorm layer class. More...

#include <virtual_batch_norm.hpp>

Public Member Functions
	VirtualBatchNorm ()
	Create the VirtualBatchNorm object. More...
template<typename eT >
	VirtualBatchNorm (const arma::Mat< eT > &referenceBatch, const size_t size, const double eps=1e-8)
	Create the VirtualBatchNorm layer object for a specified number of input units. More...
void	Reset ()
	Reset the layer parameters.
template<typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Forward pass of the Virtual Batch Normalization layer. More...
template<typename eT >
void	Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Backward pass through the layer. More...
template<typename eT >
void	Gradient (const arma::Mat< eT > &, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
	Calculate the gradient using the output delta and the input activations. More...
OutputDataType const &	Parameters () const
	Get the parameters.
OutputDataType &	Parameters ()
	Modify the parameters.
OutputDataType const &	OutputParameter () const
	Get the output parameter.
OutputDataType &	OutputParameter ()
	Modify the output parameter.
OutputDataType const &	Delta () const
	Get the delta.
OutputDataType &	Delta ()
	Modify the delta.
OutputDataType const &	Gradient () const
	Get the gradient.
OutputDataType &	Gradient ()
	Modify the gradient.
size_t	InSize () const
	Get the number of input units.
double	Epsilon () const
	Get the epsilon value.
template<typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer.

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >

Declaration of the VirtualBatchNorm layer class.

Instead of using the batch statistics for normalizing on a mini-batch, it uses a reference subset of the data for calculating the normalization statistics.

For more information, refer to the following paper,

@article{Goodfellow2016,
  author  = {Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung,
             Alec Radford, Xi Chen},
  title   = {Improved Techniques for Training GANs},
  year    = {2016},
  url     = {https://arxiv.org/abs/1606.03498},
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Constructor & Destructor Documentation

VirtualBatchNorm() [1/2]

template<typename InputDataType , typename OutputDataType >

mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >::VirtualBatchNorm

(

)

Create the VirtualBatchNorm object.

Artificial Neural Network.

VirtualBatchNorm() [2/2]

template<typename InputDataType , typename OutputDataType >

template<typename eT >

mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >::VirtualBatchNorm	(	const arma::Mat< eT > &	referenceBatch,
		const size_t	size,
		const double	eps = 1e-8
	)

Create the VirtualBatchNorm layer object for a specified number of input units.

Parameters

referenceBatch	The data from which the normalization statistics are computed.
size	The number of input units / channels.
eps	The epsilon added to variance to ensure numerical stability.

Member Function Documentation

Backward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >::Backward	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Backward pass through the layer.

Parameters

*	(input) The input activations.
gy	The backpropagated error.
g	The calculated gradient.

Forward()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >::Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Forward pass of the Virtual Batch Normalization layer.

Transforms the input data into zero mean and unit variance, scales the data by a factor gamma and shifts it by beta.

Parameters

input	Input data for the layer.
output	Resulting output activations.

Gradient()

template<typename InputDataType , typename OutputDataType >

template<typename eT >

void mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >::Gradient	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	error,
		arma::Mat< eT > &	gradient
	)

Calculate the gradient using the output delta and the input activations.

Parameters

*	(input) The input activations.
error	The calculated error.
gradient	The calculated gradient.

---

The documentation for this class was generated from the following files:

• src/mlpack/methods/ann/layer/layer_types.hpp
• src/mlpack/methods/ann/layer/virtual_batch_norm.hpp
• src/mlpack/methods/ann/layer/virtual_batch_norm_impl.hpp