mlpack::nn::SparseAutoencoderFunction Class Reference

This is a class for the sparse autoencoder objective function. More...

#include <sparse_autoencoder_function.hpp>

Public Member Functions
	SparseAutoencoderFunction (const arma::mat &data, const size_t visibleSize, const size_t hiddenSize, const double lambda=0.0001, const double beta=3, const double rho=0.01)
	Construct the sparse autoencoder objective function with the given parameters. More...
const arma::mat	InitializeWeights ()
	Initializes the parameters of the model to suitable values. More...
double	Evaluate (const arma::mat &parameters) const
	Evaluates the objective function of the sparse autoencoder model using the given parameters. More...
void	Gradient (const arma::mat &parameters, arma::mat &gradient) const
	Evaluates the gradient values of the objective function given the current set of parameters. More...
void	Sigmoid (const arma::mat &x, arma::mat &output) const
	Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))]. More...
const arma::mat &	GetInitialPoint () const
	Return the initial point for the optimization.
void	VisibleSize (const size_t visible)
	Sets size of the visible layer.
size_t	VisibleSize () const
	Gets size of the visible layer.
void	HiddenSize (const size_t hidden)
	Sets size of the hidden layer.
size_t	HiddenSize () const
	Gets the size of the hidden layer.
void	Lambda (const double l)
	Sets the L2-regularization parameter.
double	Lambda () const
	Gets the L2-regularization parameter.
void	Beta (const double b)
	Sets the KL divergence parameter.
double	Beta () const
	Gets the KL divergence parameter.
void	Rho (const double r)
	Sets the sparsity parameter.
double	Rho () const
	Gets the sparsity parameter.

Detailed Description

This is a class for the sparse autoencoder objective function.

It can be used to create learning models like self-taught learning, stacked autoencoders, conditional random fields (CRFs), and so forth.

Constructor & Destructor Documentation

SparseAutoencoderFunction()

SparseAutoencoderFunction::SparseAutoencoderFunction	(	const arma::mat &	data,
		const size_t	visibleSize,
		const size_t	hiddenSize,
		const double	lambda = 0.0001,
		const double	beta = 3,
		const double	rho = 0.01
	)

Construct the sparse autoencoder objective function with the given parameters.

Parameters

data	The data matrix.
visibleSize	Size of input vector expected at the visible layer.
hiddenSize	Size of input vector expected at the hidden layer.
lambda	L2-regularization parameter.
beta	KL divergence parameter.
rho	Sparsity parameter.

Member Function Documentation

Evaluate()

double SparseAutoencoderFunction::Evaluate

(

const arma::mat &

parameters

)

const

Evaluates the objective function of the sparse autoencoder model using the given parameters.

Evaluates the objective function given the parameters.

The cost function has terms for the reconstruction error, regularization cost and the sparsity cost. The objective function takes a low value when the model is able to reconstruct the data well using weights which are low in value and when the average activations of neurons in the hidden layers agrees well with the sparsity parameter 'rho'.

Parameters

parameters

Current values of the model parameters.

Gradient()

void SparseAutoencoderFunction::Gradient	(	const arma::mat &	parameters,
		arma::mat &	gradient
	)		const

Evaluates the gradient values of the objective function given the current set of parameters.

Calculates and stores the gradient values given a set of parameters.

The function performs a feedforward pass and computes the error in reconstructing the data points. It then uses the backpropagation algorithm to compute the gradient values.

Parameters

parameters	Current values of the model parameters.
gradient	Matrix where gradient values will be stored.

InitializeWeights()

const arma::mat SparseAutoencoderFunction::InitializeWeights

(

)

Initializes the parameters of the model to suitable values.

Initializes the parameter weights if the initial point is not passed to the constructor.

The weights w1, w2 are initialized to randomly in the range [-r, r] where 'r' is decided using the sizes of the visible and hidden layers. The biases b1, b2 are initialized to 0.

Sigmoid()

void mlpack::nn::SparseAutoencoderFunction::Sigmoid ( const arma::mat &  x, arma::mat &  output  ) const

inline

Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))].

Parameters

x	Matrix of real values for which we require the sigmoid activation.
output	Output matrix.

---

The documentation for this class was generated from the following files:

• src/mlpack/methods/sparse_autoencoder/sparse_autoencoder_function.hpp
• src/mlpack/methods/sparse_autoencoder/sparse_autoencoder_function.cpp