mlpack
Public Types | Public Member Functions | List of all members
mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType > Class Template Reference

The implementation of the RBM module. More...

#include <rbm.hpp>

Public Types

using NetworkType = RBM< InitializationRuleType, DataType, PolicyType >
 
typedef DataType::elem_type ElemType
 

Public Member Functions

 RBM (arma::Mat< ElemType > predictors, InitializationRuleType initializeRule, const size_t visibleSize, const size_t hiddenSize, const size_t batchSize=1, const size_t numSteps=1, const size_t negSteps=1, const size_t poolSize=2, const ElemType slabPenalty=8, const ElemType radius=1, const bool persistence=false)
 Initialize all the parameters of the network using initializeRule. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, BinaryRBM >::value, void >::type Reset ()
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type Reset ()
 
template<typename OptimizerType , typename... CallbackType>
double Train (OptimizerType &optimizer, CallbackType &&... callbacks)
 Train the RBM on the given input data. More...
 
double Evaluate (const arma::Mat< ElemType > &parameters, const size_t i, const size_t batchSize)
 Evaluate the RBM network with the given parameters. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, BinaryRBM >::value, double >::type FreeEnergy (const arma::Mat< ElemType > &input)
 This function calculates the free energy of the BinaryRBM. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, double >::type FreeEnergy (const arma::Mat< ElemType > &input)
 This function calculates the free energy of the SpikeSlabRBM. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, BinaryRBM >::value, void >::type Phase (const InputType &input, DataType &gradient)
 Calculates the gradient of the RBM network on the provided input. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type Phase (const InputType &input, DataType &gradient)
 Calculates the gradient of the RBM network on the provided input. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, BinaryRBM >::value, void >::type SampleHidden (const arma::Mat< ElemType > &input, arma::Mat< ElemType > &output)
 This function samples the hidden layer given the visible layer using Bernoulli function. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type SampleHidden (const arma::Mat< ElemType > &input, arma::Mat< ElemType > &output)
 This function samples the slab outputs from the Normal distribution with mean given by: \( h_i*\alpha^{-1}*W_i^T*v \) and variance: \( \alpha^{-1} \). More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, BinaryRBM >::value, void >::type SampleVisible (arma::Mat< ElemType > &input, arma::Mat< ElemType > &output)
 This function samples the visible layer given the hidden layer using Bernoulli function. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type SampleVisible (arma::Mat< ElemType > &input, arma::Mat< ElemType > &output)
 Sample Hidden function samples the slab outputs from the Normal distribution with mean given by: \( h_i*\alpha^{-1}*W_i^T*v \) and variance: \( \alpha^{-1} \). More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, BinaryRBM >::value, void >::type VisibleMean (InputType &input, DataType &output)
 The function calculates the mean for the visible layer. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type VisibleMean (InputType &input, DataType &output)
 The function calculates the mean of the Normal distribution of P(v|s, h). More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, BinaryRBM >::value, void >::type HiddenMean (const InputType &input, DataType &output)
 The function calculates the mean for the hidden layer. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type HiddenMean (const InputType &input, DataType &output)
 The function calculates the mean of the Normal distribution of P(s|v, h). More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type SpikeMean (const InputType &visible, DataType &spikeMean)
 The function calculates the mean of the distribution P(h|v), where mean is given by: \( sigm(v^T*W_i*\alpha_i^{-1}*W_i^T*v + b_i) \). More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type SampleSpike (InputType &spikeMean, DataType &spike)
 The function samples the spike function using Bernoulli distribution. More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type SlabMean (const DataType &visible, DataType &spike, DataType &slabMean)
 The function calculates the mean of Normal distribution of P(s|v, h), where the mean is given by: \( h_i*\alpha^{-1}*W_i^T*v \). More...
 
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type SampleSlab (InputType &slabMean, DataType &slab)
 The function samples from the Normal distribution of P(s|v, h), where the mean is given by: \( h_i*\alpha^{-1}*W_i^T*v \) and variance is given by: \( \alpha^{-1} \). More...
 
void Gibbs (const arma::Mat< ElemType > &input, arma::Mat< ElemType > &output, const size_t steps=SIZE_MAX)
 This function does the k-step Gibbs Sampling. More...
 
void Gradient (const arma::Mat< ElemType > &parameters, const size_t i, arma::Mat< ElemType > &gradient, const size_t batchSize)
 Calculates the gradients for the RBM network. More...
 
void Shuffle ()
 Shuffle the order of function visitation. More...
 
size_t NumFunctions () const
 Return the number of separable functions (the number of predictor points).
 
size_t NumSteps () const
 Return the number of steps of Gibbs Sampling.
 
const arma::Mat< ElemType > & Parameters () const
 Return the parameters of the network.
 
arma::Mat< ElemType > & Parameters ()
 Modify the parameters of the network.
 
arma::Cube< ElemType > const & Weight () const
 Get the weights of the network.
 
arma::Cube< ElemType > & Weight ()
 Modify the weights of the network.
 
DataType const & VisibleBias () const
 Return the visible bias of the network.
 
DataType & VisibleBias ()
 Modify the visible bias of the network.
 
DataType const & HiddenBias () const
 Return the hidden bias of the network.
 
DataType & HiddenBias ()
 Modify the hidden bias of the network.
 
DataType const & SpikeBias () const
 Get the regularizer associated with spike variables.
 
DataType & SpikeBias ()
 Modify the regularizer associated with spike variables.
 
ElemType const & SlabPenalty () const
 Get the regularizer associated with slab variables.
 
DataType const & VisiblePenalty () const
 Get the regularizer associated with visible variables.
 
DataType & VisiblePenalty ()
 Modify the regularizer associated with visible variables.
 
size_t const & VisibleSize () const
 Get the visible size.
 
size_t const & HiddenSize () const
 Get the hidden size.
 
size_t const & PoolSize () const
 Get the pool size.
 
template<typename Archive >
void serialize (Archive &ar, const uint32_t version)
 Serialize the model.
 

Detailed Description

template<typename InitializationRuleType, typename DataType = arma::mat, typename PolicyType = BinaryRBM>
class mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >

The implementation of the RBM module.

A Restricted Boltzmann Machines (RBM) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs. RBMs have found applications in dimensionality reduction, classification, collaborative filtering, feature learning and topic modelling. They can be trained in either supervised or unsupervised ways, depending on the task. They are a variant of Boltzmann machines, with the restriction that the neurons must form a bipartite graph.

Template Parameters
InitializationRuleTypeRule used to initialize the network.
DataTypeThe type of matrix to be used.
PolicyTypeThe RBM variant to be used (BinaryRBM or SpikeSlabRBM).

Constructor & Destructor Documentation

◆ RBM()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::RBM ( arma::Mat< ElemType >  predictors,
InitializationRuleType  initializeRule,
const size_t  visibleSize,
const size_t  hiddenSize,
const size_t  batchSize = 1,
const size_t  numSteps = 1,
const size_t  negSteps = 1,
const size_t  poolSize = 2,
const ElemType  slabPenalty = 8,
const ElemType  radius = 1,
const bool  persistence = false 
)

Initialize all the parameters of the network using initializeRule.

Parameters
predictorsTraining data to be used.
initializeRuleInitializationRule object for initializing the network parameter.
visibleSizeNumber of visible neurons.
hiddenSizeNumber of hidden neurons.
batchSizeBatch size to be used for training.
numStepsNumber of Gibbs Sampling steps.
negStepsNumber of negative samples to average negative gradient.
poolSizeNumber of hidden neurons to pool together.
slabPenaltyRegulariser of slab variables.
radiusFeasible regions for visible layer samples.
persistenceIndicates whether to use Persistent CD or not.

Member Function Documentation

◆ Evaluate()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
double mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::Evaluate ( const arma::Mat< ElemType > &  parameters,
const size_t  i,
const size_t  batchSize 
)

Evaluate the RBM network with the given parameters.

The function is needed for monitoring the progress of the network.

Parameters
parametersMatrix model parameters.
iIndex of the data point.
batchSizeVariable to store the present number of inputs.

◆ FreeEnergy() [1/2]

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, double >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::FreeEnergy ( const arma::Mat< ElemType > &  input)

This function calculates the free energy of the BinaryRBM.

The free energy is given by: \( -b^Tv - \sum_{i=1}^M log(1 + e^{c_j+v^TW_j}) \).

Parameters
inputThe visible neurons.

◆ FreeEnergy() [2/2]

template<typename InitializationRuleType , typename DataType = arma::mat, typename PolicyType = BinaryRBM>
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if<std::is_same<Policy, SpikeSlabRBM>::value, double>::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::FreeEnergy ( const arma::Mat< ElemType > &  input)

This function calculates the free energy of the SpikeSlabRBM.

The free energy is given by: \( v^t$$\Delta$v - $\sum_{i=1}^N \) \( \log{ \sqrt{\frac{(-2\pi)^K}{\prod_{m=1}^{K}(\alpha_i)_m}}} \) - \( \sum_{i=1}^N \log(1+\exp( b_i + \sum_{m=1}^k \frac{(v(w_i)_m^t)^2}{2(\alpha_i)_m}) \)

Parameters
inputThe visible layer neurons.

◆ Gibbs()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
void mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::Gibbs ( const arma::Mat< ElemType > &  input,
arma::Mat< ElemType > &  output,
const size_t  steps = SIZE_MAX 
)

This function does the k-step Gibbs Sampling.

Parameters
inputInput to the Gibbs function.
outputUsed for storing the negative sample.
stepsNumber of Gibbs Sampling steps taken.

◆ Gradient()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
void mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::Gradient ( const arma::Mat< ElemType > &  parameters,
const size_t  i,
arma::Mat< ElemType > &  gradient,
const size_t  batchSize 
)

Calculates the gradients for the RBM network.

Parameters
parametersThe current parameters of the network.
iIndex of the data point.
gradientVariable to store the present gradient.
batchSizeVariable to store the present number of inputs.

◆ HiddenMean() [1/2]

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::HiddenMean ( const InputType &  input,
DataType &  output 
)

The function calculates the mean for the hidden layer.

Parameters
inputVisible neurons.
outputHidden neuron activations.

◆ HiddenMean() [2/2]

template<typename InitializationRuleType , typename DataType = arma::mat, typename PolicyType = BinaryRBM>
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if<std::is_same<Policy, SpikeSlabRBM>::value, void>::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::HiddenMean ( const InputType &  input,
DataType &  output 
)

The function calculates the mean of the Normal distribution of P(s|v, h).

The mean is given by: \( h_i*\alpha^{-1}*W_i^T*v \) The variance is given by: \( \alpha^{-1} \)

Parameters
inputVisible layer neurons.
outputConsists of both the spike samples and slab samples.

◆ Phase() [1/2]

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::Phase ( const InputType &  input,
DataType &  gradient 
)

Calculates the gradient of the RBM network on the provided input.

Parameters
inputThe provided input data.
gradientStores the gradient of the RBM network.

◆ Phase() [2/2]

template<typename InitializationRuleType , typename DataType = arma::mat, typename PolicyType = BinaryRBM>
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if<std::is_same<Policy, SpikeSlabRBM>::value, void>::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::Phase ( const InputType &  input,
DataType &  gradient 
)

Calculates the gradient of the RBM network on the provided input.

Parameters
inputThe provided input data.
gradientStores the gradient of the RBM network.

◆ SampleHidden() [1/2]

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SampleHidden ( const arma::Mat< ElemType > &  input,
arma::Mat< ElemType > &  output 
)

This function samples the hidden layer given the visible layer using Bernoulli function.

Parameters
inputVisible layer input.
outputThe sampled hidden layer.

◆ SampleHidden() [2/2]

template<typename InitializationRuleType , typename DataType = arma::mat, typename PolicyType = BinaryRBM>
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if<std::is_same<Policy, SpikeSlabRBM>::value, void>::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SampleHidden ( const arma::Mat< ElemType > &  input,
arma::Mat< ElemType > &  output 
)

This function samples the slab outputs from the Normal distribution with mean given by: \( h_i*\alpha^{-1}*W_i^T*v \) and variance: \( \alpha^{-1} \).

Parameters
inputConsists of both visible and spike variables.
outputSampled slab neurons.

◆ SampleSlab()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SampleSlab ( InputType &  slabMean,
DataType &  slab 
)

The function samples from the Normal distribution of P(s|v, h), where the mean is given by: \( h_i*\alpha^{-1}*W_i^T*v \) and variance is given by: \( \alpha^{-1} \).

Parameters
slabMeanMean of the Normal distribution of the slab neurons.
slabSampled slab variable from the Normal distribution.

◆ SampleSpike()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SampleSpike ( InputType &  spikeMean,
DataType &  spike 
)

The function samples the spike function using Bernoulli distribution.

Parameters
spikeMeanIndicates P(h|v).
spikeSampled binary spike variables.

◆ SampleVisible() [1/2]

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SampleVisible ( arma::Mat< ElemType > &  input,
arma::Mat< ElemType > &  output 
)

This function samples the visible layer given the hidden layer using Bernoulli function.

Parameters
inputHidden layer of the network.
outputThe sampled visible layer.

◆ SampleVisible() [2/2]

template<typename InitializationRuleType , typename DataType = arma::mat, typename PolicyType = BinaryRBM>
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if<std::is_same<Policy, SpikeSlabRBM>::value, void>::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SampleVisible ( arma::Mat< ElemType > &  input,
arma::Mat< ElemType > &  output 
)

Sample Hidden function samples the slab outputs from the Normal distribution with mean given by: \( h_i*\alpha^{-1}*W_i^T*v \) and variance: \( \alpha^{-1} \).

Parameters
inputHidden layer of the network.
outputThe sampled visible layer.

◆ Shuffle()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
void mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::Shuffle ( )

Shuffle the order of function visitation.

This may be called by the optimizer.

◆ SlabMean()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SlabMean ( const DataType &  visible,
DataType &  spike,
DataType &  slabMean 
)

The function calculates the mean of Normal distribution of P(s|v, h), where the mean is given by: \( h_i*\alpha^{-1}*W_i^T*v \).

Parameters
visibleThe visible layer neurons.
spikeThe spike variables from hidden layer.
slabMeanThe mean of the Normal distribution of slab neurons.

◆ SpikeMean()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::SpikeMean ( const InputType &  visible,
DataType &  spikeMean 
)

The function calculates the mean of the distribution P(h|v), where mean is given by: \( sigm(v^T*W_i*\alpha_i^{-1}*W_i^T*v + b_i) \).

Parameters
visibleThe visible layer neurons.
spikeMeanIndicates P(h|v).

◆ Train()

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename OptimizerType , typename... CallbackType>
double mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::Train ( OptimizerType &  optimizer,
CallbackType &&...  callbacks 
)

Train the RBM on the given input data.

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

Template Parameters
OptimizerTypeType of optimizer to use to train the model.
CallbackTypesTypes of Callback functions.
Parameters
optimizerOptimizer type.
callbacksCallback Functions for ensmallen optimizer OptimizerType. See https://www.ensmallen.org/docs.html#callback-documentation.
Returns
The final objective of the trained model (NaN or Inf on error).

◆ VisibleMean() [1/2]

template<typename InitializationRuleType , typename DataType , typename PolicyType >
template<typename Policy , typename InputType >
std::enable_if< std::is_same< Policy, SpikeSlabRBM >::value, void >::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::VisibleMean ( InputType &  input,
DataType &  output 
)

The function calculates the mean for the visible layer.

Parameters
inputHidden neurons from the hidden layer of the network.
outputVisible neuron activations.

◆ VisibleMean() [2/2]

template<typename InitializationRuleType , typename DataType = arma::mat, typename PolicyType = BinaryRBM>
template<typename Policy = PolicyType, typename InputType = DataType>
std::enable_if<std::is_same<Policy, SpikeSlabRBM>::value, void>::type mlpack::ann::RBM< InitializationRuleType, DataType, PolicyType >::VisibleMean ( InputType &  input,
DataType &  output 
)

The function calculates the mean of the Normal distribution of P(v|s, h).

The mean is given by: \( \Lambda^{-1} \sum_{i=1}^N W_i * s_i * h_i \)

Parameters
inputConsists of both the spike and slab variables.
outputMean of the of the Normal distribution.

The documentation for this class was generated from the following files: