Implementation of the Dueling Deep Q-Learning network. More...

#include <dueling_dqn.hpp>

Public Member Functions
	DuelingDQN ()
	Default constructor.

	DuelingDQN (const int inputDim, const int h1, const int h2, const int outputDim, const bool isNoisy=false, InitType init=InitType(), OutputLayerType outputLayer=OutputLayerType())
	Construct an instance of DuelingDQN class. More...

	DuelingDQN (FeatureNetworkType &featureNetwork, AdvantageNetworkType &advantageNetwork, ValueNetworkType &valueNetwork, const bool isNoisy=false)
	Construct an instance of DuelingDQN class from a pre-constructed network. More...

	DuelingDQN (const DuelingDQN &)
	Copy constructor.

void	operator= (const DuelingDQN &model)
	Copy assignment operator.

void	Predict (const arma::mat state, arma::mat &actionValue)
	Predict the responses to a given set of predictors. More...

void	Forward (const arma::mat state, arma::mat &actionValue)
	Perform the forward pass of the states in real batch mode. More...

void	Backward (const arma::mat state, arma::mat &target, arma::mat &gradient)
	Perform the backward pass of the state in real batch mode. More...

void	ResetParameters ()
	Resets the parameters of the network.

void	ResetNoise ()
	Resets noise of the network, if the network is of type noisy.

const arma::mat &	Parameters () const
	Return the Parameters.

arma::mat &	Parameters ()
	Modify the Parameters.

Detailed Description

template<typename OutputLayerType = EmptyLoss<>, typename InitType = GaussianInitialization, typename CompleteNetworkType = FFN<OutputLayerType, InitType>, typename FeatureNetworkType = Sequential<>, typename AdvantageNetworkType = Sequential<>, typename ValueNetworkType = Sequential<>>
class mlpack::rl::DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >

Implementation of the Dueling Deep Q-Learning network.

For more information, see the following.

@misc{wang2015dueling,
  author  = {Ziyu Wang, Tom Schaul, Matteo Hessel,Hado van Hasselt,
             Marc Lanctot, Nando de Freitas},
  title   = {Dueling Network Architectures for Deep Reinforcement Learning},
  year    = {2015},
  url     = {https://arxiv.org/abs/1511.06581}
}

Template Parameters

OutputLayerType	The output layer type of the network.
InitType	The initialization type used for the network.
CompleteNetworkType	The type of network used for full dueling dqn.
FeatureNetworkType	The type of network used for feature network.
AdvantageNetworkType	The type of network used for advantage network.
ValueNetworkType	The type of network used for value network.

Constructor & Destructor Documentation

◆ DuelingDQN() [1/2]

template<typename OutputLayerType = EmptyLoss<>, typename InitType = GaussianInitialization, typename CompleteNetworkType = FFN<OutputLayerType, InitType>, typename FeatureNetworkType = Sequential<>, typename AdvantageNetworkType = Sequential<>, typename ValueNetworkType = Sequential<>>

mlpack::rl::DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >::DuelingDQN	(	const int	inputDim,
		const int	h1,
		const int	h2,
		const int	outputDim,
		const bool	isNoisy = `false`,
		InitType	init = `InitType()`,
		OutputLayerType	outputLayer = `OutputLayerType()`
	)

inline

Construct an instance of DuelingDQN class.

Parameters

inputDim	Number of inputs.
h1	Number of neurons in hiddenlayer-1.
h2	Number of neurons in hiddenlayer-2.
outputDim	Number of neurons in output layer.
isNoisy	Specifies whether the network needs to be of type noisy.
init	Specifies the initialization rule for the network.
outputLayer	Specifies the output layer type for network.

◆ DuelingDQN() [2/2]

template<typename OutputLayerType = EmptyLoss<>, typename InitType = GaussianInitialization, typename CompleteNetworkType = FFN<OutputLayerType, InitType>, typename FeatureNetworkType = Sequential<>, typename AdvantageNetworkType = Sequential<>, typename ValueNetworkType = Sequential<>>

mlpack::rl::DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >::DuelingDQN	(	FeatureNetworkType &	featureNetwork,
		AdvantageNetworkType &	advantageNetwork,
		ValueNetworkType &	valueNetwork,
		const bool	isNoisy = `false`
	)

inline

Construct an instance of DuelingDQN class from a pre-constructed network.

Parameters

featureNetwork	The feature network to be used by DuelingDQN class.
advantageNetwork	The advantage network to be used by DuelingDQN class.
valueNetwork	The value network to be used by DuelingDQN class.
isNoisy	Specifies whether the network needs to be of type noisy.

Member Function Documentation

◆ Backward()

template<typename OutputLayerType = EmptyLoss<>, typename InitType = GaussianInitialization, typename CompleteNetworkType = FFN<OutputLayerType, InitType>, typename FeatureNetworkType = Sequential<>, typename AdvantageNetworkType = Sequential<>, typename ValueNetworkType = Sequential<>>

void mlpack::rl::DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >::Backward	(	const arma::mat	state,
		arma::mat &	target,
		arma::mat &	gradient
	)

inline

Perform the backward pass of the state in real batch mode.

Parameters

state	The input state.
target	The training target.
gradient	The gradient.

◆ Forward()

template<typename OutputLayerType = EmptyLoss<>, typename InitType = GaussianInitialization, typename CompleteNetworkType = FFN<OutputLayerType, InitType>, typename FeatureNetworkType = Sequential<>, typename AdvantageNetworkType = Sequential<>, typename ValueNetworkType = Sequential<>>

void mlpack::rl::DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >::Forward	(	const arma::mat	state,
		arma::mat &	actionValue
	)

inline

Perform the forward pass of the states in real batch mode.

Parameters

state	The input state.
actionValue	Matrix to put output action values of states input.

◆ Predict()

template<typename OutputLayerType = EmptyLoss<>, typename InitType = GaussianInitialization, typename CompleteNetworkType = FFN<OutputLayerType, InitType>, typename FeatureNetworkType = Sequential<>, typename AdvantageNetworkType = Sequential<>, typename ValueNetworkType = Sequential<>>

void mlpack::rl::DuelingDQN< OutputLayerType, InitType, CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >::Predict	(	const arma::mat	state,
		arma::mat &	actionValue
	)

inline

Predict the responses to a given set of predictors.

The responses will reflect the output of the given output layer as returned by the output layer function.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters

state	Input state.
actionValue	Matrix to put output action values of states input.

The documentation for this class was generated from the following file:

src/mlpack/methods/reinforcement_learning/q_networks/dueling_dqn.hpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ DuelingDQN() [1/2]

◆ DuelingDQN() [2/2]

Member Function Documentation

◆ Backward()

◆ Forward()

◆ Predict()