Implementation of Double Pole Cart Balancing task. More...

#include <double_pole_cart.hpp>

Classes
class	Action
	Implementation of action of Double Pole Cart. More...

class	State
	Implementation of the state of Double Pole Cart. More...

Public Member Functions
	DoublePoleCart (const size_t maxSteps=0, const double m1=0.1, const double m2=0.01, const double l1=0.5, const double l2=0.05, const double gravity=9.8, const double massCart=1.0, const double forceMag=10.0, const double tau=0.02, const double thetaThresholdRadians=36 2 3.1416/360, const double xThreshold=2.4, const double doneReward=0.0)
	Construct a Double Pole Cart instance using the given constants. More...

double	Sample (const State &state, const Action &action, State &nextState)
	Dynamics of Double Pole Cart instance. More...

void	Dsdt (const State &state, const Action &action, arma::vec &dydx)
	This is the ordinary differential equations required for estimation of next state through RK4 method. More...

void	RK4 (const State &state, const Action &action, arma::vec &dydx, State &nextState)
	This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation. More...

double	Sample (const State &state, const Action &action)
	Dynamics of Double Pole Cart. More...

State	InitialSample ()
	Initial state representation is randomly generated within [-0.05, 0.05]. More...

bool	IsTerminal (const State &state) const
	This function checks if the car has reached the terminal state. More...

size_t	StepsPerformed () const
	Get the number of steps performed.

size_t	MaxSteps () const
	Get the maximum number of steps allowed.

size_t &	MaxSteps ()
	Set the maximum number of steps allowed.

Detailed Description

Implementation of Double Pole Cart Balancing task.

This is an extension of the existing CartPole environment. The environment comprises of a cart of a cart with two poles of different lengths and masses. The agent is meant to balance the poles by applying force on the cart.

Constructor & Destructor Documentation

◆ DoublePoleCart()

mlpack::rl::DoublePoleCart::DoublePoleCart	(	const size_t	maxSteps = `0`,
		const double	m1 = `0.1`,
		const double	m2 = `0.01`,
		const double	l1 = `0.5`,
		const double	l2 = `0.05`,
		const double	gravity = `9.8`,
		const double	massCart = `1.0`,
		const double	forceMag = `10.0`,
		const double	tau = `0.02`,
		const double	thetaThresholdRadians = `36 * 2 * 3.1416 / 360`,
		const double	xThreshold = `2.4`,
		const double	doneReward = `0.0`
	)

inline

Construct a Double Pole Cart instance using the given constants.

Parameters

maxSteps	The number of steps after which the episode terminates. If the value is 0, there is no limit.
m1	The mass of the first pole.
m2	The mass of the second pole.
l1	The length of the first pole.
l2	The length of the second pole.
gravity	The gravity constant.
massCart	The mass of the cart.
forceMag	The magnitude of the applied force.
tau	The time interval.
thetaThresholdRadians	The maximum angle.
xThreshold	The maximum position.
doneReward	Reward recieved by agent on success.

Member Function Documentation

◆ Dsdt()

void mlpack::rl::DoublePoleCart::Dsdt	(	const State &	state,
		const Action &	action,
		arma::vec &	dydx
	)

inline

This is the ordinary differential equations required for estimation of next state through RK4 method.

Parameters

state	The current state.
action	The action taken.
dydx	The differential.

◆ InitialSample()

State mlpack::rl::DoublePoleCart::InitialSample ( )

inline

Initial state representation is randomly generated within [-0.05, 0.05].

Returns: Initial state for each episode.

◆ IsTerminal()

bool mlpack::rl::DoublePoleCart::IsTerminal ( const State & state ) const

inline

This function checks if the car has reached the terminal state.

Parameters

state The desired state.

Returns: true if state is a terminal state, otherwise false.

◆ RK4()

void mlpack::rl::DoublePoleCart::RK4	(	const State &	state,
		const Action &	action,
		arma::vec &	dydx,
		State &	nextState
	)

inline

This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation.

Parameters

state	The current state.
action	The action to be applied.
dydx	The differential.
nextState	The next state.

◆ Sample() [1/2]

double mlpack::rl::DoublePoleCart::Sample	(	const State &	state,
		const Action &	action,
		State &	nextState
	)

inline

Dynamics of Double Pole Cart instance.

Get reward and next state based on current state and current action.

Parameters

state	The current state.
action	The current action.
nextState	The next state.

Returns: reward, it's always 1.0.

When done is false, it means that the cartpole has fallen down. For this case the reward is 1.0.

◆ Sample() [2/2]

double mlpack::rl::DoublePoleCart::Sample	(	const State &	state,
		const Action &	action
	)

inline

Dynamics of Double Pole Cart.

Get reward based on current state and current action.

Parameters

state	The current state.
action	The current action.

Returns: reward, it's always 1.0.

The documentation for this class was generated from the following file:

src/mlpack/methods/reinforcement_learning/environment/double_pole_cart.hpp

Classes

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ DoublePoleCart()

Member Function Documentation

◆ Dsdt()

◆ InitialSample()

◆ IsTerminal()

◆ RK4()

◆ Sample() [1/2]

◆ Sample() [2/2]