The LogisticRegression class implements an L2-regularized logistic regression model, and supports training with multiple optimizers and classification. More...

#include <logistic_regression.hpp>

Public Member Functions
	LogisticRegression (const MatType &predictors, const arma::Row< size_t > &responses, const double lambda=0)
	Construct the LogisticRegression class with the given labeled training data. More...

	LogisticRegression (const MatType &predictors, const arma::Row< size_t > &responses, const arma::rowvec &initialPoint, const double lambda=0)
	Construct the LogisticRegression class with the given labeled training data. More...

	LogisticRegression (const size_t dimensionality=0, const double lambda=0)
	Construct the LogisticRegression class without performing any training. More...

template<typename OptimizerType >
	LogisticRegression (const MatType &predictors, const arma::Row< size_t > &responses, OptimizerType &optimizer, const double lambda)
	Construct the LogisticRegression class with the given labeled training data. More...

template<typename OptimizerType = ens::L_BFGS, typename... CallbackTypes>
double	Train (const MatType &predictors, const arma::Row< size_t > &responses, CallbackTypes &&... callbacks)
	Train the LogisticRegression model on the given input data. More...

template<typename OptimizerType , typename... CallbackTypes>
double	Train (const MatType &predictors, const arma::Row< size_t > &responses, OptimizerType &optimizer, CallbackTypes &&... callbacks)
	Train the LogisticRegression model with the given instantiated optimizer. More...

const arma::rowvec &	Parameters () const
	Return the parameters (the b vector).

arma::rowvec &	Parameters ()
	Modify the parameters (the b vector).

const double &	Lambda () const
	Return the lambda value for L2-regularization.

double &	Lambda ()
	Modify the lambda value for L2-regularization.

template<typename VecType >
size_t	Classify (const VecType &point, const double decisionBoundary=0.5) const
	Classify the given point. More...

void	Classify (const MatType &dataset, arma::Row< size_t > &labels, const double decisionBoundary=0.5) const
	Classify the given points, returning the predicted labels for each point. More...

void	Classify (const MatType &dataset, arma::mat &probabilities) const
	Classify the given points, returning class probabilities for each point. More...

double	ComputeAccuracy (const MatType &predictors, const arma::Row< size_t > &responses, const double decisionBoundary=0.5) const
	Compute the accuracy of the model on the given predictors and responses, optionally using the given decision boundary. More...

double	ComputeError (const MatType &predictors, const arma::Row< size_t > &responses) const
	Compute the error of the model. More...

template<typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the model.

Detailed Description

template<typename MatType = arma::mat>
class mlpack::regression::LogisticRegression< MatType >

The LogisticRegression class implements an L2-regularized logistic regression model, and supports training with multiple optimizers and classification.

The class supports different observation types via the MatType template parameter; for instance, logistic regression can be performed on sparse datasets by specifying arma::sp_mat as the MatType parameter.

LogisticRegression can be used for general classification tasks, but the class is restricted to support only two classes. For multiclass logistic regression, see mlpack::regression::SoftmaxRegression.

Template Parameters

MatType Type of data matrix.

Constructor & Destructor Documentation

◆ LogisticRegression() [1/4]

template<typename MatType >

mlpack::regression::LogisticRegression< MatType >::LogisticRegression	(	const MatType &	predictors,
		const arma::Row< size_t > &	responses,
		const double	lambda = `0`
	)

Construct the LogisticRegression class with the given labeled training data.

This will train the model. Optionally, specify lambda, which is the penalty parameter for L2-regularization. If not specified, it is set to 0, which results in standard (unregularized) logistic regression.

It is not possible to set a custom optimizer with this constructor. Either use a constructor that does not train and call Train() with a custom optimizer type, or use the constructor that takes an instantiated optimizer. (This unfortunate situation is a language restriction of C++.)

Parameters

predictors	Input training variables.
responses	Outputs resulting from input training variables.
lambda	L2-regularization parameter.

◆ LogisticRegression() [2/4]

template<typename MatType >

mlpack::regression::LogisticRegression< MatType >::LogisticRegression	(	const MatType &	predictors,
		const arma::Row< size_t > &	responses,
		const arma::rowvec &	initialPoint,
		const double	lambda = `0`
	)

Construct the LogisticRegression class with the given labeled training data.

This will train the model. Optionally, specify lambda, which is the penalty parameter for L2-regularization. If not specified, it is set to 0, which results in standard (unregularized) logistic regression.

It is not possible to set a custom optimizer with this constructor. Either use a constructor that does not train and call Train() with a custom optimizer type, or use the constructor that takes an instantiated optimizer. (This unfortunate situation is a language restriction of C++.)

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
initialPoint	Initial model to train with.
lambda	L2-regularization parameter.

◆ LogisticRegression() [3/4]

template<typename MatType >

mlpack::regression::LogisticRegression< MatType >::LogisticRegression	(	const size_t	dimensionality = `0`,
		const double	lambda = `0`
	)

Construct the LogisticRegression class without performing any training.

The dimensionality of the data (which will be used to set the size of the parameters vector) must be specified, and all of the parameters in the model will be set to 0. Note that the dimensionality may be changed later by directly modifying the parameters vector (using Parameters()).

Parameters

dimensionality	Dimensionality of the data.
lambda	L2-regularization parameter.

◆ LogisticRegression() [4/4]

template<typename MatType >

template<typename OptimizerType >

mlpack::regression::LogisticRegression< MatType >::LogisticRegression	(	const MatType &	predictors,
		const arma::Row< size_t > &	responses,
		OptimizerType &	optimizer,
		const double	lambda
	)

Construct the LogisticRegression class with the given labeled training data.

This will train the model. This overload takes an already instantiated optimizer (which holds the LogisticRegressionFunction error function, which must also be instantiated), so that the optimizer can be configured before the training is run by this constructor. The update policy of the optimizer can be set through the policy argument. The predictors and responses and initial point are all taken from the error function contained in the optimizer.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
optimizer	Instantiated optimizer with instantiated error function.
lambda	L2-regularization parameter.

Member Function Documentation

◆ Classify() [1/3]

template<typename MatType >

template<typename VecType >

size_t mlpack::regression::LogisticRegression< MatType >::Classify	(	const VecType &	point,
		const double	decisionBoundary = `0.5`
	)		const

Classify the given point.

The predicted label is returned. Optionally, specify the decision boundary; logistic regression returns a value between 0 and 1. If the value is greater than the decision boundary, the response is taken to be 1; otherwise, it is 0. By default the decision boundary is 0.5.

Parameters

point	Point to classify.
decisionBoundary	Decision boundary (default 0.5).

Returns: Predicted label of point.

◆ Classify() [2/3]

template<typename MatType >

void mlpack::regression::LogisticRegression< MatType >::Classify	(	const MatType &	dataset,
		arma::Row< size_t > &	labels,
		const double	decisionBoundary = `0.5`
	)		const

Classify the given points, returning the predicted labels for each point.

Optionally, specify the decision boundary; logistic regression returns a value between 0 and 1. If the value is greater than the decision boundary, the response is taken to be 1; otherwise, it is 0. By default the decision boundary is 0.5.

Parameters

dataset	Set of points to classify.
labels	Predicted labels for each point.
decisionBoundary	Decision boundary (default 0.5).

◆ Classify() [3/3]

template<typename MatType >

void mlpack::regression::LogisticRegression< MatType >::Classify	(	const MatType &	dataset,
		arma::mat &	probabilities
	)		const

Classify the given points, returning class probabilities for each point.

Parameters

dataset	Set of points to classify.
probabilities	Class probabilities for each point (output).

◆ ComputeAccuracy()

template<typename MatType >

double mlpack::regression::LogisticRegression< MatType >::ComputeAccuracy	(	const MatType &	predictors,
		const arma::Row< size_t > &	responses,
		const double	decisionBoundary = `0.5`
	)		const

Compute the accuracy of the model on the given predictors and responses, optionally using the given decision boundary.

The responses should be either 0 or 1. Logistic regression returns a value between 0 and 1. If the value is greater than the decision boundary, the response is taken to be 1; otherwise, it is 0. By default, the decision boundary is 0.5.

The accuracy is returned as a percentage, between 0 and 100.

Parameters

predictors	Input predictors.
responses	Vector of responses.
decisionBoundary	Decision boundary (default 0.5).

Returns: Percentage of responses that are predicted correctly.

◆ ComputeError()

template<typename MatType >

double mlpack::regression::LogisticRegression< MatType >::ComputeError	(	const MatType &	predictors,
		const arma::Row< size_t > &	responses
	)		const

Compute the error of the model.

This returns the negative objective function of the logistic regression log-likelihood function. For the model to be optimal, the negative log-likelihood function should be minimized.

Parameters

predictors	Input predictors.
responses	Vector of responses.

◆ Train() [1/2]

template<typename MatType >

template<typename OptimizerType , typename... CallbackTypes>

double mlpack::regression::LogisticRegression< MatType >::Train	(	const MatType &	predictors,
		const arma::Row< size_t > &	responses,
		CallbackTypes &&...	callbacks
	)

Train the LogisticRegression model on the given input data.

By default, the L-BFGS optimization algorithm is used, but others can be specified (such as ens::SGD).

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

Template Parameters

OptimizerType	Type of optimizer to use to train the model.
CallbackTypes	Types of Callback Functions.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
callbacks	Callback function for ensmallen optimizer `OptimizerType`. See https://www.ensmallen.org/docs.html#callback-documentation.

Returns: The final objective of the trained model (NaN or Inf on error)

◆ Train() [2/2]

template<typename MatType >

template<typename OptimizerType , typename... CallbackTypes>

double mlpack::regression::LogisticRegression< MatType >::Train	(	const MatType &	predictors,
		const arma::Row< size_t > &	responses,
		OptimizerType &	optimizer,
		CallbackTypes &&...	callbacks
	)

Train the LogisticRegression model with the given instantiated optimizer.

Using this overload allows configuring the instantiated optimizer before training is performed.

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

Template Parameters

OptimizerType	Type of optimizer to use to train the model.
CallbackTypes	Types of Callback Functions.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
optimizer	Instantiated optimizer with instantiated error function.
callbacks	Callback function for ensmallen optimizer `OptimizerType`. See https://www.ensmallen.org/docs.html#callback-documentation.

Returns: The final objective of the trained model (NaN or Inf on error)

The documentation for this class was generated from the following files:

src/mlpack/methods/logistic_regression/logistic_regression.hpp
src/mlpack/methods/logistic_regression/logistic_regression_impl.hpp

Public Member Functions

Detailed Description

template<typename MatType = arma::mat> class mlpack::regression::LogisticRegression< MatType >

Constructor & Destructor Documentation

◆ LogisticRegression() [1/4]

◆ LogisticRegression() [2/4]

◆ LogisticRegression() [3/4]

◆ LogisticRegression() [4/4]

Member Function Documentation

◆ Classify() [1/3]

◆ Classify() [2/3]

◆ Classify() [3/3]

◆ ComputeAccuracy()

◆ ComputeError()

◆ Train() [1/2]

◆ Train() [2/2]

template<typename MatType = arma::mat>
class mlpack::regression::LogisticRegression< MatType >