mlpack::data::CharExtract Class Reference

The class is used to split a string into characters. More...

#include <char_extract.hpp>

Public Types
using	TokenType = int
	The type of the token which the tokenizer extracts.

Public Member Functions
int	operator() (boost::string_view &str) const
	The function extracts the first character from the given string view and removes it from the view. More...

Static Public Member Functions
static bool	IsTokenEmpty (const int token)
	The function returns true if the given token is equal to EOF. More...

Detailed Description

The class is used to split a string into characters.

Member Function Documentation

IsTokenEmpty()

static bool mlpack::data::CharExtract::IsTokenEmpty ( const int  token )

inlinestatic

The function returns true if the given token is equal to EOF.

Parameters

token

The given token.

operator()()

int mlpack::data::CharExtract::operator() ( boost::string_view &  str ) const

inline

The function extracts the first character from the given string view and removes it from the view.

Each charecter is casted to unsigned char i.e. it belongs to [0, 255]. The functon returns EOF provided that the input string is empty.

Parameters

str	String view to retrieve the next token from.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/data/tokenizers/char_extract.hpp