crawlserv++
[under development]
Application for crawling and analyzing textual content of websites.
|
Abstract class as base for module-specific configurations. More...
#include <Config.hpp>
Classes | |
class | Exception |
Class for configuration exceptions. More... | |
Construction and Destruction | |
Config ()=default | |
Default constructor. More... | |
virtual | ~Config ()=default |
Default destructor. More... | |
Configuration Loader | |
void | loadConfig (const std::string &configJson, LogQueue &warningsTo) |
Loads a configuration. More... | |
Copy and Move | |
Config (Config &)=delete | |
Deleted copy constructor. More... | |
Config & | operator= (Config &)=delete |
Deleted copy assignment operator. More... | |
Config (Config &&)=delete | |
Deleted move constructor. More... | |
Config & | operator= (Config &&)=delete |
Deleted move assignment operator. More... | |
Parsing Options | |
enum | StringParsingOption { Default = 0, SQL, SubURL, URL, Trim } |
Options for parsing strings. More... | |
enum | CharParsingOption { FromNumber = 0, FromString } |
Options for parsing char's . More... | |
Configuration Parsing | |
void | category (const std::string &category) |
Sets the category of the subsequent configuration items to be checked for. More... | |
void | option (const std::string &name, bool &target) |
Checks for a configuration option of type bool . More... | |
void | option (const std::string &name, std::vector< bool > &target) |
Checks for a configuration option of type array of bool's . More... | |
void | option (const std::string &name, char &target, CharParsingOption opt) |
Checks for a configuration option of type char . More... | |
void | option (const std::string &name, std::vector< char > &target, CharParsingOption opt) |
Checks for a configuration option of type array of char's . More... | |
void | option (const std::string &name, std::int16_t &target) |
Checks for a configuration option of type 16-bit integer. More... | |
void | option (const std::string &name, std::vector< std::int16_t > &target) |
Checks for a configuration option of type array of 16-bit integers. More... | |
void | option (const std::string &name, std::int32_t &target) |
Checks for a configuration option of type 32-bit integer. More... | |
void | option (const std::string &name, std::vector< std::int32_t > &target) |
Checks for a configuration option of type array of 32-bit integers. More... | |
void | option (const std::string &name, std::int64_t &target) |
Checks for a configuration option of type 64-bit integer. More... | |
void | option (const std::string &name, std::vector< std::int64_t > &target) |
Checks for a configuration option of type array of 64-bit integers. More... | |
void | option (const std::string &name, std::uint8_t &target) |
Checks for a configuration option of type unsigned 8-bit integer. More... | |
void | option (const std::string &name, std::vector< std::uint8_t > &target) |
Checks for a configuration option of type array of unsigned 8-bit integers. More... | |
void | option (const std::string &name, std::uint16_t &target) |
Checks for a configuration option of type unsigned 16-bit integer. More... | |
void | option (const std::string &name, std::vector< std::uint16_t > &target) |
Checks for a configuration option of type array of unsigned 16-bit integers. More... | |
void | option (const std::string &name, std::uint32_t &target) |
Checks for a configuration option of type unsigned 32-bit integer. More... | |
void | option (const std::string &name, std::vector< std::uint32_t > &target) |
Checks for a configuration option of type array of unsigned 32-bit integers. More... | |
void | option (const std::string &name, std::uint64_t &target) |
Checks for a configuration option of type unsigned 64-bit integer. More... | |
void | option (const std::string &name, std::vector< std::uint64_t > &target) |
Checks for a configuration option of type array of unsigned 64-bit integers. More... | |
void | option (const std::string &name, float &target) |
Checks for a configuration option of type floating-point number. More... | |
void | option (const std::string &name, std::vector< float > &target) |
Checks for a configuration option of type array of floating-point numbers. More... | |
void | option (const std::string &name, std::string &target, StringParsingOption opt=Default) |
Checks for a configuration option of type string. More... | |
void | option (const std::string &name, std::vector< std::string > &target, StringParsingOption opt=Default) |
Checks for a configuration option of type array of strings. More... | |
void | warning (const std::string &warning) |
Adds a warning to the logging queue. More... | |
Module-specific Configuration Parsing | |
virtual void | parseBasicOption () |
Parses a basic option. More... | |
virtual void | resetBase () |
Resets basic options. More... | |
virtual void | parseOption ()=0 |
Parses a module-specific configuration option. More... | |
virtual void | checkOptions ()=0 |
Checks the module-specific configuration after parsing. More... | |
virtual void | reset ()=0 |
Resets the module-specific configuration. More... | |
Abstract class as base for module-specific configurations.
To add an option to a configuration, add a call to any of the option() functions (after a new call to category(), if necessary) to the local parseOption() implementation. It will be called for each configuration item to be parsed, checking whether this item contains the specified option and writing it to the specified target if it does.
|
protected |
|
protected |
|
default |
Default constructor.
|
virtualdefault |
Default destructor.
Reimplemented in crawlservpp::Module::Analyzer::Config.
|
delete |
Deleted copy constructor.
|
delete |
Deleted move constructor.
|
inlineprotected |
Sets the category of the subsequent configuration items to be checked for.
category | Constant reference to a string containing the name of the category. |
References crawlservpp::Struct::ConfigItem::category.
Referenced by crawlservpp::Module::Analyzer::Algo::TermsOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::Assoc::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AllTokens::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::TopicModelling::parseAlgoOption(), crawlservpp::Network::Config::parseBasicOption(), crawlservpp::Module::Parser::Config::parseOption(), crawlservpp::Module::Analyzer::Config::parseOption(), crawlservpp::Module::Crawler::Config::parseOption(), crawlservpp::Module::Extractor::Config::parseOption(), and crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo().
|
protectedpure virtual |
Checks the module-specific configuration after parsing.
Needs to be overridden by module-specific configuration child classes.
Implemented in crawlservpp::Module::Extractor::Config, crawlservpp::Module::Crawler::Config, crawlservpp::Module::Analyzer::Config, and crawlservpp::Module::Parser::Config.
Referenced by loadConfig().
|
inline |
Loads a configuration.
configJson | Constant reference to a string containing the configuration as JSON. |
warningsTo | Reference to a queue to which warnings will be added that occur during the parsing of the configuration, also known as the "logging queue". |
Module::Config::Exception | if the configuration JSON cannot be parsed. |
References crawlservpp::Struct::ConfigItem::category, checkOptions(), crawlservpp::Struct::ConfigItem::name, parseBasicOption(), crawlservpp::Helper::Json::parseRapid(), crawlservpp::Struct::ConfigItem::str(), crawlservpp::Struct::ConfigItem::value, and crawlservpp::Main::Exception::view().
Referenced by crawlservpp::Module::Analyzer::Thread::cleanUpQueries(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), and crawlservpp::Module::Crawler::Thread::onReset().
Deleted copy assignment operator.
Deleted move assignment operator.
|
inlineprotected |
Checks for a configuration option of type bool
.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a boolean variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
Referenced by crawlservpp::Module::Analyzer::Algo::Assoc::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AllTokens::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::TopicModelling::parseAlgoOption(), crawlservpp::Network::Config::parseBasicOption(), crawlservpp::Module::Parser::Config::parseOption(), crawlservpp::Module::Analyzer::Config::parseOption(), crawlservpp::Module::Crawler::Config::parseOption(), and crawlservpp::Module::Extractor::Config::parseOption().
|
inlineprotected |
Checks for a configuration option of type array of bool's
.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector of bool's into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type char
.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable of the type char into which the value of the configuration entry will be written if it is encountered. |
opt | Parsing options used for the configuration option. |
Module::Config::Exception | if no category has been set. |
References FromNumber, FromString, crawlservpp::Helper::Strings::getFirstOrEscapeChar(), crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of char's
.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector of char's into which the value of the configuration entry will be written if it is encountered. |
opt | Parsing options used for the configuration option. |
Module::Config::Exception | if no category has been set. |
References FromNumber, FromString, crawlservpp::Helper::Strings::getFirstOrEscapeChar(), crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type 16-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of 16-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type 32-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of 32-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type 64-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of 64-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type unsigned 8-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of unsigned 8-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type unsigned 16-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of unsigned 16-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type unsigned 32-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of unsigned 32-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type unsigned 64-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of unsigned 64-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type floating-point number.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of floating-point numbers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type string.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a string into which the value of the configuration entry will be stored if it is encountered. |
opt | Parsing option for the configuration entry. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Helper::Strings::checkSQLName(), Default, crawlservpp::Struct::ConfigItem::name, SQL, crawlservpp::Struct::ConfigItem::str(), SubURL, crawlservpp::Helper::Strings::trim(), Trim, URL, and crawlservpp::Struct::ConfigItem::value.
|
inlineprotected |
Checks for a configuration option of type array of strings.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
name | Constant reference to a string containing the name of the option to check for. |
target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
opt | Parsing option for the configuration entry. |
Module::Config::Exception | if no category has been set. |
References crawlservpp::Helper::Strings::checkSQLName(), Default, crawlservpp::Struct::ConfigItem::name, SQL, crawlservpp::Struct::ConfigItem::str(), SubURL, crawlservpp::Helper::Strings::trim(), Trim, URL, and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedvirtual |
Parses a basic option.
Might be overridden by child classes.
Can be used by abstract classes to add additional configuration entries without being the final implementation, as in Network::Config.
Reimplemented in crawlservpp::Network::Config.
References parseOption().
Referenced by loadConfig().
|
protectedpure virtual |
Parses a module-specific configuration option.
Needs to be overridden by module-specific configuration child classes.
Implemented in crawlservpp::Module::Extractor::Config, crawlservpp::Network::Config, crawlservpp::Module::Crawler::Config, crawlservpp::Module::Analyzer::Config, and crawlservpp::Module::Parser::Config.
Referenced by parseBasicOption().
|
protectedpure virtual |
Resets the module-specific configuration.
Needs to be overridden by module-specific configuration child classes.
Implemented in crawlservpp::Module::Extractor::Config, crawlservpp::Network::Config, crawlservpp::Module::Crawler::Config, crawlservpp::Module::Analyzer::Config, and crawlservpp::Module::Parser::Config.
Referenced by resetBase().
|
inlineprotectedvirtual |
Resets basic options.
Might be overridden by child classes.
Can be used by abstract classes to reset additional configuration entries without being the final implementation, as in Network::Config.
Reimplemented in crawlservpp::Network::Config.
References crawlservpp::Module::protocols, and reset().
Referenced by crawlservpp::Module::Analyzer::Thread::onReset(), and crawlservpp::Module::Parser::Thread::onReset().
|
inlineprotected |
Adds a warning to the logging queue.
warning | Constant reference to a string containing the warning. |
Module::Config::Exception | if no log queue is active. |
Referenced by crawlservpp::Module::Analyzer::Algo::Assoc::checkAlgoOptions(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::checkAlgoOptions(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::checkAlgoOptions(), crawlservpp::Module::Parser::Config::checkOptions(), crawlservpp::Module::Analyzer::Config::checkOptions(), crawlservpp::Module::Crawler::Config::checkOptions(), crawlservpp::Module::Extractor::Config::checkOptions(), and crawlservpp::Network::Config::parseBasicOption().