|
crawlserv++
[under development]
Application for crawling and analyzing textual content of websites.
|
Abstract configuration for analyzers, to be implemented by algorithm classes. More...
#include <Config.hpp>


Classes | |
| struct | Entries |
| Configuration entries for analyzer threads. More... | |
Construction and Destruction | |
| Config ()=default | |
| Default constructor. More... | |
| ~Config () override=default | |
| Default destructor. More... | |
Configuration | |
| struct crawlservpp::Module::Analyzer::Config::Entries | config |
| Configuration of the analyzer. More... | |
Copy and Move | |
| Config (Config &)=delete | |
| Deleted copy constructor. More... | |
| Config & | operator= (Config &)=delete |
| Deleted copy assignment operator. More... | |
| Config (Config &&)=delete | |
| Deleted move constructor. More... | |
| Config & | operator= (Config &&)=delete |
| Deleted move assignment operator. More... | |
Analyzer-Specific Configuration Parsing | |
| void | parseOption () override |
| Parses an analyzer-specific configuration option. More... | |
| void | checkOptions () override |
| Checks the analyzer-specific configuration options. More... | |
| void | reset () override |
| Resets the analyzer-specific configuration options. More... | |
Algorithm-Specific Configuration Parsing | |
| virtual void | parseAlgoOption ()=0 |
| Parses an algorithm-specific configuration entry. More... | |
| virtual void | checkAlgoOptions ()=0 |
| Checks the algorithm-specific configuration. More... | |
| virtual void | resetAlgo ()=0 |
| Resets the algorithm-specific configuration. More... | |
Configuration Loader | |
| void | loadConfig (const std::string &configJson, LogQueue &warningsTo) |
| Loads a configuration. More... | |
Parsing Options | |
| enum | StringParsingOption { Default = 0, SQL, SubURL, URL, Trim } |
| Options for parsing strings. More... | |
| enum | CharParsingOption { FromNumber = 0, FromString } |
Options for parsing char's. More... | |
Configuration Parsing | |
| void | category (const std::string &category) |
| Sets the category of the subsequent configuration items to be checked for. More... | |
| void | option (const std::string &name, bool &target) |
Checks for a configuration option of type bool. More... | |
| void | option (const std::string &name, std::vector< bool > &target) |
Checks for a configuration option of type array of bool's. More... | |
| void | option (const std::string &name, char &target, CharParsingOption opt) |
Checks for a configuration option of type char. More... | |
| void | option (const std::string &name, std::vector< char > &target, CharParsingOption opt) |
Checks for a configuration option of type array of char's. More... | |
| void | option (const std::string &name, std::int16_t &target) |
| Checks for a configuration option of type 16-bit integer. More... | |
| void | option (const std::string &name, std::vector< std::int16_t > &target) |
| Checks for a configuration option of type array of 16-bit integers. More... | |
| void | option (const std::string &name, std::int32_t &target) |
| Checks for a configuration option of type 32-bit integer. More... | |
| void | option (const std::string &name, std::vector< std::int32_t > &target) |
| Checks for a configuration option of type array of 32-bit integers. More... | |
| void | option (const std::string &name, std::int64_t &target) |
| Checks for a configuration option of type 64-bit integer. More... | |
| void | option (const std::string &name, std::vector< std::int64_t > &target) |
| Checks for a configuration option of type array of 64-bit integers. More... | |
| void | option (const std::string &name, std::uint8_t &target) |
| Checks for a configuration option of type unsigned 8-bit integer. More... | |
| void | option (const std::string &name, std::vector< std::uint8_t > &target) |
| Checks for a configuration option of type array of unsigned 8-bit integers. More... | |
| void | option (const std::string &name, std::uint16_t &target) |
| Checks for a configuration option of type unsigned 16-bit integer. More... | |
| void | option (const std::string &name, std::vector< std::uint16_t > &target) |
| Checks for a configuration option of type array of unsigned 16-bit integers. More... | |
| void | option (const std::string &name, std::uint32_t &target) |
| Checks for a configuration option of type unsigned 32-bit integer. More... | |
| void | option (const std::string &name, std::vector< std::uint32_t > &target) |
| Checks for a configuration option of type array of unsigned 32-bit integers. More... | |
| void | option (const std::string &name, std::uint64_t &target) |
| Checks for a configuration option of type unsigned 64-bit integer. More... | |
| void | option (const std::string &name, std::vector< std::uint64_t > &target) |
| Checks for a configuration option of type array of unsigned 64-bit integers. More... | |
| void | option (const std::string &name, float &target) |
| Checks for a configuration option of type floating-point number. More... | |
| void | option (const std::string &name, std::vector< float > &target) |
| Checks for a configuration option of type array of floating-point numbers. More... | |
| void | option (const std::string &name, std::string &target, StringParsingOption opt=Default) |
| Checks for a configuration option of type string. More... | |
| void | option (const std::string &name, std::vector< std::string > &target, StringParsingOption opt=Default) |
| Checks for a configuration option of type array of strings. More... | |
| void | warning (const std::string &warning) |
| Adds a warning to the logging queue. More... | |
Module-specific Configuration Parsing | |
| virtual void | parseBasicOption () |
| Parses a basic option. More... | |
| virtual void | resetBase () |
| Resets basic options. More... | |
Abstract configuration for analyzers, to be implemented by algorithm classes.
|
protectedinherited |
|
protectedinherited |
|
default |
Default constructor.
|
overridevirtualdefault |
Default destructor.
Reimplemented from crawlservpp::Module::Config.
|
delete |
Deleted copy constructor.
|
delete |
Deleted move constructor.
|
inlineprotectedinherited |
Sets the category of the subsequent configuration items to be checked for.
| category | Constant reference to a string containing the name of the category. |
References crawlservpp::Struct::ConfigItem::category.
Referenced by crawlservpp::Module::Analyzer::Algo::TermsOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::Assoc::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AllTokens::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::TopicModelling::parseAlgoOption(), crawlservpp::Network::Config::parseBasicOption(), crawlservpp::Module::Parser::Config::parseOption(), parseOption(), crawlservpp::Module::Crawler::Config::parseOption(), crawlservpp::Module::Extractor::Config::parseOption(), and crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo().
|
protectedpure virtual |
Checks the algorithm-specific configuration.
Needs to be implemented by the (child) class for the specific algorithm.
Implemented in crawlservpp::Module::Analyzer::Algo::TopicModelling, crawlservpp::Module::Analyzer::Algo::SentimentOverTime, crawlservpp::Module::Analyzer::Algo::Empty, crawlservpp::Module::Analyzer::Algo::AllTokens, crawlservpp::Module::Analyzer::Algo::AssocOverTime, crawlservpp::Module::Analyzer::Algo::Assoc, crawlservpp::Module::Analyzer::Algo::WordsOverTime, crawlservpp::Module::Analyzer::Algo::CorpusGenerator, crawlservpp::Module::Analyzer::Algo::ExtractIds, and crawlservpp::Module::Analyzer::Algo::TermsOverTime.
Referenced by checkOptions().
|
inlineoverrideprotectedvirtual |
Checks the analyzer-specific configuration options.
Implements crawlservpp::Module::Config.
References checkAlgoOptions(), config, crawlservpp::Module::Analyzer::defaultPercentageCorpusSlices, crawlservpp::Module::Analyzer::Config::Entries::generalCorpusSlicing, crawlservpp::Module::Analyzer::Config::Entries::generalInputFields, crawlservpp::Module::Analyzer::Config::Entries::generalInputSources, crawlservpp::Module::Analyzer::Config::Entries::generalInputTables, crawlservpp::Module::Analyzer::maxPercentageCorpusSlices, crawlservpp::Module::Analyzer::minPercentageCorpusSlices, crawlservpp::Module::Analyzer::Config::Entries::tokenizerDicts, crawlservpp::Module::Analyzer::Config::Entries::tokenizerLanguages, crawlservpp::Module::Analyzer::Config::Entries::tokenizerManipulators, crawlservpp::Module::Analyzer::Config::Entries::tokenizerModels, and crawlservpp::Module::Config::warning().
|
inlineinherited |
Loads a configuration.
| configJson | Constant reference to a string containing the configuration as JSON. |
| warningsTo | Reference to a queue to which warnings will be added that occur during the parsing of the configuration, also known as the "logging queue". |
| Module::Config::Exception | if the configuration JSON cannot be parsed. |
References crawlservpp::Struct::ConfigItem::category, crawlservpp::Module::Config::checkOptions(), crawlservpp::Struct::ConfigItem::name, crawlservpp::Module::Config::parseBasicOption(), crawlservpp::Helper::Json::parseRapid(), crawlservpp::Struct::ConfigItem::str(), crawlservpp::Struct::ConfigItem::value, and crawlservpp::Main::Exception::view().
Referenced by crawlservpp::Module::Analyzer::Thread::cleanUpQueries(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), and crawlservpp::Module::Crawler::Thread::onReset().
Deleted copy assignment operator.
Deleted move assignment operator.
|
inlineprotectedinherited |
Checks for a configuration option of type bool.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a boolean variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
Referenced by crawlservpp::Module::Analyzer::Algo::Assoc::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::AllTokens::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::parseAlgoOption(), crawlservpp::Module::Analyzer::Algo::TopicModelling::parseAlgoOption(), crawlservpp::Network::Config::parseBasicOption(), crawlservpp::Module::Parser::Config::parseOption(), parseOption(), crawlservpp::Module::Crawler::Config::parseOption(), and crawlservpp::Module::Extractor::Config::parseOption().
|
inlineprotectedinherited |
Checks for a configuration option of type array of bool's.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector of bool's into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type char.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable of the type char into which the value of the configuration entry will be written if it is encountered. |
| opt | Parsing options used for the configuration option. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Module::Config::FromNumber, crawlservpp::Module::Config::FromString, crawlservpp::Helper::Strings::getFirstOrEscapeChar(), crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of char's.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector of char's into which the value of the configuration entry will be written if it is encountered. |
| opt | Parsing options used for the configuration option. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Module::Config::FromNumber, crawlservpp::Module::Config::FromString, crawlservpp::Helper::Strings::getFirstOrEscapeChar(), crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type 16-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of 16-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type 32-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of 32-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type 64-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of 64-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type unsigned 8-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of unsigned 8-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type unsigned 16-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of unsigned 16-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type unsigned 32-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of unsigned 32-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type unsigned 64-bit integer.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of unsigned 64-bit integers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type floating-point number.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a variable into which the value of the configuration entry will be written if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of floating-point numbers.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Struct::ConfigItem::name, crawlservpp::Struct::ConfigItem::str(), and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type string.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a string into which the value of the configuration entry will be stored if it is encountered. |
| opt | Parsing option for the configuration entry. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Helper::Strings::checkSQLName(), crawlservpp::Module::Config::Default, crawlservpp::Struct::ConfigItem::name, crawlservpp::Module::Config::SQL, crawlservpp::Struct::ConfigItem::str(), crawlservpp::Module::Config::SubURL, crawlservpp::Helper::Strings::trim(), crawlservpp::Module::Config::Trim, crawlservpp::Module::Config::URL, and crawlservpp::Struct::ConfigItem::value.
|
inlineprotectedinherited |
Checks for a configuration option of type array of strings.
Ignores the option and adds a warning to the warnings queue, if the requested type does not match the data type in the configuration JSON.
| name | Constant reference to a string containing the name of the option to check for. |
| target | Reference to a vector into which the value of the configuration entry will be stored if it is encountered. |
| opt | Parsing option for the configuration entry. |
| Module::Config::Exception | if no category has been set. |
References crawlservpp::Helper::Strings::checkSQLName(), crawlservpp::Module::Config::Default, crawlservpp::Struct::ConfigItem::name, crawlservpp::Module::Config::SQL, crawlservpp::Struct::ConfigItem::str(), crawlservpp::Module::Config::SubURL, crawlservpp::Helper::Strings::trim(), crawlservpp::Module::Config::Trim, crawlservpp::Module::Config::URL, and crawlservpp::Struct::ConfigItem::value.
|
protectedpure virtual |
Parses an algorithm-specific configuration entry.
Needs to be implemented by the (child) class for the specific algorithm.
Implemented in crawlservpp::Module::Analyzer::Algo::TopicModelling, crawlservpp::Module::Analyzer::Algo::SentimentOverTime, crawlservpp::Module::Analyzer::Algo::Empty, crawlservpp::Module::Analyzer::Algo::AllTokens, crawlservpp::Module::Analyzer::Algo::AssocOverTime, crawlservpp::Module::Analyzer::Algo::Assoc, crawlservpp::Module::Analyzer::Algo::WordsOverTime, crawlservpp::Module::Analyzer::Algo::CorpusGenerator, crawlservpp::Module::Analyzer::Algo::ExtractIds, and crawlservpp::Module::Analyzer::Algo::TermsOverTime.
Referenced by parseOption().
|
inlineprotectedvirtualinherited |
Parses a basic option.
Might be overridden by child classes.
Can be used by abstract classes to add additional configuration entries without being the final implementation, as in Network::Config.
Reimplemented in crawlservpp::Network::Config.
References crawlservpp::Module::Config::parseOption().
Referenced by crawlservpp::Module::Config::loadConfig().
|
inlineoverrideprotectedvirtual |
Parses an analyzer-specific configuration option.
Implements crawlservpp::Module::Config.
References crawlservpp::Module::Config::category(), config, crawlservpp::Module::Analyzer::Config::Entries::filterDateEnable, crawlservpp::Module::Analyzer::Config::Entries::filterDateFrom, crawlservpp::Module::Analyzer::Config::Entries::filterDateTo, crawlservpp::Module::Analyzer::Config::Entries::filterQueryAll, crawlservpp::Module::Analyzer::Config::Entries::filterQueryQueries, crawlservpp::Module::Analyzer::Config::Entries::generalCorpusChecks, crawlservpp::Module::Analyzer::Config::Entries::generalCorpusSlicing, crawlservpp::Module::Analyzer::Config::Entries::generalInputFields, crawlservpp::Module::Analyzer::Config::Entries::generalInputSources, crawlservpp::Module::Analyzer::Config::Entries::generalInputTables, crawlservpp::Module::Analyzer::Config::Entries::generalLogging, crawlservpp::Module::Analyzer::Config::Entries::generalRestartAfter, crawlservpp::Module::Analyzer::Config::Entries::generalSleepMySql, crawlservpp::Module::Analyzer::Config::Entries::generalSleepWhenFinished, crawlservpp::Module::Analyzer::Config::Entries::generalTargetTable, crawlservpp::Module::Analyzer::Config::Entries::groupDateFillGaps, crawlservpp::Module::Analyzer::Config::Entries::groupDateResolution, crawlservpp::Module::Config::option(), parseAlgoOption(), crawlservpp::Module::Analyzer::Config::Entries::tokenizerDicts, crawlservpp::Module::Analyzer::Config::Entries::tokenizerFreeMemoryEvery, crawlservpp::Module::Analyzer::Config::Entries::tokenizerLanguages, crawlservpp::Module::Analyzer::Config::Entries::tokenizerManipulators, crawlservpp::Module::Analyzer::Config::Entries::tokenizerModels, crawlservpp::Module::Analyzer::Config::Entries::tokenizerSavePoints, crawlservpp::Module::Analyzer::Config::Entries::uploadFTP, crawlservpp::Module::Analyzer::Config::Entries::uploadProxy, crawlservpp::Module::Analyzer::Config::Entries::uploadTargetColumn, and crawlservpp::Module::Analyzer::Config::Entries::uploadVerbose.
|
inlineoverrideprotectedvirtual |
Resets the analyzer-specific configuration options.
Implements crawlservpp::Module::Config.
References config, and resetAlgo().
|
protectedpure virtual |
Resets the algorithm-specific configuration.
Needs to be implemented by the (child) class for the specific algorithm.
Implemented in crawlservpp::Module::Analyzer::Algo::TopicModelling, crawlservpp::Module::Analyzer::Algo::SentimentOverTime, crawlservpp::Module::Analyzer::Algo::Empty, crawlservpp::Module::Analyzer::Algo::AllTokens, crawlservpp::Module::Analyzer::Algo::AssocOverTime, crawlservpp::Module::Analyzer::Algo::Assoc, crawlservpp::Module::Analyzer::Algo::WordsOverTime, crawlservpp::Module::Analyzer::Algo::CorpusGenerator, crawlservpp::Module::Analyzer::Algo::ExtractIds, and crawlservpp::Module::Analyzer::Algo::TermsOverTime.
Referenced by crawlservpp::Module::Analyzer::Thread::onReset(), and reset().
|
inlineprotectedvirtualinherited |
Resets basic options.
Might be overridden by child classes.
Can be used by abstract classes to reset additional configuration entries without being the final implementation, as in Network::Config.
Reimplemented in crawlservpp::Network::Config.
References crawlservpp::Module::protocols, and crawlservpp::Module::Config::reset().
Referenced by crawlservpp::Module::Analyzer::Thread::onReset(), and crawlservpp::Module::Parser::Thread::onReset().
|
inlineprotectedinherited |
Adds a warning to the logging queue.
| warning | Constant reference to a string containing the warning. |
| Module::Config::Exception | if no log queue is active. |
Referenced by crawlservpp::Module::Analyzer::Algo::Assoc::checkAlgoOptions(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::checkAlgoOptions(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::checkAlgoOptions(), crawlservpp::Module::Parser::Config::checkOptions(), checkOptions(), crawlservpp::Module::Crawler::Config::checkOptions(), crawlservpp::Module::Extractor::Config::checkOptions(), and crawlservpp::Network::Config::parseBasicOption().
| struct crawlservpp::Module::Analyzer::Config::Entries crawlservpp::Module::Analyzer::Config::config |
Configuration of the analyzer.
Referenced by crawlservpp::Module::Analyzer::Thread::addCorpora(), crawlservpp::Module::Analyzer::Thread::checkCorpusSources(), checkOptions(), crawlservpp::Module::Analyzer::Thread::cleanUpQueries(), crawlservpp::Module::Analyzer::Thread::getTargetTableName(), crawlservpp::Module::Analyzer::Algo::CorpusGenerator::onAlgoInit(), crawlservpp::Module::Analyzer::Thread::onTick(), parseOption(), reset(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::TopicModelling::resetAlgo(), and crawlservpp::Module::Analyzer::Thread::uploadResult().