|
crawlserv++
[under development]
Application for crawling and analyzing textual content of websites.
|
#include "../Config.hpp"#include <algorithm>#include <cstdint>#include <string>#include <vector>

Go to the source code of this file.
Classes | |
| class | crawlservpp::Module::Analyzer::Config |
| Abstract configuration for analyzers, to be implemented by algorithm classes. More... | |
| struct | crawlservpp::Module::Analyzer::Config::Entries |
| Configuration entries for analyzer threads. More... | |
Namespaces | |
| crawlservpp::Module::Analyzer | |
| Namespace for analyzer classes. | |
Constants | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalInputSourcesParsing {0} |
| An analyzer uses a parsing table as data source. More... | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalInputSourcesExtracting {1} |
| An analyzer uses an extracting table as data source. More... | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalInputSourcesAnalyzing {2} |
| An analyzer uses an analyzing table as data source. More... | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalInputSourcesCrawling {3} |
| An analyzer uses a crawling table as data source. More... | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalLoggingSilent {0} |
| Logging is disabled. More... | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalLoggingDefault {1} |
| Default logging is enabled. More... | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalLoggingExtended {2} |
| Extended logging is enabled. More... | |
| constexpr std::uint8_t | crawlservpp::Module::Analyzer::generalLoggingVerbose {3} |
| Verbose logging is enabled. More... | |
| constexpr std::int32_t | crawlservpp::Module::Analyzer::defaultRestartAfter {-1} |
| Default time (in s) after which to restart analysis once it has been completed (-1=deactivated). More... | |
| constexpr std::uint64_t | crawlservpp::Module::Analyzer::defaultSleepMySqlS {60} |
| Default time (in s) to wait before last try to re-connect to MySQL server. More... | |
| constexpr std::uint64_t | crawlservpp::Module::Analyzer::defaultSleepWhenFinishedMs {5000} |
| Default time (in ms) to wait each tick when finished. More... | |
| constexpr auto | crawlservpp::Module::Analyzer::minPercentageCorpusSlices {1} |
| Minimum percentage of the maximum length for corpus slices. More... | |
| constexpr auto | crawlservpp::Module::Analyzer::maxPercentageCorpusSlices {99} |
| Maximum percentage of the maximum length for corpus slices. More... | |
| constexpr auto | crawlservpp::Module::Analyzer::defaultPercentageCorpusSlices {30} |
| Default percentage of the maximum length for corpus slices. More... | |
| constexpr auto | crawlservpp::Module::Analyzer::defaultFreeMemoryEvery {100000000} |
| Default number of processed bytes in a continuous corpus after which memory will be freed. More... | |