crawlserv++
[under development]
Application for crawling and analyzing textual content of websites.
|
Abstract class providing module-independent thread functionality. More...
#include <Thread.hpp>
Classes | |
class | Exception |
Class for generic thread exceptions. More... | |
Construction and Destruction | |
Thread (Main::Database &dbBase, const ThreadOptions &threadOptions, const ThreadStatus &threadStatus) | |
Constructor initializing a previously interrupted thread. More... | |
Thread (Main::Database &dbBase, const ThreadOptions &threadOptions) | |
Constructor initializing a new thread. More... | |
virtual | ~Thread ()=default |
Default destructor. More... | |
Getters | |
std::uint64_t | getId () const |
Gets the ID of the thread. More... | |
std::uint64_t | getWebsite () const |
Gets the ID of the website used by the thread. More... | |
std::uint64_t | getUrlList () const |
Gets the ID of the URL list used by the thread. More... | |
std::uint64_t | getConfig () const |
Gets the ID of the configuration used by the thread. More... | |
bool | isShutdown () const |
Checks whether the thread is shutting down or has shut down. More... | |
bool | isRunning () const |
Checks whether the thread is still supposed to run. More... | |
bool | isFinished () const |
Checks whether the shutdown of the thread has been finished. More... | |
bool | isPaused () const |
Checks whether the thread has been paused. More... | |
Thread Control | |
void | start () |
Starts running the thread. More... | |
bool | pause () |
Pauses the thread. More... | |
void | unpause () |
Unpauses the thread. More... | |
void | stop () |
Shuts down the thread. More... | |
void | interrupt () |
Interrupts the thread due to an exception. More... | |
void | end () |
Waits for the thread until shutdown is completed. More... | |
void | reset () |
Will reset the thread before the next tick. More... | |
Time Travel | |
void | warpTo (std::uint64_t target) |
Jumps to the specified target ID ("time travel"). More... | |
Copy and Move | |
Thread (Thread &)=delete | |
Deleted copy constructor. More... | |
Thread & | operator= (Thread &)=delete |
Deleted copy assignment operator. More... | |
Thread (Thread &&)=delete | |
Deleted move constructor. More... | |
Thread & | operator= (Thread &&)=delete |
Deleted move assignment operator. More... | |
Database Connection | |
Database | database |
Database connection for the thread. More... | |
Configuration | |
std::string | websiteNamespace |
Namespace of the website used by the thread. More... | |
std::string | urlListNamespace |
Namespace of the URL list used by the thread. More... | |
std::string | configuration |
JSON string of the configuration used by the thread. More... | |
Protected Getters | |
bool | isInterrupted () const |
Checks whether the thread has been interrupted. More... | |
std::string | getStatusMessage () const |
Gets the current status message. More... | |
float | getProgress () const |
Gets the current progress, in percent. More... | |
std::uint64_t | getLast () const |
Gets the value of the last ID processed by the thread. More... | |
std::int64_t | getWarpedOverAndReset () |
Gets the number of IDs that have been jumped over, and resets them. More... | |
Protected Setters | |
void | setStatusMessage (const std::string &statusMessage) |
Sets the status message of the thread. More... | |
void | setProgress (float newProgress) |
Sets the progress of the thread. More... | |
void | setLast (std::uint64_t lastId) |
Sets the last ID processed by the thread. More... | |
void | incrementLast () |
Increments the last ID processed by the thread. More... | |
void | incrementProcessed () |
Increments the number of IDs processed by the thread. More... | |
Protected Thread Control | |
void | sleep (std::uint64_t ms) const |
Lets the thread sleep for the specified number of milliseconds. More... | |
void | allowPausing () |
Allows the thread to be paused. More... | |
void | disallowPausing () |
Disallows the thread to be paused. More... | |
void | pauseByThread () |
Forces the thread to pause. More... | |
Logging | |
bool | isLogLevel (std::uint8_t level) const |
Checks whether a certain logging level is enabled. More... | |
void | log (std::uint8_t level, const std::string &logEntry) |
Adds a thread-specific log entry to the database, if the current logging level is high enough. More... | |
void | log (std::uint8_t level, std::queue< std::string > &logEntries) |
Adds multiple thread-specific log entries to the database, if the current logging level is high enough. More... | |
Events | |
virtual void | onInit ()=0 |
Initializes the module. More... | |
virtual void | onTick ()=0 |
Performs a module tick. More... | |
virtual void | onPause ()=0 |
Pauses the module. More... | |
virtual void | onUnpause ()=0 |
Unpauses the module. More... | |
virtual void | onClear ()=0 |
Clears the module. More... | |
virtual void | onReset ()=0 |
Resets the module. More... | |
Abstract class providing module-independent thread functionality.
crawlservpp::Module::Thread::Thread | ( | Main::Database & | dbBase, |
const ThreadOptions & | threadOptions, | ||
const ThreadStatus & | threadStatus | ||
) |
Constructor initializing a previously interrupted thread.
dbBase | Reference to the main database connection. |
threadOptions | Constant reference to a structure containing the options for the thread. |
threadStatus | Constant reference to a structure containing the last known status of the thread. |
References configuration, database, getConfig(), crawlservpp::Main::Database::getConfiguration(), getUrlList(), crawlservpp::Main::Database::getUrlListNamespace(), getWebsite(), crawlservpp::Main::Database::getWebsiteNamespace(), crawlservpp::Struct::ThreadStatus::id, crawlservpp::Struct::ThreadStatus::paused, crawlservpp::Module::Database::setOptions(), crawlservpp::Main::Database::setThreadStatus(), crawlservpp::Struct::ThreadStatus::status, crawlservpp::Module::statusPrefixInterrupted, crawlservpp::Module::statusPrefixPaused, urlListNamespace, and websiteNamespace.
crawlservpp::Module::Thread::Thread | ( | Main::Database & | dbBase, |
const ThreadOptions & | threadOptions | ||
) |
Constructor initializing a new thread.
dbBase | Reference to the main database connection. |
threadOptions | Constant reference to a structure containing the options for the thread. |
References crawlservpp::Main::Database::addThread(), database, and crawlservpp::Module::Database::setThreadId().
|
virtualdefault |
Default destructor.
|
delete |
Deleted copy constructor.
|
delete |
Deleted move constructor.
|
protected |
Allows the thread to be paused.
Threads are pausable by default. Use this function if pausing has been disallowed via disallowPausing().
Thread-safe: Can be used by both the module and the main thread.
|
protected |
Disallows the thread to be paused.
Thread-safe: Can be used by both the module and the main thread.
Referenced by crawlservpp::Module::Analyzer::Algo::AllTokens::AllTokens(), crawlservpp::Module::Analyzer::Algo::Assoc::Assoc(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::AssocOverTime(), crawlservpp::Module::Analyzer::Algo::CorpusGenerator::CorpusGenerator(), crawlservpp::Module::Analyzer::Algo::ExtractIds::ExtractIds(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::SentimentOverTime(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::TermsOverTime(), crawlservpp::Module::Analyzer::Algo::TopicModelling::TopicModelling(), and crawlservpp::Module::Analyzer::Algo::WordsOverTime::WordsOverTime().
void crawlservpp::Module::Thread::end | ( | ) |
Waits for the thread until shutdown is completed.
References crawlservpp::Main::Database::deleteThread().
Referenced by crawlservpp::Module::Crawler::Thread::onReset(), and crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo().
std::uint64_t crawlservpp::Module::Thread::getConfig | ( | ) | const |
Gets the ID of the configuration used by the thread.
Thread-safe: Can be used by both the module and the main thread, because the configuration is not changed after starting the thread.
References crawlservpp::Struct::ThreadOptions::config.
Referenced by Thread().
std::uint64_t crawlservpp::Module::Thread::getId | ( | ) | const |
Gets the ID of the thread.
Thread-safe: Can be used by both the module and the main thread.
|
protected |
Gets the value of the last ID processed by the thread.
Referenced by crawlservpp::Module::Crawler::Thread::onInit(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Parser::Thread::onTick(), and crawlservpp::Module::Extractor::Thread::onTick().
|
protected |
Gets the current progress, in percent.
Thread-safe: Can be used by both the module and the main thread.
0.F
(none) and 1.F
(done). Referenced by crawlservpp::Module::Parser::Thread::onReset(), and crawlservpp::Module::Extractor::Thread::onReset().
|
protected |
Gets the current status message.
Thread-safe: Can be used by both the module and the main thread.
Referenced by log(), crawlservpp::Module::Parser::Thread::onClear(), crawlservpp::Module::Extractor::Thread::onClear(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), and crawlservpp::Module::Crawler::Thread::onReset().
std::uint64_t crawlservpp::Module::Thread::getUrlList | ( | ) | const |
Gets the ID of the URL list used by the thread.
Thread-safe: Can be used by both the module and the main thread, because the URL list is not changed after starting the thread.
References crawlservpp::Struct::ThreadOptions::urlList.
Referenced by Thread().
|
protected |
Gets the number of IDs that have been jumped over, and resets them.
Resets the number of IDs jumped over to zero.
Referenced by crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Parser::Thread::onTick(), and crawlservpp::Module::Extractor::Thread::onTick().
std::uint64_t crawlservpp::Module::Thread::getWebsite | ( | ) | const |
Gets the ID of the website used by the thread.
Thread-safe: Can be used by both the module and the main thread, because the website is not changed after starting the thread.
References crawlservpp::Struct::ThreadOptions::website.
Referenced by crawlservpp::Module::Crawler::Thread::onReset(), and Thread().
|
protected |
Increments the last ID processed by the thread.
Also sets the number of processed IDs, make sure to increment it before if the ID has been processed.
References database, and crawlservpp::Module::Database::setThreadLast().
|
protected |
Increments the number of IDs processed by the thread.
Referenced by crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), and crawlservpp::Module::Crawler::Thread::onReset().
void crawlservpp::Module::Thread::interrupt | ( | ) |
bool crawlservpp::Module::Thread::isFinished | ( | ) | const |
Checks whether the shutdown of the thread has been finished.
Thread-safe: Can be used by both the module and the main thread.
|
protected |
Checks whether the thread has been interrupted.
Thread-safe: Can be used by both the module and the main thread.
|
protected |
Checks whether a certain logging level is enabled.
level | The logging level to be checked for. |
References database, and crawlservpp::Module::Database::isLogLevel().
Referenced by crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Parser::Thread::onTick(), and crawlservpp::Module::Extractor::Thread::onTick().
bool crawlservpp::Module::Thread::isPaused | ( | ) | const |
Checks whether the thread has been paused.
Thread-safe: Can be used by both the module and the main thread.
bool crawlservpp::Module::Thread::isRunning | ( | ) | const |
Checks whether the thread is still supposed to run.
Thread-safe: Can be used by both the module and the main thread.
Referenced by crawlservpp::Module::Analyzer::Thread::cleanUpQueries(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::ExtractIds::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::CorpusGenerator::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Assoc::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AllTokens::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Empty::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::TopicModelling::onAlgoInit(), crawlservpp::Module::Analyzer::Thread::onInit(), crawlservpp::Module::Parser::Thread::onInit(), crawlservpp::Module::Extractor::Thread::onInit(), crawlservpp::Module::Crawler::Thread::onInit(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Parser::Thread::onTick(), crawlservpp::Module::Analyzer::Algo::ExtractIds::resetAlgo(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::Assoc::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AllTokens::resetAlgo(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo(), and crawlservpp::Module::Analyzer::Algo::TopicModelling::resetAlgo().
bool crawlservpp::Module::Thread::isShutdown | ( | ) | const |
Checks whether the thread is shutting down or has shut down.
Thread-safe: Can be used by both the module and the main thread.
|
protected |
Adds a thread-specific log entry to the database, if the current logging level is high enough.
Removes invalid UTF-8 characters if necessary.
If debug logging is active, the entry will be written to the logging file as well.
The log entry will not be written to the database, if the current logging level is lower than the specified logging level. The logging level does not affect the writing of logging entries being to the logging file when debug logging is active.
level | The logging level for the entry. The entry will only be written to the database, if the current logging level is at least the logging level for the entry. |
logEntry | Constant reference to a string containing the log entry. |
References database, and crawlservpp::Module::Database::log().
Referenced by crawlservpp::Module::Analyzer::Thread::addCorpora(), crawlservpp::Module::Analyzer::Algo::TopicModelling::checkAlgoOptions(), crawlservpp::Module::Analyzer::Thread::cleanUpQueries(), crawlservpp::Module::Analyzer::Thread::finished(), log(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::ExtractIds::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::CorpusGenerator::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Assoc::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AllTokens::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Empty::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::TopicModelling::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AllTokens::onAlgoTick(), crawlservpp::Module::Parser::Thread::onClear(), crawlservpp::Module::Extractor::Thread::onClear(), crawlservpp::Module::Crawler::Thread::onClear(), crawlservpp::Module::Analyzer::Thread::onReset(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Parser::Thread::onTick(), crawlservpp::Module::Extractor::Thread::onTick(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::ExtractIds::resetAlgo(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::Assoc::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AllTokens::resetAlgo(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::TopicModelling::resetAlgo(), and crawlservpp::Module::Analyzer::Thread::uploadResult().
|
protected |
Adds multiple thread-specific log entries to the database, if the current logging level is high enough.
Removes invalid UTF-8 characters if necessary.
If debug logging is active, the entries will be written to the logging file as well.
The log entries will not be written to the database, if the current logging level is lower than the specified logging level. The logging level does not affect the writing of logging entries being to the logging file when debug logging is active.
level | The logging level for the entries. The entries will only be written to the database, if the current logging level is at least the logging level for the entry. |
logEntries | Reference to a queue of strings containing the log entries to be written. It will be emptied regardless whether the log entries will be written to the database. |
References crawlservpp::Main::Database::connect(), database, getStatusMessage(), crawlservpp::Main::Database::getThreadPauseTime(), crawlservpp::Main::Database::getThreadRunTime(), crawlservpp::Module::Database::log(), log(), crawlservpp::Helper::DateTime::now(), onClear(), onInit(), onPause(), onReset(), onTick(), onUnpause(), pause(), pauseByThread(), crawlservpp::Module::Database::prepare(), crawlservpp::Helper::DateTime::secondsToString(), setLast(), setStatusMessage(), crawlservpp::Main::Database::setThreadPauseTime(), crawlservpp::Main::Database::setThreadRunTime(), and crawlservpp::Module::sleepOnConnectionErrorS.
|
protectedpure virtual |
Clears the module.
Needs to be implemented by the (child) class for the specific module.
Implemented in crawlservpp::Module::Crawler::Thread, crawlservpp::Module::Extractor::Thread, crawlservpp::Module::Parser::Thread, and crawlservpp::Module::Analyzer::Thread.
Referenced by log().
|
protectedpure virtual |
Initializes the module.
Needs to be implemented by the (child) class for the specific module.
Implemented in crawlservpp::Module::Crawler::Thread, crawlservpp::Module::Extractor::Thread, crawlservpp::Module::Parser::Thread, and crawlservpp::Module::Analyzer::Thread.
Referenced by log().
|
protectedpure virtual |
Pauses the module.
Needs to be implemented by the (child) class for the specific module.
Implemented in crawlservpp::Module::Crawler::Thread, crawlservpp::Module::Extractor::Thread, crawlservpp::Module::Parser::Thread, and crawlservpp::Module::Analyzer::Thread.
Referenced by log().
|
protectedpure virtual |
Resets the module.
Needs to be implemented by the (child) class for the specific module.
Implemented in crawlservpp::Module::Crawler::Thread, crawlservpp::Module::Extractor::Thread, crawlservpp::Module::Parser::Thread, and crawlservpp::Module::Analyzer::Thread.
Referenced by log().
|
protectedpure virtual |
Performs a module tick.
Needs to be implemented by the (child) class for the specific module.
Implemented in crawlservpp::Module::Crawler::Thread, crawlservpp::Module::Extractor::Thread, crawlservpp::Module::Parser::Thread, and crawlservpp::Module::Analyzer::Thread.
Referenced by log().
|
protectedpure virtual |
Unpauses the module.
Needs to be implemented by the (child) class for the specific module.
Implemented in crawlservpp::Module::Crawler::Thread, crawlservpp::Module::Extractor::Thread, crawlservpp::Module::Parser::Thread, and crawlservpp::Module::Analyzer::Thread.
Referenced by log().
Deleted copy assignment operator.
Deleted move assignment operator.
bool crawlservpp::Module::Thread::pause | ( | ) |
Pauses the thread.
References crawlservpp::Main::Database::setThreadStatus().
Referenced by log().
|
protected |
Forces the thread to pause.
References database, and crawlservpp::Main::Database::setThreadStatus().
Referenced by log(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), and crawlservpp::Module::Analyzer::Thread::pause().
void crawlservpp::Module::Thread::reset | ( | ) |
Will reset the thread before the next tick.
|
protected |
Sets the last ID processed by the thread.
Also sets the number of processed IDs, make sure to increment it before if the ID has been processed.
lastId | The last ID processed by the thread. |
References database, and crawlservpp::Module::Database::setThreadLast().
Referenced by log(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), and crawlservpp::Module::Crawler::Thread::onReset().
|
protected |
Sets the progress of the thread.
newProgress | The new progress of the thread, between 0.f (none), and 1.f (done). |
References database, and crawlservpp::Module::Database::setThreadProgress().
Referenced by crawlservpp::Module::Analyzer::Thread::finished(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::ExtractIds::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::CorpusGenerator::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Assoc::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AllTokens::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Empty::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::TopicModelling::onAlgoInit(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Analyzer::Thread::onTick(), crawlservpp::Module::Parser::Thread::onTick(), crawlservpp::Module::Extractor::Thread::onTick(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::ExtractIds::resetAlgo(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::Assoc::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AllTokens::resetAlgo(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo(), and crawlservpp::Module::Analyzer::Algo::TopicModelling::resetAlgo().
|
protected |
Sets the status message of the thread.
statusMessage | Constant reference to a string containing the new status message to be set. |
References database, and crawlservpp::Main::Database::setThreadStatus().
Referenced by crawlservpp::Module::Analyzer::Thread::cleanUpQueries(), crawlservpp::Module::Analyzer::Thread::finished(), log(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::ExtractIds::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::CorpusGenerator::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Assoc::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::AllTokens::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::Empty::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::TopicModelling::onAlgoInit(), crawlservpp::Module::Analyzer::Algo::CorpusGenerator::onAlgoTick(), crawlservpp::Module::Analyzer::Algo::AllTokens::onAlgoTick(), crawlservpp::Module::Analyzer::Algo::TopicModelling::onAlgoTick(), crawlservpp::Module::Parser::Thread::onClear(), crawlservpp::Module::Extractor::Thread::onClear(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Parser::Thread::onTick(), crawlservpp::Module::Extractor::Thread::onTick(), crawlservpp::Module::Analyzer::Algo::TermsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::ExtractIds::resetAlgo(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::Assoc::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AllTokens::resetAlgo(), crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::TopicModelling::resetAlgo(), and crawlservpp::Module::Analyzer::Thread::uploadResult().
|
protected |
Lets the thread sleep for the specified number of milliseconds.
The sleep will be interrupted if the thread is stopped.
Thread-safe: Can be used by both the module and the main thread.
ms | The number of milliseconds for the thread to sleep, if it is not stopped. |
References crawlservpp::Module::sleepMs.
Referenced by crawlservpp::Module::Analyzer::Algo::CorpusGenerator::onAlgoTick(), crawlservpp::Module::Crawler::Thread::onReset(), crawlservpp::Module::Analyzer::Thread::onTick(), crawlservpp::Module::Parser::Thread::onTick(), and crawlservpp::Module::Extractor::Thread::onTick().
void crawlservpp::Module::Thread::start | ( | ) |
Starts running the thread.
void crawlservpp::Module::Thread::stop | ( | ) |
void crawlservpp::Module::Thread::unpause | ( | ) |
Unpauses the thread.
References crawlservpp::Main::Database::setThreadStatus().
Referenced by interrupt(), and stop().
void crawlservpp::Module::Thread::warpTo | ( | std::uint64_t | target | ) |
Jumps to the specified target ID ("time travel").
Skips the normal process of determining the next ID once the current ID has been processed.
Thread-safe: Can be used by both the module and the main thread.
target | The target ID that should be processed next. |
Module::Thread::Exception | if no target is specified, i.e. the target ID is zero. |
|
protected |
JSON string of the configuration used by the thread.
Referenced by Thread().
|
protected |
Database connection for the thread.
Referenced by incrementLast(), isLogLevel(), log(), pauseByThread(), setLast(), setProgress(), setStatusMessage(), and Thread().
|
protected |
Namespace of the URL list used by the thread.
Referenced by crawlservpp::Module::Analyzer::Thread::getTargetTableName(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), and Thread().
|
protected |
Namespace of the website used by the thread.
Referenced by crawlservpp::Module::Analyzer::Thread::getTargetTableName(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Module::Crawler::Thread::onReset(), and Thread().