crawlserv++  [under development]
Application for crawling and analyzing textual content of websites.
crawlservpp::Module Namespace Reference

Namespace for the different modules run by threads. More...

Namespaces

 Analyzer
 Namespace for analyzer classes.
 
 Crawler
 Namespace for crawler classes.
 
 Extractor
 Namespace for extractor classes.
 
 Parser
 Namespace for parser classes.
 

Classes

class  Config
 Abstract class as base for module-specific configurations. More...
 
class  Database
 Class handling database access for threads. More...
 
class  Thread
 Abstract class providing module-independent thread functionality. More...
 

Constants

constexpr std::array protocols
 Protocols supported by the libcurl library. More...
 
constexpr auto sqlArg1 {1}
 First argument in a SQL query. More...
 
constexpr auto sqlArg2 {2}
 Second argument in a SQL query. More...
 
constexpr auto sqlArg3 {3}
 Third argument in a SQL query. More...
 
constexpr auto sleepOnConnectionErrorS {10}
 Number of seconds to sleep on connection errors. More...
 
constexpr auto sleepMs {800}
 Number of milliseconds to sleep before checking whether the thread is still running. More...
 
constexpr auto statusPrefixInterrupted {"INTERRUPTED "sv}
 Status message prefix for interrupted threads. More...
 
constexpr auto statusPrefixPaused {"PAUSED "sv}
 Status message prefix for paused threads. More...
 

Detailed Description

Namespace for the different modules run by threads.

Variable Documentation

◆ protocols

constexpr std::array crawlservpp::Module::protocols
inline
Initial value:
{
"ftp://"sv,
"scp://"sv,
"smb://"sv,
"http://"sv,
"ftps://"sv,
"sftp://"sv,
"file://"sv,
"dict://"sv,
"imap://"sv,
"ldap://"sv,
"pop3://"sv,
"rtmp://"sv,
"rtsp://"sv,
"smbs://"sv,
"smtp://"sv,
"tftp://"sv,
"https://"sv,
"imaps://"sv,
"ldaps://"sv,
"pop3s://"sv,
"smtps://"sv,
"gopher://"sv,
"telnet://"sv
}

Protocols supported by the libcurl library.

Referenced by crawlservpp::Module::Config::resetBase().

◆ sleepMs

constexpr auto crawlservpp::Module::sleepMs {800}
inline

Number of milliseconds to sleep before checking whether the thread is still running.

Referenced by crawlservpp::Module::Thread::sleep().

◆ sleepOnConnectionErrorS

constexpr auto crawlservpp::Module::sleepOnConnectionErrorS {10}
inline

Number of seconds to sleep on connection errors.

Referenced by crawlservpp::Module::Thread::log().

◆ sqlArg1

constexpr auto crawlservpp::Module::sqlArg1 {1}
inline

◆ sqlArg2

constexpr auto crawlservpp::Module::sqlArg2 {2}
inline

◆ sqlArg3

constexpr auto crawlservpp::Module::sqlArg3 {3}
inline

◆ statusPrefixInterrupted

constexpr auto crawlservpp::Module::statusPrefixInterrupted {"INTERRUPTED "sv}
inline

Status message prefix for interrupted threads.

Referenced by crawlservpp::Module::Thread::Thread().

◆ statusPrefixPaused

constexpr auto crawlservpp::Module::statusPrefixPaused {"PAUSED "sv}
inline

Status message prefix for paused threads.

Referenced by crawlservpp::Module::Thread::Thread().