crawlserv++  [under development]
Application for crawling and analyzing textual content of websites.
crawlservpp::Helper::Json Namespace Reference

Namespace for global JSON helper functions. More...

Classes

class  Exception
 Class for JSON exceptions. More...
 

Constants

constexpr auto unicodeEscapeLength {6}
 The length of an escaped Unicode character in JSON code (including the '\u'). More...
 
constexpr auto unicodeEscapeDigit1 {2}
 The offset of the first Unicode character digit in JSON code (from the '\'). More...
 
constexpr auto unicodeEscapeDigit2 {3}
 The offset of the second Unicode character digit in JSON code (from the '\'). More...
 
constexpr auto unicodeEscapeDigit3 {4}
 The offset of the third Unicode character digit in JSON code (from the '\'). More...
 
constexpr auto unicodeEscapeDigit4 {5}
 The offset of the fourth Unicode character digit in JSON code (from the '\'). More...
 
constexpr auto numDebugChars {25}
 The number of characters to show before and behind a JSON error. More...
 

Stringification

std::string stringify (const std::vector< std::string > &vectorToStringify)
 Stringifies a vector of strings into one string containing a JSON array. More...
 
std::string stringify (const std::string &stringToStringify)
 Converts a string into a JSON array with the string as the only element inside it. More...
 
std::string stringify (const char *stringToStringify)
 Converts a string into a JSON array with the string as the only element inside it. More...
 
std::string stringify (const std::vector< std::vector< std::pair< std::string, std::string >>> &vectorToStringify)
 Converts a vector of vectors of string pairs into a JSON array with corresponding objects containing [key, value] pairs. More...
 
std::string stringify (const Struct::TextMap &textMapToStringify)
 Converts a text map into a JSON array with corresponding objects containing [key, value] pairs. More...
 
std::string stringify (const rapidjson::Value &value)
 Stringifies a JSON value using the RapidJSON. More...
 
std::string stringify (const jsoncons::json &json)
 Stringifies a JSON value using jsoncons. More...
 

Parsing

std::string cleanCopy (std::string_view json)
 Copies and cleans the given JSON code to prepare it for parsing. More...
 
rapidjson::Document parseRapid (std::string_view json)
 Parses JSON code using RapidJSON. More...
 
jsoncons::json parseCons (std::string_view json)
 Parses JSON code using jsoncons. More...
 
Struct::TextMap parseTextMapJson (std::string_view json)
 Parses JSON code using RapidJSON and converts it into a text map. More...
 
std::vector< std::pair< std::size_t, std::size_t > > parsePosLenPairsJson (std::string_view json)
 Parses JSON code using RapidJSON and converts it into [pos;length] pairs. More...
 

Memory

static void free (rapidjson::Document &target)
 Frees memory by swapping. More...
 

Detailed Description

Namespace for global JSON helper functions.

Function Documentation

◆ cleanCopy()

std::string crawlservpp::Helper::Json::cleanCopy ( std::string_view  json)
inline

Copies and cleans the given JSON code to prepare it for parsing.

Removes control characters and corrects escape sequences in the given JSON code.

If the given JSON code is empty, an empty string will be returned.

Note
In standard JSON, allowed escape sequence names are: " \ / b f n r t, as well as u + 4 hex digits.
Parameters
jsonA string view containing the JSON code to be copied and cleaned.
Returns
A copy of the string containing the cleaned JSON code.

References unicodeEscapeDigit1, unicodeEscapeDigit2, unicodeEscapeDigit3, unicodeEscapeDigit4, and unicodeEscapeLength.

Referenced by parseCons(), and parseRapid().

◆ free()

void crawlservpp::Helper::Json::free ( rapidjson::Document &  target)
inlinestatic

◆ parseCons()

jsoncons::json crawlservpp::Helper::Json::parseCons ( std::string_view  json)
inline

Parses JSON code using jsoncons.

If the initial parsing fails and the JSON code contains backslashes, it tries again after escaping these backslashes – i.e., replacing '\' with '\'.

For more information about jsoncons, see its GitHub repository.

Parameters
jsonA string view containing the JSON code to parse.
Returns
A jsoncons object containing the parsed JSON.
Exceptions
Json::Exceptionif an error occurs while parsing the given JSON code.
See also
parseRapid

References cleanCopy(), and crawlservpp::Helper::Strings::replaceAll().

Referenced by crawlservpp::Query::Container::addSubSetsFromQueryOnSubSet(), crawlservpp::Query::Container::getBoolFromQuery(), crawlservpp::Query::Container::getBoolFromQueryOnSubSet(), crawlservpp::Query::Container::getMultiFromQuery(), crawlservpp::Query::Container::getMultiFromQueryOnSubSet(), crawlservpp::Query::Container::getSingleFromQuery(), crawlservpp::Query::Container::getSingleFromQueryOnSubSet(), crawlservpp::Query::Container::reserveForSubSets(), crawlservpp::Query::Container::setSubSetsFromQuery(), and crawlservpp::Main::Server::tick().

◆ parsePosLenPairsJson()

std::vector< std::pair< std::size_t, std::size_t > > crawlservpp::Helper::Json::parsePosLenPairsJson ( std::string_view  json)
inline

Parses JSON code using RapidJSON and converts it into [pos;length] pairs.

If the given JSON code is empty, an empty array will be returned.

For more information about RapidJSON, see its GitHub repository.

Parameters
jsonA string view containing the JSON code to be parsed and converted into pairs of numbers.
Returns
The pairs parsed from the given JSON code.
Exceptions
Json::Exceptionif the given string view does not contain valid JSON code or the contained JSON code does not describe a valid array of [pos;length] pairs.
See also
parseRapid

References parseRapid().

Referenced by crawlservpp::Module::Analyzer::Database::checkSources().

◆ parseRapid()

rapidjson::Document crawlservpp::Helper::Json::parseRapid ( std::string_view  json)
inline

◆ parseTextMapJson()

Struct::TextMap crawlservpp::Helper::Json::parseTextMapJson ( std::string_view  json)
inline

Parses JSON code using RapidJSON and converts it into a text map.

If the given JSON code is empty, an empty text map will be returned.

For more information about RapidJSON, see its GitHub repository.

Parameters
jsonA string view containing the JSON code to be parsed and converted into a text map.
Returns
The text map parsed from the given JSON code.
Exceptions
Json::Exceptionif the given string view does not contain valid JSON code or the contained JSON code does not describe a valid text map.
See also
parseRapid, Struct::TextMap

References parseRapid().

Referenced by crawlservpp::Module::Analyzer::Database::checkSources().

◆ stringify() [1/7]

std::string crawlservpp::Helper::Json::stringify ( const std::vector< std::string > &  vectorToStringify)
inline

Stringifies a vector of strings into one string containing a JSON array.

Uses RapidJSON for conversion into JSON. For more information about RapidJSON, see its GitHub repository.

Parameters
vectorToStringifyA const reference to the vector of strings to be combined and converted into valid JSON code.
Returns
The copy of a string containing valid JSON code representing an array contining all strings in the given vector.

Referenced by crawlservpp::Module::Analyzer::Database::checkSources(), crawlservpp::Main::Database::duplicateWebsite(), crawlservpp::Query::JsonPointer::getAll(), crawlservpp::Query::JsonPointer::getFirst(), crawlservpp::Module::Parser::Thread::onReset(), crawlservpp::Module::Extractor::Thread::onReset(), crawlservpp::Query::Container::reserveForSubSets(), crawlservpp::Main::Server::tick(), and crawlservpp::Module::Analyzer::Thread::uploadResult().

◆ stringify() [2/7]

std::string crawlservpp::Helper::Json::stringify ( const std::string &  stringToStringify)
inline

Converts a string into a JSON array with the string as the only element inside it.

Uses RapidJSON for conversion into JSON. For more information about RapidJSON, see its GitHub repository.

Note
A string view cannot be used, because the underlying API requires the constant reference to a string.
Parameters
stringToStringifyA const reference to the string to be converted into a JSON array.
Returns
The copy of a string containing valid JSON code representing an array containing the given string as its only element.

◆ stringify() [3/7]

std::string crawlservpp::Helper::Json::stringify ( const char *  stringToStringify)
inline

Converts a string into a JSON array with the string as the only element inside it.

Uses RapidJSON for conversion into JSON. For more information about RapidJSON, see its GitHub repository.

Note
A string view cannot be used, because the underlying API requires the constant reference to a string.
Parameters
stringToStringifyA const pointer to a null-terminated string to be converted into a JSON array.
Returns
The copy of a string containing valid JSON code representing an array containing the given string as its only element.

References stringify().

◆ stringify() [4/7]

std::string crawlservpp::Helper::Json::stringify ( const std::vector< std::vector< std::pair< std::string, std::string >>> &  vectorToStringify)
inline

Converts a vector of vectors of string pairs into a JSON array with corresponding objects containing [key, value] pairs.

Uses RapidJSON for conversion into JSON. For more information about RapidJSON, see its GitHub repository.

Parameters
vectorToStringifyA const reference to the vector containing vectors of string pairs, each of which represents a [key, value] pair.
Returns
The copy of a string containing valid JSON code representing an array of objects containing the given [key, value] pairs.

◆ stringify() [5/7]

std::string crawlservpp::Helper::Json::stringify ( const Struct::TextMap textMapToStringify)
inline

Converts a text map into a JSON array with corresponding objects containing [key, value] pairs.

Uses RapidJSON for conversion into JSON. For more information about RapidJSON, see its GitHub repository.

Parameters
textMapToStringifyA const reference to the text map to be represented as [key, value] pairs.
Returns
The copy of a string containing valid JSON code representing an array of objects containing the [key, value] pairs of the text map.
See also
Struct::TextMap, Module::Analyzer::Database::createCorpus

References crawlservpp::Helper::Utf8::length().

◆ stringify() [6/7]

std::string crawlservpp::Helper::Json::stringify ( const rapidjson::Value &  value)
inline

Stringifies a JSON value using the RapidJSON.

For more information about RapidJSON, see its GitHub repository.

Parameters
valueThe RapidJSON value to be stringified.
Returns
A copy of the string containing the representation of the given JSON value.

◆ stringify() [7/7]

std::string crawlservpp::Helper::Json::stringify ( const jsoncons::json &  json)
inline

Stringifies a JSON value using jsoncons.

For more information about jsoncons, see its GitHub repository.

Parameters
jsonThe jsoncons value to be stringified.
Returns
A copy of the string containing the representation of the given JSON value.

Referenced by stringify().

Variable Documentation

◆ numDebugChars

constexpr auto crawlservpp::Helper::Json::numDebugChars {25}
inline

The number of characters to show before and behind a JSON error.

Referenced by parseRapid().

◆ unicodeEscapeDigit1

constexpr auto crawlservpp::Helper::Json::unicodeEscapeDigit1 {2}
inline

The offset of the first Unicode character digit in JSON code (from the '\').

Referenced by cleanCopy().

◆ unicodeEscapeDigit2

constexpr auto crawlservpp::Helper::Json::unicodeEscapeDigit2 {3}
inline

The offset of the second Unicode character digit in JSON code (from the '\').

Referenced by cleanCopy().

◆ unicodeEscapeDigit3

constexpr auto crawlservpp::Helper::Json::unicodeEscapeDigit3 {4}
inline

The offset of the third Unicode character digit in JSON code (from the '\').

Referenced by cleanCopy().

◆ unicodeEscapeDigit4

constexpr auto crawlservpp::Helper::Json::unicodeEscapeDigit4 {5}
inline

The offset of the fourth Unicode character digit in JSON code (from the '\').

Referenced by cleanCopy().

◆ unicodeEscapeLength

constexpr auto crawlservpp::Helper::Json::unicodeEscapeLength {6}
inline

The length of an escaped Unicode character in JSON code (including the '\u').

Referenced by cleanCopy().