|
crawlserv++
[under development]
Application for crawling and analyzing textual content of websites.
|
Namespace for global byte operation helper functions. More...
Typedefs | |
| using | Bytes = std::vector< std::uint8_t > |
Constants | |
| constexpr auto | first {0} |
| Index of the first byte. More... | |
| constexpr auto | second {1} |
| Index of the second byte. More... | |
| constexpr auto | third {2} |
| Index of the third byte. More... | |
| constexpr auto | fourth {3} |
| Index of the fourth byte. More... | |
| constexpr auto | fifth {4} |
| Index of the fifth byte. More... | |
| constexpr auto | sixth {5} |
| Index of the sixth byte. More... | |
| constexpr auto | seventh {6} |
| Index of the seventh byte. More... | |
| constexpr auto | eighth {7} |
| Index of the eighth byte. More... | |
| constexpr auto | oneByteBits {8} |
| One byte in bits. More... | |
| constexpr auto | twoBytesBits {16} |
| Two bytes in bits. More... | |
| constexpr auto | threeBytesBits {24} |
| Three bytes in bits. More... | |
| constexpr auto | fourBytesBits {32} |
| Four bytes in bits. More... | |
| constexpr auto | fiveBytesBits {40} |
| Five bytes in bits. More... | |
| constexpr auto | sixBytesBits {48} |
| Six bytes in bits. More... | |
| constexpr auto | sevenBytesBits {56} |
| Seven bytes in bits. More... | |
| constexpr auto | sizeEight {8} |
| Size of eight bytes. More... | |
| constexpr auto | sizeFour {4} |
| Size of four bytes. More... | |
| constexpr auto | sizeTwo {2} |
| Size of two bytes. More... | |
Endianness | |
| bool | isBigEndian () noexcept |
| Returns whether the machine running this code uses big endianness. More... | |
| bool | isFloatBigEndian () noexcept |
| Returns whether the machine running this code uses big endianness for floating-point numbers. More... | |
Bytes-to-Number Conversion | |
| std::uint64_t | bytesToUInt64 (const Bytes &bytes, std::size_t &pos) |
| Retrieve an unsigned 64-bit number from a vector of bytes. More... | |
| std::uint64_t | bytesToUInt64 (const Bytes &bytes, std::size_t &pos, std::size_t len) |
| Retrieve an unsigned 64-bit number from a vector of bytes. More... | |
| std::int64_t | bytesToInt64 (const Bytes &bytes, std::size_t &pos) |
| Retrieve a signed 64-bit number from a vector of bytes. More... | |
| std::int64_t | bytesToInt64 (const Bytes &bytes, std::size_t &pos, std::size_t len) |
| Retrieve a signed 64-bit number from a vector of bytes. More... | |
| std::uint32_t | bytesToUInt32 (const Bytes &bytes, std::size_t &pos) |
| Retrieve an unsigned 32-bit number from a vector of bytes. More... | |
| std::int32_t | bytesToInt32 (const Bytes &bytes, std::size_t &pos) |
| Retrieve a signed 32-bit number from a vector of bytes. More... | |
| std::uint16_t | bytesToUInt16 (const Bytes &bytes, std::size_t &pos) |
| Retrieve an unsigned 16-bit number from a vector of bytes. More... | |
| std::int16_t | bytesToInt16 (const Bytes &bytes, std::size_t &pos) |
| Retrieve a signed 16-bit number from a vector of bytes. More... | |
| double | bytesToDouble (const Bytes &bytes, std::size_t &pos) |
| Retrieves a IEEE 754 double-precision binary floating-point number from a vector of bytes. More... | |
Number-to-Bytes Conversion | |
| std::array< std::uint8_t, sizeEight > | uInt64ToBytes (std::uint64_t number) |
| Converts an unsigned 64-bit number to an array of eight bytes. More... | |
| std::array< std::uint8_t, sizeEight > | int64ToBytes (std::int64_t number) |
| Converts a signed 64-bit number to an array of eight bytes. More... | |
| std::array< std::uint8_t, sizeFour > | uInt32ToBytes (std::uint32_t number) |
| Converts an unsigned 32-bit number to an array of four bytes. More... | |
| std::array< std::uint8_t, sizeFour > | int32ToBytes (std::int32_t number) |
| Converts an signed 32-bit number to an array of four bytes. More... | |
| std::array< std::uint8_t, sizeTwo > | uInt16ToBytes (std::uint16_t number) |
| Converts an unsigned 16-bit number to an array of two bytes. More... | |
| std::array< std::uint8_t, sizeTwo > | int16ToBytes (std::int16_t number) |
| Converts an signed 16-bit number to an array of two bytes. More... | |
| std::array< std::uint8_t, sizeEight > | doubleToBytes (double number) |
| Converts a floating-point number with double precision to an array of four bytes. More... | |
String Representation | |
| std::string | byteToHexString (std::uint8_t byte) |
| Converts a byte to a string containing the byte in hexadecimal format. More... | |
| std::string | charToString (char c) |
| Converts a character to a string. More... | |
Namespace for global byte operation helper functions.
| using crawlservpp::Helper::Bytes::Bytes = typedef std::vector<std::uint8_t> |
|
inline |
Retrieves a IEEE 754 double-precision binary floating-point number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the floating-point number contained in the bytes. The position will be updated after reading the floating-point number from the bytes. |
References eighth, fifth, first, fourth, isFloatBigEndian(), crawlservpp::Helper::Queue::reverse(), second, seventh, sixth, sizeEight, and third.
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Retrieve a signed 16-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
References bytesToUInt16().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Retrieve a signed 32-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
References bytesToUInt32().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Retrieve a signed 64-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
References bytesToUInt64().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Retrieve a signed 64-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
| len | The actual number of bytes used by the number. |
References bytesToUInt64().
|
inline |
Retrieve an unsigned 16-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
References first, isBigEndian(), oneByteBits, crawlservpp::Helper::Queue::reverse(), second, and sizeTwo.
Referenced by bytesToInt16(), and crawlservpp::Data::PickleDict::writeTo().
|
inline |
Retrieve an unsigned 32-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
References first, fourth, isBigEndian(), oneByteBits, crawlservpp::Helper::Queue::reverse(), second, sizeFour, third, threeBytesBits, and twoBytesBits.
Referenced by bytesToInt32(), and crawlservpp::Data::PickleDict::writeTo().
|
inline |
Retrieve an unsigned 64-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
References eighth, fifth, first, fiveBytesBits, fourBytesBits, fourth, isBigEndian(), oneByteBits, crawlservpp::Helper::Queue::reverse(), second, sevenBytesBits, seventh, sixBytesBits, sixth, sizeEight, third, threeBytesBits, and twoBytesBits.
Referenced by bytesToInt64(), and crawlservpp::Data::PickleDict::writeTo().
|
inline |
Retrieve an unsigned 64-bit number from a vector of bytes.
| bytes | Constant reference to a vector containing the bytes. |
| pos | Reference to the start position of the number contained in the bytes. The position will be updated after reading the number from the bytes. |
| len | The actual number of bytes used by the number. |
References eighth, fifth, first, fiveBytesBits, fourBytesBits, fourth, isBigEndian(), oneByteBits, crawlservpp::Helper::Queue::reverse(), second, sevenBytesBits, seventh, sixBytesBits, sixth, sizeEight, third, threeBytesBits, and twoBytesBits.
|
inline |
Converts a byte to a string containing the byte in hexadecimal format.
| byte | The byte to convert into a string. |
0xHH where HH is the hexadecimal value of the byte. Referenced by charToString(), and crawlservpp::Data::PickleDict::writeTo().
|
inline |
Converts a character to a string.
If printable, the character will be simply converted into a string. If escapable, the string representation of its C escape sequence will be returned. Otherwise, its hexadecimal representation will be returned.
The resulting string will be between one and four characters long.
| c | The character to convert. |
References byteToHexString().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Converts a floating-point number with double precision to an array of four bytes.
| number | The floating-point number to convert to bytes. |
References isFloatBigEndian(), and crawlservpp::Helper::Queue::reverse().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Converts an signed 16-bit number to an array of two bytes.
| number | The number to convert to bytes. |
References isBigEndian(), and crawlservpp::Helper::Queue::reverse().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Converts an signed 32-bit number to an array of four bytes.
| number | The number to convert to bytes. |
References isBigEndian(), and crawlservpp::Helper::Queue::reverse().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Converts a signed 64-bit number to an array of eight bytes.
| number | The number to convert to bytes. |
References isBigEndian(), and crawlservpp::Helper::Queue::reverse().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inlinenoexcept |
Returns whether the machine running this code uses big endianness.
Referenced by bytesToUInt16(), bytesToUInt32(), bytesToUInt64(), int16ToBytes(), int32ToBytes(), int64ToBytes(), uInt16ToBytes(), uInt32ToBytes(), and uInt64ToBytes().
|
inlinenoexcept |
Returns whether the machine running this code uses big endianness for floating-point numbers.
Referenced by bytesToDouble(), and doubleToBytes().
|
inline |
Converts an unsigned 16-bit number to an array of two bytes.
| number | The number to convert to bytes. |
References isBigEndian(), and crawlservpp::Helper::Queue::reverse().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Converts an unsigned 32-bit number to an array of four bytes.
| number | The number to convert to bytes. |
References isBigEndian(), and crawlservpp::Helper::Queue::reverse().
Referenced by crawlservpp::Data::PickleDict::writeTo().
|
inline |
Converts an unsigned 64-bit number to an array of eight bytes.
| number | The number to convert to bytes. |
References isBigEndian(), and crawlservpp::Helper::Queue::reverse().
Referenced by crawlservpp::Data::PickleDict::writeTo().
| constexpr auto crawlservpp::Helper::Bytes::eighth {7} |
Index of the eighth byte.
Referenced by bytesToDouble(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::fifth {4} |
Index of the fifth byte.
Referenced by bytesToDouble(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::first {0} |
Index of the first byte.
Referenced by bytesToDouble(), bytesToUInt16(), bytesToUInt32(), bytesToUInt64(), crawlservpp::Data::Corpus::combineContinuous(), crawlservpp::Module::Crawler::Database::isArchivedContentExists(), crawlservpp::Module::Analyzer::Algo::WordsOverTime::resetAlgo(), crawlservpp::Module::Analyzer::Algo::Assoc::resetAlgo(), crawlservpp::Module::Analyzer::Algo::AssocOverTime::resetAlgo(), and crawlservpp::Module::Analyzer::Algo::SentimentOverTime::resetAlgo().
| constexpr auto crawlservpp::Helper::Bytes::fiveBytesBits {40} |
Five bytes in bits.
Referenced by bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::fourBytesBits {32} |
Four bytes in bits.
Referenced by bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::fourth {3} |
Index of the fourth byte.
Referenced by bytesToDouble(), bytesToUInt32(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::oneByteBits {8} |
One byte in bits.
Referenced by bytesToUInt16(), bytesToUInt32(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::second {1} |
Index of the second byte.
Referenced by bytesToDouble(), bytesToUInt16(), bytesToUInt32(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::sevenBytesBits {56} |
Seven bytes in bits.
Referenced by bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::seventh {6} |
Index of the seventh byte.
Referenced by bytesToDouble(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::sixBytesBits {48} |
Six bytes in bits.
Referenced by bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::sixth {5} |
Index of the sixth byte.
Referenced by bytesToDouble(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::sizeEight {8} |
Size of eight bytes.
Referenced by bytesToDouble(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::sizeFour {4} |
Size of four bytes.
Referenced by bytesToUInt32().
| constexpr auto crawlservpp::Helper::Bytes::sizeTwo {2} |
Size of two bytes.
Referenced by bytesToUInt16().
| constexpr auto crawlservpp::Helper::Bytes::third {2} |
Index of the third byte.
Referenced by bytesToDouble(), bytesToUInt32(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::threeBytesBits {24} |
Three bytes in bits.
Referenced by bytesToUInt32(), and bytesToUInt64().
| constexpr auto crawlservpp::Helper::Bytes::twoBytesBits {16} |
Two bytes in bits.
Referenced by bytesToUInt32(), and bytesToUInt64().