58 buffer_size = 1024 * 1024;
59 buffer.resize(buffer_size);
60 buffer_start = &buffer[0];
61 size_t bytes = source->
fetch(&buffer[0], buffer_size);
62 buffer_end = buffer_start + bytes;
virtual ~instream_document_unicoil_json()
Destructor.
Definition: instream_document_unicoil_json.h:72
std::shared_ptr< instream > source
If this object is reading from another instream then this is that instream.
Definition: instream.h:48
Read data from an input stream.
Definition: instream.h:45
Container class representing a document through the indexing pipeline.
Definition: document.h:31
static void unittest(void)
Unit test this class.
Definition: instream_document_unicoil_json.cpp:147
size_t buffer_size
Current size of the buffer.
Definition: instream_document_unicoil_json.h:42
virtual void read(document &buffer)
Read the next document from the source instream into document.
Definition: instream_document_unicoil_json.cpp:25
Base class for reading data from some input source.
std::string buffer
An internal buffer used to store lines.
Definition: instream_document_unicoil_json.h:43
instream_document_unicoil_json(std::shared_ptr< instream > &source)
Constructor.
Definition: instream_document_unicoil_json.h:55
const char * buffer_start
Pointer to the current working point in the buffer;.
Definition: instream_document_unicoil_json.h:44
Definition: compress_integer_elias_delta_simd.c:23
const char * buffer_end
Pointer to the end of the buffer.
Definition: instream_document_unicoil_json.h:45
Extract documents from a JSON formatted UniCOIL archive.
Definition: instream_document_unicoil_json.h:39
size_t fetch(void *buffer, size_t bytes)
fetch() generates a document object, sets its contents to the passed buffer, calls read() and returns...
Definition: instream.h:125