StreamArray - uhop/stream-json GitHub Wiki
StreamArray
assumes that an input token stream represents an array of objects and streams out assembled JavaScript objects:
[1, "a", [], {}, true]
// StreamArray will produce an object stream:
{key: 0, value: 1}
{key: 1, value: 'a'}
{key: 2, value: []}
{key: 3, value: {}}
{key: 4, value: true}
As every streamer, it assumes that individual objects can fit in memory, but the whole file, or any other source, should be streamed.
Warning: it cannot be used with a stream of objects produced by a JSON Streaming source, or Pick if it picks more than one object.
Introduction
const StreamArray = require('stream-json/streamers/StreamArray');
const fs = require('fs');
const pipeline = fs.createReadStream('sample.json')
.pipe(StreamArray.withParser());
pipeline.on('data', data => console.log(data));
API
Being based on StreamBase, StreamArray
has no special API.
Static methods and properties
streamArray(options)
and make(options)
make()
and streamArray()
are two aliases of the factory function. It takes options described above, and return a new instance of StreamArray
. streamArray()
helps to reduce a boilerplate when creating data processing pipelines:
const {chain} = require('stream-chain');
const {parser} = require('stream-json');
const {streamArray} = require('stream-json/streamers/StreamArray');
const fs = require('fs');
const pipeline = chain([
fs.createReadStream('sample.json'),
parser(),
streamArray()
]);
let objectCounter = 0;
pipeline.on('data', () => ++objectCounter);
pipeline.on('end', console.log(`Found ${objectCounter} objects.`));
make.Constructor
Constructor property of make()
(and streamArray()
) is set to StreamArray
. It can be used for indirect creating of streamers or metaprogramming if needed.
withParser()
withParser()
takes one argument:
options
is an object described in Parser's options. It is used to initialize both streams (aParser
instance and a stream returned bymake()
).
It returns a stream produced by stream-chain, which wraps the pipeline. The most important utility of withParser()
is that it correctly sets object modes of the returned stream: object mode for the Readable part and text mode for the Writable part.
This static method is created using withParser() utility. It simplifies a case when a stream should be immediately preceded by a parser.
const StreamArray = require('stream-json/streamers/StreamArray');
const fs = require('fs');
const pipeline = fs.createReadStream('sample.json')
.pipe(StreamArray.withParser());
let objectCounter = 0;
pipeline.on('data', () => ++objectCounter);
pipeline.on('end', console.log(`Found ${objectCounter} objects.`));