StreamArray - uhop/stream-json GitHub Wiki

StreamArray assumes that an input token stream represents an array of objects and streams out assembled JavaScript objects:

[1, "a", [], {}, true]
// StreamArray will produce an object stream:
{key: 0, value: 1}
{key: 1, value: 'a'}
{key: 2, value: []}
{key: 3, value: {}}
{key: 4, value: true}

As every streamer, it assumes that individual objects can fit in memory, but the whole file, or any other source, should be streamed.

Warning: it cannot be used with a stream of objects produced by a JSON Streaming source, or Pick if it picks more than one object.

Introduction

const {streamArray} = require('stream-json/streamers/stream-array.js');
const fs = require('fs');

const pipeline = fs.createReadStream('sample.json').pipe(streamArray.withParserAsStream());

pipeline.on('data', data => console.log(data));

API

Being based on StreamBase, StreamArray has no special API.

Static methods and properties

streamArray(options)

streamArray() is the factory function. It takes options described above and returns a function for use in chain():

const {chain} = require('stream-chain');
const {parser} = require('stream-json');
const {streamArray} = require('stream-json/streamers/stream-array.js');

const fs = require('fs');

const pipeline = chain([fs.createReadStream('sample.json'), parser(), streamArray()]);

let objectCounter = 0;
pipeline.on('data', () => ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));

streamArray.asStream(options)

Returns a Duplex stream (object-mode both sides) wrapping streamArray() for .pipe() usage.

withParser()

withParser() takes one argument:

  • options — combined Parser and streamer options. Passed to both the parser and streamArray().

Returns a Duplex stream (text-mode writable, object-mode readable) wrapping a parser() + streamArray() pipeline via stream-chain.

Built with the withParser() utility.

const {streamArray} = require('stream-json/streamers/stream-array.js');
const fs = require('fs');

const pipeline = fs.createReadStream('sample.json').pipe(streamArray.withParserAsStream());

let objectCounter = 0;
pipeline.on('data', () => ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));