Pick - uhop/stream-json GitHub Wiki
Pick
is a token item filter based on FilterBase. It selects objects from a stream ignoring the rest. Its filter is not called for subobjects of selected objects.
Unless Pick
selects just one object, it produces a stream of objects similar to JSON Streaming supported by Parser. Usually, the result is piped through StreamValues. Other streamers could be unsuitable.
Introduction
const Pick = require('stream-json/filters/Pick');
const {streamArray} = require('stream-json/streamers/StreamArray');
const {chain} = require('stream-chain');
const fs = require('fs');
// our data stream:
// {total: 123456789, meta: {...}, data: [...]}
// we are interested in 'data'
const pipeline = chain([
fs.createReadStream('sample.json'),
Pick.withParser({filter: 'data'}),
streamArray()
]);
pipeline.on('data', data => console.log(data));
More complex example:
const {pick} = require('stream-json/filters/Pick');
const {parser} = require('stream-json/Parser');
const {streamValues} = require('stream-json/streamers/StreamValues');
// our data stream: array of objects,
// each object looks like that:
// {total: 123456789, meta: {...}, data: [...]}
// we want the content of all 'data' subobjects
const pipeline = chain([
fs.createReadStream('sample.json'),
parser(),
pick({filter: /^\d+\.data\.\d+/}),
streamValues()
]);
pipeline.on('data', data => console.log(data));
API
Pick
has no special API. Based on FilterBase
it uses the following options:
filter
- If it returns a truthy value, the current object is streamed out with all its possible subobjects.
- It is called only for the following tokens:
startObject
,startArray
,startString
,startNumber
,stringValue
,numberValue
,nullValue
,trueValue
,falseValue
.
pathSeparator
once
See their definitions in FilterBase.
Static methods and properties
pick(options)
and make(options)
make()
and pick()
are two aliases of the factory function. It takes options described above, and return a new instance of Pick
. pick()
helps to reduce a boilerplate when creating data processing pipelines:
const {chain} = require('stream-chain');
const {parser} = require('stream-json/Parser');
const {pick} = require('stream-json/filters/Pick');
const fs = require('fs');
const pipeline = chain([
fs.createReadStream('sample.json'),
parser(),
pick({filter: 'data'})
]);
let objectCounter = 0;
pipeline.on('data', data => data.name === 'startObject' && ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));
make.Constructor
Constructor property of make()
(and pick()
) is set to Pick
. It can be used for indirect creating of filters or metaprogramming if needed.
withParser()
withParser()
takes one argument:
options
is an object described in Parser's options. It is used to initialize both streams (aParser
instance and a stream returned bymake()
).
It returns a stream produced by stream-chain, which wraps the pipeline. The most important utility of withParser()
is that it correctly sets object modes of the returned stream: object mode for the Readable part and text mode for the Writable part.
This static method is created using withParser() utility. It simplifies a case when a stream should be immediately preceded by a parser.
const Pick = require('stream-json/filters/Pick');
const fs = require('fs');
const pipeline = fs.createReadStream('sample.json')
.pipe(Pick.withParser({filter: 'data'}));
let objectCounter = 0;
pipeline.on('data', data => data.name === 'startObject' && ++objectCounter);
pipeline.on('end', () => console.log(`Found ${objectCounter} objects.`));