jsonl Stringer - uhop/stream-json GitHub Wiki
(Since 1.6.0) jsonl/Stringer is a Transform stream. It consumes a stream of JavaScript objects and converts it to a JSONL (AKA NDJSON) file. It is very useful when you want to edit a stream with filters and custom code, and save it back to a file.
Tip: This module delegates to
stream-chain/jsonl/stringerStream. You can use it directly for the same result, plus additional options (prefix,suffix,space,emptyValue).
jsonl/Stringer has three major differences with Stringer:
- It consumes a stream of JavaScript objects, while
Stringerconsumes a stream of raw tokens. - It produces a valid JSONL file, where all values are separated by newline symbols.
- It is faster.
Introduction
const jsonlStringer = require('stream-json/jsonl/stringer.js');
const jsonlParser = require('stream-json/jsonl/parser.js');
const {chain} = require('stream-chain');
const fs = require('fs');
const zlib = require('zlib');
chain([
fs.createReadStream('data.jsonl.gz'),
zlib.createGunzip(),
jsonlParser(),
// select only active values
data => (data.value.active ? data.value : null),
jsonlStringer(),
zlib.createGzip(),
fs.createWriteStream('edited.jsonl.gz')
]);
API
jsonl/Stringer has no special API. It is based on Transform with its writable part operating in object mode and its readable part operating in text mode.
constructor(options)
options is an optional object described in details in node.js' Stream documentation. Additionally, the following optional custom flags are recognized:
replacer can be one of two optional objects:
- It can be a function, which takes two parameters and returns a value.
- It can be an array of strings or numbers to define a whitelist for object properties.
- See JSON.stringify() for more details.
Static methods and properties
jsonlStringer.stringer(options)
Alias of the factory function.
jsonlStringer.asStream(options)
Same as jsonlStringer(options) — since jsonlStringer already returns a stream, this is an identity alias for API consistency.
const jsonlStringer = require('stream-json/jsonl/stringer.js');
const jsonlParser = require('stream-json/jsonl/parser.js');
const {chain} = require('stream-chain');
const fs = require('fs');
const pipeline = chain([fs.createReadStream('sample.jsonl'), jsonlParser.asStream(), data => data.value, jsonlStringer()]);
The code above roundtrips data from sample.jsonl.