jso

jsonl

Transform a stream of JSON into a stream of Line Delimited JSON

Showing:

Popularity

Downloads/wk

187

GitHub Stars

20

Maintenance

Last Commit

7yrs ago

Contributors

0

Package

Dependencies

2

License

MIT

Type Definitions

Tree-Shakeable

No?

Categories

Readme

jso\nl

Transform a stream of JSON into a stream of Line Delimited JSON

Install

$ npm install --save jsonl

Use

From Buffers

var fs = require("fs")
var jsonl = require("jsonl")

fs.createReadStream("./in.json")
  .pipe(jsonl())
  .pipe(fs.createWriteStream("./out.json"))

in.json

[{"test":"value"},{"test":"value"},{"test":"value"},{"test":"value"}]

out.json

{"test":"value"}
{"test":"value"}
{"test":"value"}
{"test":"value"}

From Objects

var fs = require("fs")
var jsonl = require("jsonl")
var through = require("through2")
var stream = through.obj()

stream.pipe(jsonl({toBufferStream:true}))
  .pipe(fs.createWriteStream("./out.json"))

stream.push({test:"value"})
stream.push({test:"value"})
stream.push({test:"value"})
stream.push({test:"value"})
stream.end()

out.json

{"test":"value"}
{"test":"value"}
{"test":"value"}
{"test":"value"}

Depth

To get the results you expect, you will likely need to know the structure of your incoming data. You may have to pass a depth property, which corresponds to the layer of the property in a serialized, nested JSON object.

By default, jsonl will use a depth of 1 when reading data from a Buffer stream (expecting objects to be nested in an array), and a depth of 0 from a stream in object mode.

/*0*/[
/*1*/  {
/*2*/    test: "value"
/*1*/  },
/*1*/  {
/*2*/    test: "value"
/*1*/  }
/*0*/]

Plucking

To filter the incoming data based on properties, you can select specific fields to be plucked out of the incoming object.

* You will need to specify a depth property for the nested level of the property.

var fs = require("fs")
var jsonl = require("jsonl")

fs.createReadStream("./in.json")
  .pipe(jsonl({pluck:["category"], depth:2}))
  .pipe(fs.createWriteStream("./out.json"))

in.json

[{"category": "cactus heights", "question":"?", "answer": "!"},{"category": "giraffe shoe sizes", "question":"?", "answer": "!"}]

out.json

{"category":"cactus heights"}
{"category":"giraffe shoe sizes"}

API

var jsonl = require("jsonl")([opts])

opts.depth

  • Type: Number (default: 1)

The depth of the objects in the incoming data to pluck out. This is what you want for an array of objects, such as:

[{"this":"that"},{"this":"that"}]

opts.objectMode

  • Type: Boolean (default: false)

Convert data into an object stream.

opts.pluck

  • Type: Array|String (default: [])

Only return select properties from JSON objects.

opts.separator

  • Type: String (default: \n)

String to separate object data with.

opts.toBufferStream

  • Type: Boolean (default: false)

Set this to true when you have an object stream that you would like converted to a stream of line delimited JSON buffers.

If set, this defaults opts.depth to 0, but can still be overridden.

License

MIT © Stephen Sawchuk

Rate & Review

Great Documentation0
Easy to Use0
Performant0
Highly Customizable0
Bleeding Edge0
Responsive Maintainers0
Poor Documentation0
Hard to Use0
Slow0
Buggy0
Abandoned0
Unwelcoming Community0
100
No reviews found
Be the first to rate

Alternatives

No alternatives found

Tutorials

No tutorials found
Add a tutorial