What is NDJSON (JSON Lines) and when should I use it?

NDJSON (newline-delimited JSON), also called JSON Lines or JSONL, is a text format where every line is one complete JSON value, usually an object, and lines are separated by \n. Use the NDJSON format when you have many records that you write or read one at a time: logs, event streams, bulk imports and large exports. For a single config file or an API response, plain JSON is still the better choice.

What the NDJSON format looks like

An NDJSON file is a stack of one-line JSON documents with no surrounding array and no commas between records. Here are three log events:

{"level":"info","msg":"server started","env":"prod"}
{"level":"warn","msg":"slow query","table":"orders"}
{"level":"error","msg":"db timeout","retry":true}

The JSON Lines site has three rules: UTF-8, a valid JSON value on each line, and \n as the terminator. null counts as a value. A blank line doesn't. No byte order mark.

The NDJSON spec says the JSON texts "MUST NOT contain newlines or carriage returns", so pretty-printed objects break the file. Newlines inside strings are fine, because JSON escapes them as \n.

Is NDJSON the same as JSON Lines?

NDJSON and JSON Lines describe the same bytes in almost every real file. Google's BigQuery docs say it directly: "The ndJSON format is the same as the JSON Lines format."

The differences are small, but a strict tool will reject a file over them:

When that happens, check the extension or Content-Type it expects, then whether the file ends with a newline.

Why JSON Lines beats one big JSON array

The format exists because one record per line lets you process data as a stream. A JSON array isn't valid until the parser reaches the closing ]. In JSON Lines, every line is already a finished document.

• Adding a record means writing one line to the end of the file. Nothing gets rewritten.
• Memory stays flat when you stream, whether the file holds ten rows or ten million.
• A bad record only breaks its own line, and the spec says a parser "SHOULD raise an error" for it rather than guessing.
• JSON Lines recommends gzip or bzip2 for compression, giving .jsonl.gz or .jsonl.bz2 files.

head, wc -l and grep all work per record, which they can't do on a single minified array.

When to use the NDJSON format

Use NDJSON when your data is a list of independent records and at least one side processes them one at a time.

1. Logs and audit trails. Each event is written as it happens, and log shippers read line by line.
2. Bulk APIs. The Elasticsearch bulk API takes newline-delimited JSON, accepts application/x-ndjson, and requires the final line to end with a newline character. Actions and sources must not be pretty printed.
3. BigQuery loads newline-delimited JSON from Cloud Storage, and "each JSON object must be on a separate line in the file."
4. Machine learning datasets and exports, where each line is one training example or one exported row.

The final-newline rule bites often. Build a bulk body with lines.join("\n"), send it to Elasticsearch, and the request is rejected because the last action has no trailing \n. Write lines.join("\n") + "\n".

When not to use it

Skip JSON Lines when the data is one document, not a list. Config files, API responses for a single resource, and anything a person edits by hand are clearer as regular JSON. Our guide to what JSON format is covers that case.

The limitation: an NDJSON file is not valid JSON as a whole. MDN's JSON.parse page says a string that breaks the JSON grammar throws a SyntaxError, and a second top-level object on line two breaks it. Python's json.loads raises a JSONDecodeError in the same spot, per the Python json docs. Our own validator reports the JSON.parse error too, so paste one line at a time to check a single record.

How to read and write NDJSON in code

Reading is a loop over lines, and writing is one serialize call per record plus a newline. In Python:

import json

with open("events.jsonl", encoding="utf-8") as f:
    for line in f:
        if line.strip():
            event = json.loads(line)

with open("out.jsonl", "w", encoding="utf-8") as f:
    for row in rows:
        f.write(json.dumps(row) + "\n")

In Node.js, JSON.stringify writes a single line unless you pass an indent, so records.map(r => JSON.stringify(r)).join("\n") + "\n" is a valid file. The Python reader skips blank lines to be lenient. The JSON Lines spec treats them as invalid.

Convert between JSON and NDJSON with jq

jq turns a JSON array into JSON Lines and back with one flag each. The jq manual says -c puts "each JSON object on a single line", and -s reads "the entire input stream into a large array".

# JSON array -> NDJSON
jq -c '.[]' users.json > users.jsonl

# NDJSON -> JSON array
jq -s '.' users.jsonl > users.json

A single object becomes one valid line once it's minified, which a JSON minifier also does. For more jq formatting options, see pretty-printing JSON with jq. If the records are flat, converting JSON to CSV may suit a spreadsheet better.

NDJSON vs JSON text sequences

JSON text sequences are a third, standardized option that people mix up with NDJSON. RFC 7464 defines them with the media type application/json-seq. Each JSON text starts with an ASCII Record Separator (0x1E) and ends with a Line Feed (0x0A), and the LF acts as a "canary" to detect truncation.

My take: use JSON Lines for files and NDJSON's application/x-ndjson for HTTP, because that's what Elasticsearch and BigQuery speak. JSON text sequences are the cleaner standard on paper. In practice few tools expect the 0x1E byte, and most editors don't show it, so debugging is harder.

FAQ

Is NDJSON the same as JSONL?

In practice, yes. Both put one JSON value on each line, separated by a newline. The specs differ in small details: the file extension (.ndjson vs .jsonl), the media type, and how blank lines are treated.

Is an NDJSON file valid JSON?

No, not as a whole. Each line is valid JSON on its own, but a standard parser such as JSON.parse expects one document and throws an error when it meets the second line.

What MIME type should I use for NDJSON?

The NDJSON spec says application/x-ndjson. JSON Lines mentions application/jsonl but notes it is not yet standardized.

Can NDJSON contain pretty-printed JSON?

No. The NDJSON spec says JSON texts must not contain newlines or carriage returns, so every record has to be written on one line.

How do I convert a JSON array to NDJSON?

With jq, run jq -c '.[]' on the file. The -c flag puts each JSON object on a single line.

Sources

• JSON Lines
• NDJSON specification
• RFC 7464: JavaScript Object Notation (JSON) Text Sequences
• Elasticsearch bulk API
• BigQuery: Loading JSON data from Cloud Storage
• jq manual
• Python json module
• MDN: JSON.parse()