json/docs/mkdocs/docs/features/json_view.md

Zero-copy JSON views

#!cpp <nlohmann/json_view.hpp> adds a read-only, non-owning way to look at a parsed JSON text, as an alternative to building a basic_json tree with parse().

The problem

basic_json::parse() builds a tree of basic_json values: one allocation for every array and object, and every string copied into its own std::string. That is the right trade-off when the program goes on to read and write the value freely, but it does more work than necessary when only a small part of a large JSON text is actually needed, or when the same text is parsed over and over (many small messages, for instance) and most of the resulting tree is thrown away almost immediately.

The idea

basic_json_document::parse() parses the same JSON grammar, with the same options, but instead of a tree it builds a flat index of the values it found: one 16-byte entry per value (and one per object key), in document order. Integers are converted while parsing and stored in their entry. Floating-point numbers and strings are referenced in place in the source text: a float is converted when it is read, and only strings that contain escape sequences are decoded, into one shared buffer owned by the document.

basic_json_view is a small, trivially copyable handle (two pointers) into that index. It gives you the read-only, type-inspection part of the basic_json interface -- type() and the is_*() predicates, size()/empty() -- as well as element access (operator[], at, front/back), lookup (find, contains, count), and iteration (begin/end, items) -- without ever allocating a basic_json value. When you do need an actual basic_json value for a subtree, materialize() builds exactly the one parse() would have produced for it.

How to use it

Include <nlohmann/json_view.hpp> in addition to (or instead of) <nlohmann/json.hpp>. Parse into a json_document, inspect its root(), and materialize() when you need a real value. materialize() works on any view, not just the root, so you can turn just the subtree you need into a value:

??? example "Example: parse a document, inspect its root, and materialize it"

```cpp
--8<-- "examples/json_document.cpp"
```

Output:

```json
--8<-- "examples/json_document.output"
```

ordered_json_document is the equivalent for ordered_json, just as ordered_json is to json.

Ownership and lifetime

A document either borrows the text it was parsed from, or owns its own copy of it; call owns_source() to find out which happened. parse() decides this from the value category and type of its argument (an lvalue #!cpp std::string is borrowed; an rvalue #!cpp std::string is moved in, owned without a copy; a stream is read into an owned buffer; and so on -- see parse's Notes for the full table). parse_copy() always owns a copy, regardless of the input.

!!! warning "A borrowed document depends on your buffer"

If a document borrows its text, that text **must outlive the document** (and every view taken from it) and must
stay **unmodified** for as long as the document exists, since floating-point values and unescaped strings are
read from it lazily. Reading or writing through a view after the underlying buffer is gone is undefined behavior,
exactly as it would be for a dangling `#!cpp std::string_view`.

A view is valid only while all of the following hold:

• the document is alive,
• the document has not been re-parsed since the view was taken (with read() or parse() into it),
• the document has not had shrink_to_fit() called on it since, and
• if the document borrows its source text, that text is still alive and unmodified.

Moving the basic_json_document<> itself is fine and does not invalidate its views: the index is a separate heap allocation that keeps its address across the move. Take a fresh view from root() whenever any of the other conditions above was not met.

A view can dangle, exactly like a #!cpp std::string_view: nothing stops you from using it after its document is gone. This is deliberate. A view is two pointers and trivially copyable; making it safe (a #!cpp std::shared_ptr in the document and a #!cpp std::weak_ptr in each view) would make it larger, add reference counting to every copy, and a lock() to every access -- exactly the costs the view exists to avoid. What is caught is the obvious case: because a view dies with its document, root() is not callable on a temporary document: #!cpp auto v = json_document::parse(text).root(); does not compile. Give the document a name first.

??? example "Example: borrowed and owned documents, and when views become invalid"

```cpp
--8<-- "examples/json_view_ownership.cpp"
```

Output:

```json
--8<-- "examples/json_view_ownership.output"
```

What is the same as parse()

• Accept/reject. accept() and parse() accept and reject exactly the same inputs as basic_json::accept()/basic_json::parse(), with the same ignore_comments and ignore_trailing_commas options.
• Errors. A failing parse throws the same exception -- the same id, message, and position -- because on failure the library's own parser is run on the same bytes to produce the diagnostic. Valid input never pays for this, and the replay builds no values: it costs about a lexing pass up to the error. #!cpp allow_exceptions == false skips the replay and gives a discarded document instead of throwing, just as it gives a discarded value for #!cpp basic_json::parse().
• Number classification. An integer literal that does not fit into the 64-bit integer type becomes a floating-point value, exactly as it does for #!cpp basic_json::parse().
• Macros. JSON_STRICT_NUL_HANDLING and JSON_NOEXCEPTION/JSON_THROW_USER behave the same way they do for <nlohmann/json.hpp>. These are the macros that change what the parser accepts or how it reports errors. The others either do not apply (JSON_DIAGNOSTICS needs parent pointers in basic_json values) or only affect the values materialize() produces, through BasicJsonType.

What is different

• Only 64-bit integers. basic_json_document<BasicJsonType> requires BasicJsonType::number_integer_t and number_unsigned_t to both be 64 bits wide; this is a compile-time #!cpp static_assert.
• A 4 GiB input limit. An input of 4294967280 bytes (4 GiB minus 16 bytes) or more throws out_of_range.416, a limit #!cpp basic_json::parse() does not have. It keeps an index entry at 16 bytes: offsets and counts are 32 bits wide. The limit is 16 bytes below 2^32, because the parser reads in 16-byte steps, and a position one step past the end must still fit.
• A stream is always read to its end. There is no partial/streaming read of an #!cpp std::istream: the stream is read to its end into a buffer the document owns, and parsing starts after that.
• No source positions on materialize(). Even with JSON_DIAGNOSTIC_POSITIONS enabled, materialize() does not set them. The index has the start offset of every value, but not the end of arrays, objects, and escaped strings, which the positions also need.
• Objects iterate in document order. begin()/ end() and items() visit an object's members in the order they appear in the source text. basic_json's default object_t is a std::map, which sorts by key, so iterating a materialize()d value can print members in a different order than iterating the view they came from.
• Chained access is safe. operator[] with a missing key, an index out of range, or an unresolvable JSON pointer returns a discarded view, and operator[] on a discarded view returns a discarded view without throwing: #!cpp v["a"]["b"][0] can be tested once at the end. Type errors on values that exist (a key on an array, an index on an object) still throw, and at throws for every missing value.
• Duplicate keys: lookups find the first member, parse() keeps the last. See Duplicate keys below.
• No JSON_DIAGNOSTICS path. Exceptions thrown by basic_json_view's own element access and lookup functions never carry the JSON Pointer path JSON_DIAGNOSTICS would otherwise add: the view has no basic_json value to point at, so the exception is created without one, regardless of how BasicJsonType was built.
• Ordering comparisons are not provided by basic_json_view -- there is no #!cpp operator<. operator== and operator!= are provided, though: two views, or a view and a BasicJsonType value, compare equal exactly when materialize() or parse() would produce equal values for them, without ever building a tree to do it. For ordering, too, materialize() is the way to get a value you can compare.

Duplicate keys

!!! warning "A lookup in a view and in the parsed value can give different answers"

If an object in the source text repeats a key, [`operator[]`](../api/basic_json_view/operator%5B%5D.md),
[`at`](../api/basic_json_view/at.md), [`find`](../api/basic_json_view/find.md),
[`value`](../api/basic_json_view/value.md), [`contains`](../api/basic_json_view/contains.md), and JSON pointer
resolution all return the **first** member with that key. `basic_json::parse()` instead keeps the **last** value
of a repeated key, so for `#!cpp {"a":1,"a":2}`, `#!cpp view["a"]` is `#!cpp 1` while `#!cpp parse(text)["a"]` is
`#!cpp 2`. [`count`](../api/basic_json_view/count.md) counts every member with the key, so it is `#!cpp 2` here.

begin()/end() and items() visit every occurrence, in document order, and size() counts all of them, so #!cpp view.size() can be larger than #!cpp view.materialize().size().

Why the first? A lookup can stop as soon as it finds a match. Returning the last member would force every lookup to scan all members of the object, even when the key is found at the very first one: this made lookups in small objects 1.6 to 3.4 times slower. The hash index of the keys of objects with 128 members or more leads to the first member of a key as well. Other zero-copy parsers that index the source text, such as yyjson and simdjson, also return the first member. RFC 8259 only says that names within an object SHOULD be unique and that the behavior of a receiver that sees duplicates is unpredictable, so neither choice is wrong.

What stays the same as parse()? materialize() and get<std::map<...>>() replay every member in order, so they keep the last value exactly like #!cpp basic_json::parse() (at the position of the first occurrence of the key, for an ordered_json).

How do I get the value parse() would give? Either call #!cpp view.materialize() and look the key up in the result, or iterate the members and keep the last match:

// the last member with the key "a", as parse() would keep it
json_view last;
for (const auto item : view.items())
{
    if (item.key() == "a")
    {
        last = item.value();
    }
}

If you cannot trust the source text to have unique keys, check size() against the number of distinct keys, or reject duplicates when iterating.

Getting values out without copying

get<T>() converts many T directly from the flat index, without ever building a basic_json value for the conversion: #!cpp bool, arithmetic types, #!cpp std::nullptr_t, #!cpp std::string/other #!cpp std::basic_strings (copied once), basic_json/ordered_json (via materialize()), basic_json_view itself, #!cpp std::vector<U>, and #!cpp std::map/#!cpp std::unordered_map with string-like keys. Every other type -- #!cpp std::list, #!cpp std::pair, #!cpp std::array, enumerations, user types with a from_json() -- goes through materialize().get<T>() instead: the subtree is built into a real basic_json value first, exactly as parse() would, and converted from there.

Two conversions never copy at all:

• get_string() (equivalently, #!cpp get<string_view_t>()) returns a string as a string_view_t pointing into the document's source() text -- or, for a string that contains escape sequences, into the document's own buffer of decoded strings -- instead of allocating a new #!cpp std::string.
• number_token() returns a number exactly as it was written in the source, e.g. #!cpp "1.50", #!cpp "1E2", or an integer with more digits than any number type holds, instead of rounding it into a #!cpp double/#!cpp int64_t the way #!cpp get<T>() (and basic_json::parse()) would.

Both results are only valid as long as the view -- and, for a string with no escapes, the borrowed source text -- is.

Writing a view back

dump() serializes a view directly from the flat index, without ever building a basic_json value. An object's members are written in document order, not sorted by key, and every occurrence of a repeated key is written, not only the last one -- the same two ways iteration already differs from a materialize()d value, see Duplicate keys. #!cpp materialize().dump() gives a different result in both respects for a json_view.

By default, numbers are written the way basic_json::dump() would. number_format::source instead copies every number exactly as it was written in the source text -- a price like #!cpp 19.90, a long order or account ID with more digits than any number type holds, or a high-precision coordinate -- something basic_json cannot do at all, since parsing already reduces a number to its parsed #!cpp double/#!cpp int64_t value.

operator<< writes a view to a stream the way basic_json's does, using the stream's width/fill for indentation.

Choosing between json, ordered_json, the SAX interface, and json_view

	json / ordered_json	SAX interface	json_document / json_view
Ownership	owns every value	owns nothing; you decide what to keep, in your handler	borrows or owns the text; the index is always owned by the document
Mutability	freely mutable	not applicable (a one-shot event stream)	read-only
What you get	a full tree you can read, write, and keep as long as you like	a sequence of callbacks; whatever your handler builds from them	a flat index plus, on demand, materialize()d json/ordered_json values for the parts you actually use
Typical use	general-purpose JSON handling: config, request/response bodies you build or modify, anything you hold onto	validating or projecting a text into your own data structure without ever holding the whole thing as JSON	large or high-volume input where you only need part of it, or need it repeatedly, and can keep the source text (or a copy) alive for as long as the document lives

Version history

• Added in version 3.13.0.