lib/json
1
0
Fork 0
mirror of https://github.com/nlohmann/json.git synced 2025-07-31 10:24:23 +03:00
Code Activity

BJData optimized binary array type (#4513)

Browse Source
This commit is contained in:
Nebojša Cvetković
2025-01-07 17:09:19 +00:00
committed by GitHub
parent 60c48755e3
commit 2e50d5b2f3
7 changed files with 508 additions and 289 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/mkdocs/docs/api/basic_json/to_bjdata.md
View File

@ -4,13 +4,16 @@
// (1)
static std::vector<std::uint8_t> to_bjdata(const basic_json& j,
const bool use_size = false,
const bool use_type = false);
const bool use_type = false,
const bjdata_version_t version = bjdata_version_t::draft2);
// (2)
static void to_bjdata(const basic_json& j, detail::output_adapter<std::uint8_t> o,
const bool use_size = false, const bool use_type = false);
const bool use_size = false, const bool use_type = false,
const bjdata_version_t version = bjdata_version_t::draft2);
static void to_bjdata(const basic_json& j, detail::output_adapter<char> o,
const bool use_size = false, const bool use_type = false);
const bool use_size = false, const bool use_type = false,
const bjdata_version_t version = bjdata_version_t::draft2);
```
Serializes a given JSON value `j` to a byte vector using the BJData (Binary JData) serialization format. BJData aims to
@ -34,6 +37,9 @@ The exact mapping and its limitations is described on a [dedicated page](../../f
`use_type` (in)
: whether to add type annotations to container types (must be combined with `#!cpp use_size = true`); optional,
`version` (in)
: which version of BJData to use (see [draft 3](../../features/binary_formats/bjdata.md#draft-3-binary-format)); optional,
`#!cpp false` by default.
## Return value
@ -68,3 +74,4 @@ Linear in the size of the JSON value `j`.
## Version history
- Added in version 3.11.0.
- BJData version parameter (for draft3 binary encoding) added in version 3.12.0.

32
docs/mkdocs/docs/features/binary_formats/bjdata.md
View File

@ -2,10 +2,10 @@
The [BJData format](https://neurojson.org) was derived from and improved upon
[Universal Binary JSON(UBJSON)](https://ubjson.org) specification (Draft 12). Specifically, it introduces an optimized
array container for efficient storage of N-dimensional packed arrays (**ND-arrays**); it also adds 4 new type markers -
`[u] - uint16`, `[m] - uint32`, `[M] - uint64` and `[h] - float16` - to unambiguously map common binary numeric types;
furthermore, it uses little-endian (LE) to store all numerics instead of big-endian (BE) as in UBJSON to avoid
unnecessary conversions on commonly available platforms.
array container for efficient storage of N-dimensional packed arrays (**ND-arrays**); it also adds 5 new type markers -
`[u] - uint16`, `[m] - uint32`, `[M] - uint64`, `[h] - float16` and `[B] - byte` - to unambiguously map common binary
numeric types; furthermore, it uses little-endian (LE) to store all numerics instead of big-endian (BE) as in UBJSON to
avoid unnecessary conversions on commonly available platforms.
Compared to other binary JSON-like formats such as MessagePack and CBOR, both BJData and UBJSON demonstrate a rare
combination of being both binary and **quasi-human-readable**. This is because all semantic elements in BJData and
@ -49,6 +49,7 @@ The library uses the following mapping from JSON values types to BJData types ac
| string | *with shortest length indicator* | string | `S` |
| array | *see notes on optimized format/ND-array* | array | `[` |
| object | *see notes on optimized format* | map | `{` |
| binary | *see notes on binary values* | array | `[$B` |
!!! success "Complete mapping"
@ -128,15 +129,24 @@ The library uses the following mapping from JSON values types to BJData types ac
Due to diminished space saving, hampered readability, and increased security risks, in BJData, the allowed data
types following the `$` marker in an optimized array and object container are restricted to
**non-zero-fixed-length** data types. Therefore, the valid optimized type markers can only be one of `UiuImlMLhdDC`.
This also means other variable (`[{SH`) or zero-length types (`TFN`) can not be used in an optimized array or object
in BJData.
**non-zero-fixed-length** data types. Therefore, the valid optimized type markers can only be one of
`UiuImlMLhdDCB`. This also means other variable (`[{SH`) or zero-length types (`TFN`) can not be used in an
optimized array or object in BJData.
!!! info "Binary values"
If the JSON data contains the binary type, the value stored is a list of integers, as suggested by the BJData
documentation. In particular, this means that the serialization and the deserialization of JSON containing binary
values into BJData and back will result in a different JSON object.
BJData provides a dedicated `B` marker (defined in the [BJData specification (Draft 3)][BJDataBinArr]) that is used
in optimized arrays to designate binary data. This means that, unlike UBJSON, binary data can be both serialized and
deserialized.
To preserve compatibility with BJData Draft 2, the Draft 3 optimized binary array must be explicitly enabled using
the `version` parameter of [`to_bjdata`](../../api/basic_json/to_bjdata.md).
In Draft2 mode (default), if the JSON data contains the binary type, the value stored as a list of integers, as
suggested by the BJData documentation. In particular, this means that the serialization and the deserialization of
JSON containing binary values into BJData and back will result in a different JSON object.
[BJDataBinArr]: https://github.com/NeuroJSON/bjdata/blob/master/Binary_JData_Specification.md#optimized-binary-array)
??? example
@ -171,11 +181,13 @@ The library maps BJData types to JSON value types as follows:
| int32 | number_integer | `l` |
| uint64 | number_unsigned | `M` |
| int64 | number_integer | `L` |
| byte | number_unsigned | `B` |
| string | string | `S` |
| char | string | `C` |
| array | array (optimized values are supported) | `[` |
| ND-array | object (in JData annotated array format)|`[$.#[.`|
| object | object (optimized values are supported) | `{` |
| binary | binary (strongly-typed byte array) | `[$B` |
!!! success "Complete mapping"