4 ms·

MessagePack - https://msgpack.org https://msgpack.org

stkdump 3y ago
Most binary JSON variants including MessagePack deviate from the spirit of JSON in one fundamental way: they are not simple. The JSON file format can accurately be described with a few simple syntax diagram that fit comfortably on a printed page and the spec can be implmented with very little code. A binary file format comes with the theoretical further simplicity advantage that you can leave out conversion of numbers, detection of string length, (un)escaping and dealing with white space/pretty printing. Instead of taking that as gains and be done with it, such formats start introducing (space) microoptimizations, such as many different number formats, variable length numbers for sizes, a wealth of additional data formats such as dates, etc. I think there a two good variants of binary JSON equivalents: one where you have a minimal implementation that makes both reading and writing with the lowest amount of code possible, potentially feeding a SAX-like interface or constructing an in memory representation (DOM-style). The other one would be to facilitate lookups in large documents such as json_extract without parsing the whole document. That would require some clever layout of the data, potentially even as in-file hash tables or similar. I assume that is what SQLite has done.
- DemocracyFTW2 3y ago
  TBF&H, when you say additional data formats such as dates, dates (including time) are the one format that would be great to have standardized in JSON, due to their inherent complexity. A format for binary data is also missing, but that is downright trivial to do ad-hoc (string literals + encoding hint) when compared to datetimes.
  - stkdump 3y ago
    I don't know, it is trivially simple to choose either Unix timestamp (which is relatively close to what MessagePack is doing) or JSs format [1] if you like a bit more readability and time zone deltas. Or any of the zillion more complex formats such as ISO 8601, etc. In my opinion that would be part of the schema and not the file format. [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date#date_time_string_format https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Most binary JSON variants including MessagePack deviate from the spirit of JSON in one fundamental way: they are not simple. The JSON file format can accurately be described with a few simple syntax diagram that fit comfortably on a printed page and the spec can be implmented with very little code. A binary file format comes with the theoretical further simplicity advantage that you can leave out conversion of numbers, detection of string length, (un)escaping and dealing with white space/pretty printing. Instead of taking that as gains and be done with it, such formats start introducing (space) microoptimizations, such as many different number formats, variable length numbers for sizes, a wealth of additional data formats such as dates, etc. I think there a two good variants of binary JSON equivalents: one where you have a minimal implementation that makes both reading and writing with the lowest amount of code possible, potentially feeding a SAX-like interface or constructing an in memory representation (DOM-style). The other one would be to facilitate lookups in large documents such as json_extract without parsing the whole document. That would require some clever layout of the data, potentially even as in-file hash tables or similar. I assume that is what SQLite has done.

DemocracyFTW2 3y ago
TBF&H, when you say additional data formats such as dates, dates (including time) are the one format that would be great to have standardized in JSON, due to their inherent complexity. A format for binary data is also missing, but that is downright trivial to do ad-hoc (string literals + encoding hint) when compared to datetimes.
- stkdump 3y ago
  I don't know, it is trivially simple to choose either Unix timestamp (which is relatively close to what MessagePack is doing) or JSs format [1] if you like a bit more readability and time zone deltas. Or any of the zillion more complex formats such as ISO 8601, etc. In my opinion that would be part of the schema and not the file format. [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date#date_time_string_format https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...