5 ms·
No one has come up with a binary format that has the same properties and has been as successful.
by mgaunard 2y ago
No one has come up with a binary format that has the same properties and has been as successful.
- lifthrasiir 2y agoMaybe because most binary formats can't be human-readable which is one of important properties JSON do have.
- secondcoming 2y agoYou can easily write a tool to dump a binary format.
- vsnf 2y agoNow embed that tool in all my other tools so I can human-read the binary format without thinking twice and then we can chat.
- vmfunction 2y agosuch as https://en.wikipedia.org/wiki/Protocol_Buffers https://en.wikipedia.org/wiki/Protocol_Buffers ?
- nXqd 2y agolast time I read, simdjson still beat protocol buffer a bit.
- lifthrasiir 2y agoDue to the inherent constraint of JSON, the exact use case would matter a lot for such comparisons. Simdjson is generally faster when you only want a well-formedness check or a very small proportion of a large input JSON, but the "well-formedness" for JSON would be a small subset of the "well-formedness" of binary formats, and a partial parsing performance is often dominated by language bindings instead of underlying parsers (it is a wise move that simdjson also has a JSON pointer support for this reason, because that greatly reduces FFI overhead). Binary formats in comparsion tend to have a generally flat performance profile.
- chpatrick 2y agoIt's not self-documenting, you can't decode it unless you have the .proto. JSON just works.
- mort96 2y agoYeah, that's a great example of a non-human-readable binary format
- adolph 2y agoPossible future candidates: https://capnproto.org/ https://capnproto.org/ https://arrow.apache.org/ https://arrow.apache.org/
- aldanor 2y agoArrow is columnar storage, nothing relevant here. And capnproto is basically another version of pb... (see discussion above)
- adolph 2y agoLook again at Arrow. Part of the concept is the wire format is the same as the in memory format—no de/serialization. Arrow isn’t intended for storage; they point to parquet for that. Apache Arrow defines an inter-process communication (IPC) mechanism to transfer a collection of Arrow columnar arrays (called a “record batch”). It can be used synchronously between processes using the Arrow “stream format”, or asynchronously by first persisting data on storage using the Arrow “file format”. The Arrow IPC mechanism is based on the Arrow in-memory format, such that there is no translation necessary between the on-disk representation and the in-memory representation. [0] The Arrow spec aligns columnar data in memory to minimize cache misses and take advantage of the latest SIMD (Single input multiple data) and GPU operations on modern processors. [1] 0. https://arrow.apache.org/faq/ https://arrow.apache.org/faq/ 1. https://arrow.apache.org/docs/js/ https://arrow.apache.org/docs/js/
- formerly_proven 2y agomsgpack and "I can't believe it's not msgpack with the name changed to my own name" CBOR are basically that.
- flohofwoe 2y agoIFF was basically that on the Amiga: https://en.wikipedia.org/wiki/Interchange_File_Format https://en.wikipedia.org/wiki/Interchange_File_Format Unfortunately with the Amiga itself being a niche platform, the idea didn't catch on. It also sort-of requires a central registry for the top level FourCC codes (basically reserved magic number for each file format).
- SushiHippie 2y agohttps://en.m.wikipedia.org/wiki/Smile_(data_interchange_format) https://en.m.wikipedia.org/wiki/Smile_(data_interchange_form... /s