5 ms·
It seems like repeated keys would be super-easy to make more efficient in a binary format: You could write each key in stringish form once (the first time it wa
by clhodapp 7y ago
It seems like repeated keys would be super-easy to make more efficient in a binary format: You could write each key in stringish form once (the first time it was seen) and then refer to it with a numeric seen-key reference from there on.
- Matthias247 7y agoIf you want it to be infinitely streaming compatible (which CBOR is) it raises another question: For how long are identifiers valid, and do they get invalidated or updates at some point of time. The header compression in HTTP2 solves such a problem, but also introduces quite a bit of additional complexity
- clhodapp 7y agoRight. There is definitely a tradeoff between requiring retention of the identifiers, which requires keeping more state, and re-specifying them, which requires sending more data. There are definitely more sophisticated ways to handle this (see: your HTTP2 header example, which I think even includes value caching), but an easy way to chose a point on that tradeoff spectrum is to simply keep a fixed-sized ringbuffer and retain, say, the last 256 keys.
- ehsankia 7y agoAs mentioned above, for something like to work, you need pre-defined schema, such as protobuf, which yes I know is what I should be using :)
- clhodapp 7y agoYou definitely don't need a predefined schema. You simply make it a responsibility of both the reader and the writer to keep track of what stringy keys have been seen and in what order. You can then refer back into that ordered list of known names the next time a repeated name comes up.
- 08-15 7y agoThis is such a great idea, it got standardized as a CBOR extension: http://cbor.schmorp.de/stringref http://cbor.schmorp.de/stringref
- clhodapp 7y agoOh nice!