5 ms·

The article links to this[1] older HN comment which argues against CBOR. I must admit this passage made me laugh out loud: A decoder that comes across a si

The article links to this[1] older HN comment which argues against CBOR. I must admit this passage made me laugh out loud: A decoder that comes across a simple value (Section 2.3) that it does not recognize, such as a value that was added to the IANA registry after the decoder was deployed or a value that the decoder chose not to implement, might issue a warning, might stop processing altogether, might handle the error by making the unknown value available to the application as such (as is expected of generic decoders), or take some other type of action. I choose not to implement `int`. I decide instead to fill up your home folder. I'm a compliant CBOR implementation. Seems this part of the specification has been rewritten[2], so now a generic decoder is to pass on the tag and value to the user or return an error. [1]: https://news.ycombinator.com/item?id=14072598 https://news.ycombinator.com/item?id=14072598 [2]: https://www.rfc-editor.org/rfc/rfc8949.html#name-generic-encoders-and-decode https://www.rfc-editor.org/rfc/rfc8949.html#name-generic-enc...

camgunz 2y ago
Disclaimer: I wrote and maintain a MessagePack implementation. Hey that's me! Yeah they fixed that, but there's other parts of the spec that are basically unworkable like indefinite length values, "canonicalization", and tags, making it essentially MP (MP does have extension types, I should say, the virtue of tossing out CBOR's tags is you then don't have to implement things like datetimes/timezones, bignums, etc), and indeed at least FIDO tosses this all out: https://fidoalliance.org/specs/fido-v2.0-ps-20190130/fido-client-to-authenticator-protocol-v2.0-ps-20190130.html#ctap2-canonical-cbor-encoding-form https://fidoalliance.org/specs/fido-v2.0-ps-20190130/fido-cl... Beyond that, CBOR is MessagePack. The story is that Carsten Bormann wanted to create an IETF standardized MP version, the creators asked him not to (after he acted in pretty bad faith), he forked off a version, added the aforementioned ill-advised tweaks, named it after himself, and submitted it anyway. All this design by committee stuff is mostly wrong--though IETF has modified it by committee since. There's no reason an MP implementation has to be slower than a CBOR implementation. If a given library wanted to be very fast it could be. If anything, the fact that CBOR more or less requires you to allocate should put a ceiling on how fast it can really be. Or, put another way, benchmarks of dynamic language implementations of a serialization format aren't a high signal indication of its speed ceiling. If you use a dynamic language and speed is a concern to this degree, you'd write an adapter yourself, probably building on one of the low level implementations. That said, people are usually disappointed by MP's speed over JSON. A lot of engineering hours have gone into making JSON fast, to the point where I don't think it ever made sense to choose MP over it for speed reasons (there are other good reasons). Other posters here have pointed out that your metrics are usually dominated by something else. But finally, CBOR is fine! The implementations are good and it's widely used. Users of CBOR and MP alike will probably have very similar experiences unless you have a niche use case (on an embedded device that can't allocate, you really need bignums, etc).
- magicalhippo 2y ago
  > Yeah they fixed that, but there's other parts of the spec that are basically unworkable Yeah it just made me chuckle cause it was such an obvious oversight and a fun way of pointing it out. That said I totally get that writing specs are hard, so not dissing the authors as such. > There's no reason an MP implementation has to be slower than a CBOR implementation. Yeah that also struck me. Like ok that CBOR library might be faster than that MP library, but could be either is just missing some optimizations. And it didn't look like there were orders-of-magnitude differences in either case. Anyway I've only looked at CBOR and MessagePack when I dabbled with some microcontroller projects. I found both to be too big, ie couldn't find a library suitably small, either compiled size or memory requirements or both. So I ended up with JSON for those due to that. Using a SAX-like parser I could avoid dynamic allocations entirely (or close enough).
  - camgunz 2y ago
    > That said I totally get that writing specs are hard, so not dissing the authors as such. Oh definitely. Yeah maybe I come off as anti-spec or something, but in this case I just think MP was really well thought out, and then Bormann hung a bunch of stuff on it that really wasn't, and I'm salty haha. > Anyway I've only looked at CBOR and MessagePack when I dabbled with some microcontroller projects. I found both to be too big, ie couldn't find a library suitably small, either compiled size or memory requirements or both. So I ended up with JSON for those due to that. Using a SAX-like parser I could avoid dynamic allocations entirely (or close enough). Whaa? I wrote an MP implementation specifically for this use case: https://github.com/camgunz/cmp https://github.com/camgunz/cmp. JSON parsing terrifies me; there was some table of tons of JSON (de)serializers with all their weirdo bugs that I never would've thought of. There are probably pretty good test suites now though? I've never looked.
    - magicalhippo 2y ago
      > I wrote an MP implementation specifically for this use case Perhaps I missed that, can't recall. Will definitely try (again) tho, looks very promising. As for parsing JSON, the upside is that's its trivial to debug over serial, both viewing and sending, and in my case I could assume limited shenanigans and fail hard if there were issues.

camgunz 2y ago

Disclaimer: I wrote and maintain a MessagePack implementation. Hey that's me! Yeah they fixed that, but there's other parts of the spec that are basically unworkable like indefinite length values, "canonicalization", and tags, making it essentially MP (MP does have extension types, I should say, the virtue of tossing out CBOR's tags is you then don't have to implement things like datetimes/timezones, bignums, etc), and indeed at least FIDO tosses this all out: https://fidoalliance.org/specs/fido-v2.0-ps-20190130/fido-client-to-authenticator-protocol-v2.0-ps-20190130.html#ctap2-canonical-cbor-encoding-form https://fidoalliance.org/specs/fido-v2.0-ps-20190130/fido-cl... Beyond that, CBOR is MessagePack. The story is that Carsten Bormann wanted to create an IETF standardized MP version, the creators asked him not to (after he acted in pretty bad faith), he forked off a version, added the aforementioned ill-advised tweaks, named it after himself, and submitted it anyway. All this design by committee stuff is mostly wrong--though IETF has modified it by committee since. There's no reason an MP implementation has to be slower than a CBOR implementation. If a given library wanted to be very fast it could be. If anything, the fact that CBOR more or less requires you to allocate should put a ceiling on how fast it can really be. Or, put another way, benchmarks of dynamic language implementations of a serialization format aren't a high signal indication of its speed ceiling. If you use a dynamic language and speed is a concern to this degree, you'd write an adapter yourself, probably building on one of the low level implementations. That said, people are usually disappointed by MP's speed over JSON. A lot of engineering hours have gone into making JSON fast, to the point where I don't think it ever made sense to choose MP over it for speed reasons (there are other good reasons). Other posters here have pointed out that your metrics are usually dominated by something else. But finally, CBOR is fine! The implementations are good and it's widely used. Users of CBOR and MP alike will probably have very similar experiences unless you have a niche use case (on an embedded device that can't allocate, you really need bignums, etc).

magicalhippo 2y ago
> Yeah they fixed that, but there's other parts of the spec that are basically unworkable Yeah it just made me chuckle cause it was such an obvious oversight and a fun way of pointing it out. That said I totally get that writing specs are hard, so not dissing the authors as such. > There's no reason an MP implementation has to be slower than a CBOR implementation. Yeah that also struck me. Like ok that CBOR library might be faster than that MP library, but could be either is just missing some optimizations. And it didn't look like there were orders-of-magnitude differences in either case. Anyway I've only looked at CBOR and MessagePack when I dabbled with some microcontroller projects. I found both to be too big, ie couldn't find a library suitably small, either compiled size or memory requirements or both. So I ended up with JSON for those due to that. Using a SAX-like parser I could avoid dynamic allocations entirely (or close enough).
- camgunz 2y ago
  > That said I totally get that writing specs are hard, so not dissing the authors as such. Oh definitely. Yeah maybe I come off as anti-spec or something, but in this case I just think MP was really well thought out, and then Bormann hung a bunch of stuff on it that really wasn't, and I'm salty haha. > Anyway I've only looked at CBOR and MessagePack when I dabbled with some microcontroller projects. I found both to be too big, ie couldn't find a library suitably small, either compiled size or memory requirements or both. So I ended up with JSON for those due to that. Using a SAX-like parser I could avoid dynamic allocations entirely (or close enough). Whaa? I wrote an MP implementation specifically for this use case: https://github.com/camgunz/cmp https://github.com/camgunz/cmp. JSON parsing terrifies me; there was some table of tons of JSON (de)serializers with all their weirdo bugs that I never would've thought of. There are probably pretty good test suites now though? I've never looked.
  - magicalhippo 2y ago
    > I wrote an MP implementation specifically for this use case Perhaps I missed that, can't recall. Will definitely try (again) tho, looks very promising. As for parsing JSON, the upside is that's its trivial to debug over serial, both viewing and sending, and in my case I could assume limited shenanigans and fail hard if there were issues.