2 ms·

Protobuf/Thrift require predefined schemas, which is a different model. Avro is pretty similar to MsgPack in concept. It seems less popular (at least in my ci

by ken 7y ago

Protobuf/Thrift require predefined schemas, which is a different model. Avro is pretty similar to MsgPack in concept. It seems less popular (at least in my circles), and has fewer implementations, which may or may not matter. As for performance and efficiency, the first benchmark I found [1] shows MsgPack is more space efficient, and faster to serialize, while Avro is faster to de-serialize. The second benchmark I found [2] found exactly the opposite. So it's not clear to me that either is better, and I'm sure it depends on your data, which library you're using, and how you're using it. Being nearly as performant as the predefined-schema serializers, while being nearly a drop-in replacement for JSON, seems like a major and valuable use case to me. [1]: https://medium.com/@nitinpaliwal87/compression-and-serialization-techniques-benchmarking-fd1f34c1098b https://medium.com/@nitinpaliwal87/compression-and-serializa... [2]: https://github.com/saint1991/serialization-benchmark https://github.com/saint1991/serialization-benchmark

meritt 7y ago
> shows MsgPack is more space efficient While I see the same numbers as you do, there's no way a format that includes plaintext keys alongside the values is going to be smaller than a format that only includes a packed field number+type and the corresponding values. There's something fishy going on here, and I can't seem to find a link to the source behind his benchmarks?
- deleted 7y ago
  [deleted]
- ludocode 7y ago
  I'm not sure if the benchmarks above reflect this, but MessagePack allows you to use integers (or any data type) for map keys. This makes it a lot smaller than using plaintext keys, and makes it comparable to formats like Protobuf despite the lack of formal schema. (You also lose a lot of the context-free readability of messages, so it's a tradeoff.)

by ken 7y ago

meritt 7y ago

> shows MsgPack is more space efficient While I see the same numbers as you do, there's no way a format that includes plaintext keys alongside the values is going to be smaller than a format that only includes a packed field number+type and the corresponding values. There's something fishy going on here, and I can't seem to find a link to the source behind his benchmarks?

deleted 7y ago
[deleted]
ludocode 7y ago
I'm not sure if the benchmarks above reflect this, but MessagePack allows you to use integers (or any data type) for map keys. This makes it a lot smaller than using plaintext keys, and makes it comparable to formats like Protobuf despite the lack of formal schema. (You also lose a lot of the context-free readability of messages, so it's a tradeoff.)