6 ms·

JSON's appeal is that it is both compact and human readable/editable. If you're willing to sacrifice all semblance of readability/editability, then sure, let's

by cheald 14y ago

JSON's appeal is that it is both compact and human readable/editable. If you're willing to sacrifice all semblance of readability/editability, then sure, let's all do the binary marshaling format dance like it's 1983. Additionally, if you're sending data to a browser, then you're cutting the knees out of native JSON.parse implementations (Internet Explorer 8+, Firefox 3.1+, Safari 4+, Chrome 3+, and Opera 10.5+). The copy claims "half the parsing time" (just because of smaller data size), but I'm exceptionally skeptical of those claims since this is just going to move the parser back into Javascript. I whipped up a quick example to demonstrate my point: https://gist.github.com/2905882 https://gist.github.com/2905882 My results (using Chrome's console): JSON size: 386kb MsgPack size: 332kb Difference: -14% Encoding with JSON.stringify 4 ms Encoding with msgpack.pack 73 ms Difference: 18.25x slower Decoding with JSON.parse 4 ms Decoding with msgpack.unpack 13 ms Difference: 3.25x slower So MsgPack wins the pre-gzip size comparison by 14%, but then is nearly twenty times slower at encoding the data, and is over three times slower at decoding it. Furthermore, once you add in gzip: out.json.gz: 4291 bytes out.mpak.gz: 4073 bytes So, our grand savings is 218 bytes in exchange for 78ms slower parsing time. It'd still be faster to use JSON's native encoding/decoding facilities even if your client were on a 28.8k baud modem.

olsn 14y ago
For the native JavaScript implementation you are correct - no doubt about that it is faster there, however, looking at the server-side it looks the other way around: for example parsing MsgPack in Ruby is about 5x faster than parsing JSON - you can rewrite your benchmark to Ruby and try for yourself
- falien 14y ago
  But you can install a c extension implementation for ruby (or pretty much any other language that doesn't already have native JSON built in) on your server fixing that performance gap. That's exactly what you can't do on the browser JS side for MsgPack.
  - olsn 14y ago
    then you could get a c extension for MsgPack, that would have to parse less bytes than the JSON c extension and therefore be faster JSON is very good, no doubt about that - but i think that even JSON can be optimized, does not have to be by MsgPack, but probably by a binary format.
    - falien 14y ago
      My point was that you can't install a c extension in a browser. I assumed that all the MsgPack libraries would be C or equivalent performance-wise. As to text vs binary, I'll just leave this link here: http://catb.org/~esr/writings/taoup/html/textualitychapter.html http://catb.org/~esr/writings/taoup/html/textualitychapter.h...
    - olsn 14y ago
      i understood your point, however the speed in the user's browser is not important since it's just 1 operation, whereas the server has to do most of the operations decoding/encoding all that i guess there are pros and cons on both sides, certainly nice would be something like a native BSON implementation with extensions in the browser/debugger to make it readable
      - falien 14y ago
        Your blog post suggests using MsgPack as a drop in replacement for JSON for client-server communications. In that context every encode/decode operation on the server would be matched by one in the client browser. The <5% gain in performance on the server (and I think that is generous) cannot possibly make up for the performance hit of a javascript implementation of MsgPack. When MsgPack was written a lot of JSON in the browser was still not native so an argument could be made if you desperately needed that bandwidth and it worked better than average for your specific workload. That is simply not true anymore.
        olsn 14y ago
        i'm using it in Flash, so there is absolutely no JavaScript involved in my case...
lnanek2 14y ago
Wow, awesome seeing the benchmarks to back up the text of your reply. Thanks for doing that.
haberman 14y ago
> I'm exceptionally skeptical of those claims I can attest that these guys are known to post bogus benchmark numbers. The claim on their home page ("4x faster than Protocol Buffers in this test!") is incredibly misleading; their benchmark is set up in such a way that Protocol Buffers are copying the strings but Message Pack is just referencing them. It's true that the open-source release of Protocol Buffers doesn't have an easy way of referencing vs. copying, but it is still highly misleading not to mention that you're basically benchmarking memcpy(). One of these days I'll re-run their benchmark with my protobuf parser which can likewise just reference strings. I am pretty sure I will trounce their numbers.
- ralph 14y ago
  I pointed out the Protocol Buffers comparison to a friend on IRC. He came back with: "They rolled their own "high-level" serializer and deserializer for PB, built on top the lower-level stuff the documentation advises you not to use. Using the recommended interfaces, PB is faster than in their test. It is still slower than msgpack. Not sure why they'd make their test look cooked when they win in a fair test anyway. Further examination shows that the test is _mainly_ a test of std::string, for protobuf. std::string is required to use contiguous storage, whilst msgpack's test uses a msgpack-specific rope type."
zobzu 14y ago
I started reading their page and saw stuff like "take tiny, tiny data sample to have the most difference" and I started doubting. Since I saw the "faster" processing, and given how straight forward it is it parse json in C I was doubting even more. So I clicked comments, expecting to see this, so that I wouldn't have to do it myself. Ah, makes me happy, thanks!
javajosh 14y ago
Here's a conveniently executable fiddle if people want to exec cheald's benchmark themselves: http://jsfiddle.net/javajosh/SjPub/ http://jsfiddle.net/javajosh/SjPub/
hermanhermitage 14y ago
These results mirror the experiences I've had with large JSON payloads of time series data and experimenting with different wire formats. The best message sizes I have obtained by custom compressors with complex predictive models and bit level encoding. However with JavaScript on the critical path (say browser side) even a simple bit/byte bashing format like MessagePack was a performance loss and in my case the payloads were bigger. YMMV, but if you really need aggressive compression or performance none of the standard solutions (protobuff et al) come close to hand rolled solutions. If you have JavaScript on your path, I've had the most success by using JSON-to-JSON transformations utilising a predictive model (think say rows to columns and delta encoding) with gz over the top. This has got my payloads down in size without blowing out time - but it does spray the GC with lots of short lived object traffic.

by cheald 14y ago

olsn 14y ago

For the native JavaScript implementation you are correct - no doubt about that it is faster there, however, looking at the server-side it looks the other way around: for example parsing MsgPack in Ruby is about 5x faster than parsing JSON - you can rewrite your benchmark to Ruby and try for yourself

falien 14y ago
But you can install a c extension implementation for ruby (or pretty much any other language that doesn't already have native JSON built in) on your server fixing that performance gap. That's exactly what you can't do on the browser JS side for MsgPack.
- olsn 14y ago
  then you could get a c extension for MsgPack, that would have to parse less bytes than the JSON c extension and therefore be faster JSON is very good, no doubt about that - but i think that even JSON can be optimized, does not have to be by MsgPack, but probably by a binary format.
  - falien 14y ago
    My point was that you can't install a c extension in a browser. I assumed that all the MsgPack libraries would be C or equivalent performance-wise. As to text vs binary, I'll just leave this link here: http://catb.org/~esr/writings/taoup/html/textualitychapter.html http://catb.org/~esr/writings/taoup/html/textualitychapter.h...
  - olsn 14y ago
    i understood your point, however the speed in the user's browser is not important since it's just 1 operation, whereas the server has to do most of the operations decoding/encoding all that i guess there are pros and cons on both sides, certainly nice would be something like a native BSON implementation with extensions in the browser/debugger to make it readable
    - falien 14y ago
      Your blog post suggests using MsgPack as a drop in replacement for JSON for client-server communications. In that context every encode/decode operation on the server would be matched by one in the client browser. The <5% gain in performance on the server (and I think that is generous) cannot possibly make up for the performance hit of a javascript implementation of MsgPack. When MsgPack was written a lot of JSON in the browser was still not native so an argument could be made if you desperately needed that bandwidth and it worked better than average for your specific workload. That is simply not true anymore.
      - olsn 14y ago
        i'm using it in Flash, so there is absolutely no JavaScript involved in my case...

lnanek2 14y ago

Wow, awesome seeing the benchmarks to back up the text of your reply. Thanks for doing that.

haberman 14y ago

> I'm exceptionally skeptical of those claims I can attest that these guys are known to post bogus benchmark numbers. The claim on their home page ("4x faster than Protocol Buffers in this test!") is incredibly misleading; their benchmark is set up in such a way that Protocol Buffers are copying the strings but Message Pack is just referencing them. It's true that the open-source release of Protocol Buffers doesn't have an easy way of referencing vs. copying, but it is still highly misleading not to mention that you're basically benchmarking memcpy(). One of these days I'll re-run their benchmark with my protobuf parser which can likewise just reference strings. I am pretty sure I will trounce their numbers.

ralph 14y ago
I pointed out the Protocol Buffers comparison to a friend on IRC. He came back with: "They rolled their own "high-level" serializer and deserializer for PB, built on top the lower-level stuff the documentation advises you not to use. Using the recommended interfaces, PB is faster than in their test. It is still slower than msgpack. Not sure why they'd make their test look cooked when they win in a fair test anyway. Further examination shows that the test is _mainly_ a test of std::string, for protobuf. std::string is required to use contiguous storage, whilst msgpack's test uses a msgpack-specific rope type."

zobzu 14y ago

I started reading their page and saw stuff like "take tiny, tiny data sample to have the most difference" and I started doubting. Since I saw the "faster" processing, and given how straight forward it is it parse json in C I was doubting even more. So I clicked comments, expecting to see this, so that I wouldn't have to do it myself. Ah, makes me happy, thanks!

javajosh 14y ago

Here's a conveniently executable fiddle if people want to exec cheald's benchmark themselves: http://jsfiddle.net/javajosh/SjPub/ http://jsfiddle.net/javajosh/SjPub/

hermanhermitage 14y ago

These results mirror the experiences I've had with large JSON payloads of time series data and experimenting with different wire formats. The best message sizes I have obtained by custom compressors with complex predictive models and bit level encoding. However with JavaScript on the critical path (say browser side) even a simple bit/byte bashing format like MessagePack was a performance loss and in my case the payloads were bigger. YMMV, but if you really need aggressive compression or performance none of the standard solutions (protobuff et al) come close to hand rolled solutions. If you have JavaScript on your path, I've had the most success by using JSON-to-JSON transformations utilising a predictive model (think say rows to columns and delta encoding) with gz over the top. This has got my payloads down in size without blowing out time - but it does spray the GC with lots of short lived object traffic.