6 ms·

Json parsing is orders of magnitude slower than protobuf decoding.

lucideer 4y ago
I did some brief googling after reading your comment and I did find one article showing clientside protobuf being faster than JSON[0]. However they didn't isolate parsing - the only thing they measure is total request time to a Java spring application, so the JSON slowdown will include the Java JSON serialisation overhead as well as the request size/network overhead. My instinct is that these two will heavily favour protobuf making the JSON parse still likely to be faster. It also shows a difference of 388ms (protobuf) vs 396ms (JSON) which is pretty negligible. Certainly not orders of magnitude. Do you have other sources? [0] https://auth0.com/blog/beating-json-performance-with-protobuf/ https://auth0.com/blog/beating-json-performance-with-protobu...
- dwmbt 4y ago
  My own search results: [0] https://github.com/boguslaw-wojcik/encoding-benchmarks https://github.com/boguslaw-wojcik/encoding-benchmarks [1] https://github.com/alecthomas/go_serialization_benchmarks https://github.com/alecthomas/go_serialization_benchmarks [2] https://techsparx.com/nodejs/datastore/protocol-buffers.html https://techsparx.com/nodejs/datastore/protocol-buffers.html
  - lucideer 4y ago
    First two links are Go, so not relevant to client-side. Third link is also server-side, but since it's NodeJS it's at least close enough / more relevant to client-side perf. Here's the benchmark from the third link: benchmark time (avg) (min … max) --------------------------------------------------- encode-JSON 342.37 µs/iter (311.93 µs … 1.19 ms) decode-JSON 435.9 µs/iter (384.44 µs … 1.41 ms) encode-PB 946.43 µs/iter (777.38 µs … 3.13 ms) decode-PB 770.79 µs/iter (688.99 µs … 1.78 ms) encode-PBJS 696.75 µs/iter (618.43 µs … 2.43 ms) decode-PBJS 455.36 µs/iter (413.66 µs … 1.09 ms) showing JSON to be significantly faster
    - dwmbt 4y ago
      ahh yea, i'm not sure why the rest of my comment didn't upload. i was going to say that i thought the common use case for protobufs was to more ergonomically communicate between microservices? in any case, that's the only time i've ever seen it used in production. the first link is a go benchmark that i felt represented why someone would use it for those purposes, the second was linked to show that despite numerous (successful!) attempts to make deserializing/serializing data faster and smaller, JSON is still the most heavily used and i would wager it's mostly due to how easy it is to use as far as browsers are concerned. the third was a link to justify that claim and show that js-land is much, much different than go-land as far as proto's and JSON encoding/decoding are concerned!
- beastman82 4y ago
  Java to Java uncompressed in that article is 6x faster per that article. So yeah not a whole order of magnitude. I was using my experience as a guide where JSON parsing is a huge compute hog and Protobuf is not. I've never experimented w/ Javascript or compression or any of the other things in that article, I guess YMMV.
  - lucideer 4y ago
    I specifically referred to clientside in my original comment, so not talking about java to java. Clientside is always going to be the pertinent metric for UX since it's processed on the user's device.
- Thaxll 4y ago
  It's protocol 101, pb is a binary protocol with known schema so of course it has to be faster than json for encoding/decoding. Now it does not means that it's going to be faster all the time, it depends of the maturity of the library / language but on paper yes it is faster.
  - lucideer 4y ago
    > it does not means that it's going to be faster all the time, it depends of the maturity of the library / language I feel like I'm having to repeat myself a lot here as noone seems to have read the original comment correctly: we're talking about one specific language in one specific known environment here. Noone is claiming that JSON outperforms PB in general: only that it does in browsers, where it's actually relevant for UX.
    - nostrebored 4y ago
      It’s relevant for UX throughout the entire stack. Where I’m working now, we have a REST API for users to interact with and every call behind the scenes is proto. As we deal with quite large objects, the benefits of avoiding repeated serialization and deserialization add up quickly. From the user’s perspective we have a performant app, and much of this is possible due to proto.
      - lucideer 4y ago
        Thank you. Finally someone answered my original question. So it sounds like the trade-off can be worthwhile in some cases: particularly for large objects where serialisation is a significant serverside bottleneck. I'm curious: you say PB helps avoid "repeated serialisation/deserialisation": how? In my mind, architecting an app that uses JSON/PB on the wire serialisation happens once on output & deserialisation happens once on input. For both transfer formats. Surely you wouldn't be passing massive json strings around your app in memory? Also curious which is the bigger bottleneck for your large objects: input or output. How large is large?
- throw827474737 4y ago
  Oh come on... how can one assume a binary somehow TLV-encoded format is not faster than parsing strings (generall json schemaless btw, the dynamicity also adds on top, while yes, protobuf also has variable sized containers). It is like you would claim parsing a string to an int is having no overhead over a straight int (yes I know proto ufs still require the varint decoding, still huge difference). It id also not only the speed but also size is usually a magnitude off (and no, compression doesn't cut it and trades size again for computation). Sure, if size and speed do not matter it is strange that you had considered protobuf at all.. but claiming they are never needed just means you have never been to resource constrained systems? What you cite there, I assume most of that 400ms has nothing to do with the message encoding at all btw..
  - lucideer 4y ago
    (a) You're making assumptions based on rule of thumb, I'm talking about real world usage: your points make sense in theory but don't necessarily reflect reality (b) I'm talking about a narrow & specific case. PB may outperform JSON in most cases but I'm very specifically referring to browsers where JSON is native (& highly optimised) whereas PB is provided by a selection of open source libraries written in javascript. So that domain heavily favours JSON perf-wise.
    - throw827474737 4y ago
      > You're making assumptions based No, not at all... coming from embedded where apeed, memory size and also bandwidth did count, json was actually.not just worse, but just wouldn't have been feasible (because our protobufs already barely fit memory and MTU constraints).
  - AlmostAnyone 4y ago
    How can JavaScript code (PB decoder) be faster than native code (JSON parser)?
    - Aeolun 4y ago
      Much, much less processing to do. Most of pb decoding is just reading bytes until you fill your data structure.
deleted 4y ago
[deleted]
leeoniya 4y ago
at least in the frontend (without WASM), it depends. a few months ago i tested https://github.com/mapbox/pbf https://github.com/mapbox/pbf and while it was faster for deep/complex structs vs an unoptimized/repetative JSON blob, it was much slower at shallow structs and flat arrays of stuff. if you spend a bit of time to encode stuff as flat arrays to avoid mem alloc, JSON parsing wins by a lot since it goes through highly optimized C or assembly, while decoding protobuf in the JS JIT does not. of course it's not always feasible to make optimized over-the-wire JSON structs if you have a huge/complex API that can return many shapes of complex structs.
- mourner 4y ago
  At pbf speeds, decoding is usually no longer a bottleneck, but bandwidth might be when comparing with gzipped JSON. Also, one of the primary advantages of pbf is being able to decode partially and lazily (and adjust how things are decoded at low level), which is very important in use cases like vector maps.
  - leeoniya 4y ago
    > At pbf speeds, decoding is usually no longer a bottleneck, but bandwidth might be when comparing with gzipped JSON. we were streaming a few hundred float datapoints spread across a dozen(ish) flat arrays over websocket at 20-40hz and needed to decode the payload eagerly. plain JSON was a multi-factor speedup over pbf for this case. but it's fully possible i was holding it wrong, too! even when your "bottleneck" is rendering/rasterization (10ms), but your data pipe takes 3ms instead of 1ms, it's a big effect on framerate, battery, thermals, etc. i'm a big fan of your work! while i have you here, would you mind reviewing this sometime soon? ;) https://github.com/mourner/flatbush/pull/44 https://github.com/mourner/flatbush/pull/44

I did some brief googling after reading your comment and I did find one article showing clientside protobuf being faster than JSON[0]. However they didn't isolate parsing - the only thing they measure is total request time to a Java spring application, so the JSON slowdown will include the Java JSON serialisation overhead as well as the request size/network overhead. My instinct is that these two will heavily favour protobuf making the JSON parse still likely to be faster. It also shows a difference of 388ms (protobuf) vs 396ms (JSON) which is pretty negligible. Certainly not orders of magnitude. Do you have other sources? [0] https://auth0.com/blog/beating-json-performance-with-protobuf/ https://auth0.com/blog/beating-json-performance-with-protobu...

dwmbt 4y ago
My own search results: [0] https://github.com/boguslaw-wojcik/encoding-benchmarks https://github.com/boguslaw-wojcik/encoding-benchmarks [1] https://github.com/alecthomas/go_serialization_benchmarks https://github.com/alecthomas/go_serialization_benchmarks [2] https://techsparx.com/nodejs/datastore/protocol-buffers.html https://techsparx.com/nodejs/datastore/protocol-buffers.html
- lucideer 4y ago
  First two links are Go, so not relevant to client-side. Third link is also server-side, but since it's NodeJS it's at least close enough / more relevant to client-side perf. Here's the benchmark from the third link: benchmark time (avg) (min … max) --------------------------------------------------- encode-JSON 342.37 µs/iter (311.93 µs … 1.19 ms) decode-JSON 435.9 µs/iter (384.44 µs … 1.41 ms) encode-PB 946.43 µs/iter (777.38 µs … 3.13 ms) decode-PB 770.79 µs/iter (688.99 µs … 1.78 ms) encode-PBJS 696.75 µs/iter (618.43 µs … 2.43 ms) decode-PBJS 455.36 µs/iter (413.66 µs … 1.09 ms) showing JSON to be significantly faster
  - dwmbt 4y ago
    ahh yea, i'm not sure why the rest of my comment didn't upload. i was going to say that i thought the common use case for protobufs was to more ergonomically communicate between microservices? in any case, that's the only time i've ever seen it used in production. the first link is a go benchmark that i felt represented why someone would use it for those purposes, the second was linked to show that despite numerous (successful!) attempts to make deserializing/serializing data faster and smaller, JSON is still the most heavily used and i would wager it's mostly due to how easy it is to use as far as browsers are concerned. the third was a link to justify that claim and show that js-land is much, much different than go-land as far as proto's and JSON encoding/decoding are concerned!
beastman82 4y ago
Java to Java uncompressed in that article is 6x faster per that article. So yeah not a whole order of magnitude. I was using my experience as a guide where JSON parsing is a huge compute hog and Protobuf is not. I've never experimented w/ Javascript or compression or any of the other things in that article, I guess YMMV.
- lucideer 4y ago
  I specifically referred to clientside in my original comment, so not talking about java to java. Clientside is always going to be the pertinent metric for UX since it's processed on the user's device.
Thaxll 4y ago
It's protocol 101, pb is a binary protocol with known schema so of course it has to be faster than json for encoding/decoding. Now it does not means that it's going to be faster all the time, it depends of the maturity of the library / language but on paper yes it is faster.
- lucideer 4y ago
  > it does not means that it's going to be faster all the time, it depends of the maturity of the library / language I feel like I'm having to repeat myself a lot here as noone seems to have read the original comment correctly: we're talking about one specific language in one specific known environment here. Noone is claiming that JSON outperforms PB in general: only that it does in browsers, where it's actually relevant for UX.
  - nostrebored 4y ago
    It’s relevant for UX throughout the entire stack. Where I’m working now, we have a REST API for users to interact with and every call behind the scenes is proto. As we deal with quite large objects, the benefits of avoiding repeated serialization and deserialization add up quickly. From the user’s perspective we have a performant app, and much of this is possible due to proto.
    - lucideer 4y ago
      Thank you. Finally someone answered my original question. So it sounds like the trade-off can be worthwhile in some cases: particularly for large objects where serialisation is a significant serverside bottleneck. I'm curious: you say PB helps avoid "repeated serialisation/deserialisation": how? In my mind, architecting an app that uses JSON/PB on the wire serialisation happens once on output & deserialisation happens once on input. For both transfer formats. Surely you wouldn't be passing massive json strings around your app in memory? Also curious which is the bigger bottleneck for your large objects: input or output. How large is large?
throw827474737 4y ago
Oh come on... how can one assume a binary somehow TLV-encoded format is not faster than parsing strings (generall json schemaless btw, the dynamicity also adds on top, while yes, protobuf also has variable sized containers). It is like you would claim parsing a string to an int is having no overhead over a straight int (yes I know proto ufs still require the varint decoding, still huge difference). It id also not only the speed but also size is usually a magnitude off (and no, compression doesn't cut it and trades size again for computation). Sure, if size and speed do not matter it is strange that you had considered protobuf at all.. but claiming they are never needed just means you have never been to resource constrained systems? What you cite there, I assume most of that 400ms has nothing to do with the message encoding at all btw..
- lucideer 4y ago
  (a) You're making assumptions based on rule of thumb, I'm talking about real world usage: your points make sense in theory but don't necessarily reflect reality (b) I'm talking about a narrow & specific case. PB may outperform JSON in most cases but I'm very specifically referring to browsers where JSON is native (& highly optimised) whereas PB is provided by a selection of open source libraries written in javascript. So that domain heavily favours JSON perf-wise.
  - throw827474737 4y ago
    > You're making assumptions based No, not at all... coming from embedded where apeed, memory size and also bandwidth did count, json was actually.not just worse, but just wouldn't have been feasible (because our protobufs already barely fit memory and MTU constraints).
- AlmostAnyone 4y ago
  How can JavaScript code (PB decoder) be faster than native code (JSON parser)?
  - Aeolun 4y ago
    Much, much less processing to do. Most of pb decoding is just reading bytes until you fill your data structure.

deleted 4y ago

[deleted]

leeoniya 4y ago

at least in the frontend (without WASM), it depends. a few months ago i tested https://github.com/mapbox/pbf https://github.com/mapbox/pbf and while it was faster for deep/complex structs vs an unoptimized/repetative JSON blob, it was much slower at shallow structs and flat arrays of stuff. if you spend a bit of time to encode stuff as flat arrays to avoid mem alloc, JSON parsing wins by a lot since it goes through highly optimized C or assembly, while decoding protobuf in the JS JIT does not. of course it's not always feasible to make optimized over-the-wire JSON structs if you have a huge/complex API that can return many shapes of complex structs.

mourner 4y ago
At pbf speeds, decoding is usually no longer a bottleneck, but bandwidth might be when comparing with gzipped JSON. Also, one of the primary advantages of pbf is being able to decode partially and lazily (and adjust how things are decoded at low level), which is very important in use cases like vector maps.
- leeoniya 4y ago
  > At pbf speeds, decoding is usually no longer a bottleneck, but bandwidth might be when comparing with gzipped JSON. we were streaming a few hundred float datapoints spread across a dozen(ish) flat arrays over websocket at 20-40hz and needed to decode the payload eagerly. plain JSON was a multi-factor speedup over pbf for this case. but it's fully possible i was holding it wrong, too! even when your "bottleneck" is rendering/rasterization (10ms), but your data pipe takes 3ms instead of 1ms, it's a big effect on framerate, battery, thermals, etc. i'm a big fan of your work! while i have you here, would you mind reviewing this sometime soon? ;) https://github.com/mourner/flatbush/pull/44 https://github.com/mourner/flatbush/pull/44