6 ms·

Speeding up Go's built-in JSON encoder for large arrays of objects

110 points by eatonphil 5y ago

djanogo 5y ago
What is "large"?
- eatonphil 5y ago
  > And generated two datasets: one with 20 columns and 1M rows, and one with 1K columns and 10K rows. And if that's not big enough for you, the observed effects grow as I increased columns and rows.
latchkey 5y ago
I've been using jsoniter (another 'fastest' json lib) for ages. It is much faster than the built in, so 55% faster than slow isn't really meaningful. Surprised a more comprehensive benchmark evaluation was not done since this tends to be a pretty sensitive topic.
- eatonphil 5y ago
  Unlike other implementations, as far as I can tell, this one composes encoders. Under the hood it can use encoding/json.Marshal or goccy/go-json.Marshal (another existing very fast library) or any other libary that implements the json.Marshal call. This implementation is competitive (sometimes faster, sometimes slower) with good implementations like goccy/go-json on its own, and beats goccy/go-json when composing this library with goccy/go-json. This composition and goccy/go-json performance is included in the post. Maybe in a followup post I'll do more benchmarks against other fast libraries. But for this one I wanted to show the process and then just pick one fast library for comparison. Edit: also, I forgot to mention in the post but some libraries speed up encoding by requiring a fixed schema. DataStation/dsq is extremely dynamic and I'll never know the schema up front. Just another reason why I couldn't use some existing faster libraries.
  - latchkey 5y ago
    All I am saying is that the 'fastest' json libs all do comparisons against other 'fastest' json libs. I'd expect any other json lib trying to be faster to do those same comparisons and not produce arguably clickbait "55% faster" titles since your library isn't really that much faster than goccy. Pick the ones that match (dynamic) to compare against.
    - eatonphil 5y ago
      If you're not a fan of the blog post that's cool. But I posted the blog post (and wrote the post in the first place) rather than a Show HN link to the project itself because I thought the process was worth showing as much as the result.
      - mh- 5y ago
        > I thought the process was worth showing as much as the result it was, thanks for sharing.
      - strangeattractr 5y ago
        I enjoyed reading it, as someone who writes a lot of go but hasn't done any profiling of code.
      - kodah 5y ago
        I've known for some time that the JSON process of marshaling is expensive in Go; this post illuminates why and taught me how to test these kinds of things myself. It's excellent and I appreciate you sharing your methodology.
jjtheblunt 5y ago
i wonder what % of cpu cycles on AWS are spent encoding and decoding JSON.
- hhh 5y ago
  I wonder what % are just idle.
- throitallaway 5y ago
  Actually, this is horrifying to think about. I'll concede that JSON is the answer for most public HTTP APIs, but it's also used for tons of internal IPC. The energy that could be saved with more efficient alternatives is likely substantial.
  - _0w8t 5y ago
    While sending data between C++/Go/Java processes as JSON is a waste, when one part is a scripting language, JSON can be more efficient than alternatives. Recently I benchmarked extension API in Chromium. Those can send arbitrary data from C++ main browser process to the renderer process running extension JS code. Internally API use a binary format plus they verify that the data matches API scheme. It turned that for complex data writing the data to JSON on C++ side, sending the string using the same API and decoding that in JS was faster. The code to read binary data and convert that to JS plus the scheme verification was not optimized in Chromium, while JSON decoder was.
  - jjtheblunt 5y ago
    > is the answer i suspect you mean pragmatically for humans, since there is support for it in most libraries. but for concise equivalent marshalling, i wonder if YAML or TOML etc would be faster to parse. Or Amazon's Ion superset of JSON.
- nijave 5y ago
  I think a lot of the older APIs return XML and the SDKs abstract all that away. I imagine that uses even more
thricegreat 5y ago
I am building a web api for voidtools Everything in Go and this post came at the perfect time. The built-in JSON encoder isn't terrible but is noticeably slow. I am dealing with thousands to millions of objects for perspective.
pbohun 5y ago
Whenever I know I'm dealing with a large array of objects (and still want them to be user readable/editable) I typically choose to represent them in CSV format, sometimes with a special separation character. Is there an advantage besides convenience where one would use JSON? JSON requires far more bytes per row to represent large arrays of objects.
- eatonphil 5y ago
  Well CSV only has one datatype: string. JSON has a few more.
  - kroltan 5y ago
    Some people will curse me, but I have used JSON-over-CSV a few times, it's a good compromise if your data is indeed mostly tabular. To be clear, I mean encoding values as JSON, but the overall object structure in CSV, so something like: id,name,cool 1,"John Doe",false 2,"Daisy \"Obliterator of Worlds\" Wilson", true
    - saghm 5y ago
      This is super interesting. I'd imagine there's a pretty good reason that you picked this over something like an array of arrays of values or something like { "row1": [...], "row2": [...] }, but the only reason I can think of is if there was some of external constraint that made CSV necessary. Was it that simple, or was there something else?
      - kroltan 5y ago
        Embedded system with limited bandwidth, the data was mostly simple values that did not really need escaping 99% of the time (mostly no unicode or control characters in strings, but still _possible_, and we needed the streamability of array-of-structs instead of struct-of-arrays. Why not some binary format, then? Well, it's easier to debug and share the data with non-technical people (just open in excel or equivalent) without having to run a conversion step.
nosefrog 5y ago
To the author: heads up, the first code block in the "Infinite buffer" section doesn't match the text, the text is talking about using bytes.Buffer but the code is for the quoted columns caching. Neat blog post, btw :)
- eatonphil 5y ago
  Ooops, fixed. Thank you!
fullung 5y ago
Would love to see results from incorporating https://github.com/segmentio/encoding/tree/master/json https://github.com/segmentio/encoding/tree/master/json!

110 points by eatonphil 5y ago

djanogo 5y ago

What is "large"?

eatonphil 5y ago
> And generated two datasets: one with 20 columns and 1M rows, and one with 1K columns and 10K rows. And if that's not big enough for you, the observed effects grow as I increased columns and rows.

latchkey 5y ago

I've been using jsoniter (another 'fastest' json lib) for ages. It is much faster than the built in, so 55% faster than slow isn't really meaningful. Surprised a more comprehensive benchmark evaluation was not done since this tends to be a pretty sensitive topic.

eatonphil 5y ago
Unlike other implementations, as far as I can tell, this one composes encoders. Under the hood it can use encoding/json.Marshal or goccy/go-json.Marshal (another existing very fast library) or any other libary that implements the json.Marshal call. This implementation is competitive (sometimes faster, sometimes slower) with good implementations like goccy/go-json on its own, and beats goccy/go-json when composing this library with goccy/go-json. This composition and goccy/go-json performance is included in the post. Maybe in a followup post I'll do more benchmarks against other fast libraries. But for this one I wanted to show the process and then just pick one fast library for comparison. Edit: also, I forgot to mention in the post but some libraries speed up encoding by requiring a fixed schema. DataStation/dsq is extremely dynamic and I'll never know the schema up front. Just another reason why I couldn't use some existing faster libraries.
- latchkey 5y ago
  All I am saying is that the 'fastest' json libs all do comparisons against other 'fastest' json libs. I'd expect any other json lib trying to be faster to do those same comparisons and not produce arguably clickbait "55% faster" titles since your library isn't really that much faster than goccy. Pick the ones that match (dynamic) to compare against.
  - eatonphil 5y ago
    If you're not a fan of the blog post that's cool. But I posted the blog post (and wrote the post in the first place) rather than a Show HN link to the project itself because I thought the process was worth showing as much as the result.
    - mh- 5y ago
      > I thought the process was worth showing as much as the result it was, thanks for sharing.
    - strangeattractr 5y ago
      I enjoyed reading it, as someone who writes a lot of go but hasn't done any profiling of code.
    - kodah 5y ago
      I've known for some time that the JSON process of marshaling is expensive in Go; this post illuminates why and taught me how to test these kinds of things myself. It's excellent and I appreciate you sharing your methodology.

jjtheblunt 5y ago

i wonder what % of cpu cycles on AWS are spent encoding and decoding JSON.

hhh 5y ago
I wonder what % are just idle.
throitallaway 5y ago
Actually, this is horrifying to think about. I'll concede that JSON is the answer for most public HTTP APIs, but it's also used for tons of internal IPC. The energy that could be saved with more efficient alternatives is likely substantial.
- _0w8t 5y ago
  While sending data between C++/Go/Java processes as JSON is a waste, when one part is a scripting language, JSON can be more efficient than alternatives. Recently I benchmarked extension API in Chromium. Those can send arbitrary data from C++ main browser process to the renderer process running extension JS code. Internally API use a binary format plus they verify that the data matches API scheme. It turned that for complex data writing the data to JSON on C++ side, sending the string using the same API and decoding that in JS was faster. The code to read binary data and convert that to JS plus the scheme verification was not optimized in Chromium, while JSON decoder was.
- jjtheblunt 5y ago
  > is the answer i suspect you mean pragmatically for humans, since there is support for it in most libraries. but for concise equivalent marshalling, i wonder if YAML or TOML etc would be faster to parse. Or Amazon's Ion superset of JSON.
nijave 5y ago
I think a lot of the older APIs return XML and the SDKs abstract all that away. I imagine that uses even more

thricegreat 5y ago

I am building a web api for voidtools Everything in Go and this post came at the perfect time. The built-in JSON encoder isn't terrible but is noticeably slow. I am dealing with thousands to millions of objects for perspective.

pbohun 5y ago

Whenever I know I'm dealing with a large array of objects (and still want them to be user readable/editable) I typically choose to represent them in CSV format, sometimes with a special separation character. Is there an advantage besides convenience where one would use JSON? JSON requires far more bytes per row to represent large arrays of objects.

eatonphil 5y ago
Well CSV only has one datatype: string. JSON has a few more.
- kroltan 5y ago
  Some people will curse me, but I have used JSON-over-CSV a few times, it's a good compromise if your data is indeed mostly tabular. To be clear, I mean encoding values as JSON, but the overall object structure in CSV, so something like: id,name,cool 1,"John Doe",false 2,"Daisy \"Obliterator of Worlds\" Wilson", true
  - saghm 5y ago
    This is super interesting. I'd imagine there's a pretty good reason that you picked this over something like an array of arrays of values or something like { "row1": [...], "row2": [...] }, but the only reason I can think of is if there was some of external constraint that made CSV necessary. Was it that simple, or was there something else?
    - kroltan 5y ago
      Embedded system with limited bandwidth, the data was mostly simple values that did not really need escaping 99% of the time (mostly no unicode or control characters in strings, but still _possible_, and we needed the streamability of array-of-structs instead of struct-of-arrays. Why not some binary format, then? Well, it's easier to debug and share the data with non-technical people (just open in excel or equivalent) without having to run a conversion step.

nosefrog 5y ago

To the author: heads up, the first code block in the "Infinite buffer" section doesn't match the text, the text is talking about using bytes.Buffer but the code is for the quoted columns caching. Neat blog post, btw :)

eatonphil 5y ago
Ooops, fixed. Thank you!

fullung 5y ago

Would love to see results from incorporating https://github.com/segmentio/encoding/tree/master/json https://github.com/segmentio/encoding/tree/master/json!