2 ms·

On the one hand, this is somewhat true, but on the other hand, there are lots of efficiencies you can get without trading resiliency, and the author sets up a f

On the one hand, this is somewhat true, but on the other hand, there are lots of efficiencies you can get without trading resiliency, and the author sets up a few false dichotomies here. The author mentions JSON vs struct serialization as an example, but flatbuffers and protobufs (or any binary protocol format that builds in an ID and a version number) give you the same resiliency benefits mentioned here. You don't have to go all the way to raw structs to gain efficiency over using JSON. Only the last tiny bit of efficiency (the gap between a struct and a flatbuffer) actually comes at a meaningful resiliency cost. The same goes for single-threaded vs multi-threaded services and single-machine vs horizontal scaling. With some thought, you can do all of these things and create highly-scalable systems very efficiently - not quite as efficiently as a single thread, but a lot more efficiently than the standard "web backend" thing of creating 1000 microservices using 10 different databases and 50 caching layers for that single task. That efficiency is available at no cost to resiliency. The common thread of "web" solutions like microservices and JSON is that they help you save developer time. That is orthogonal to the efficiency vs resiliency tradeoff.

anonymoushn 4y ago
This comes up a lot, do you have any benchmarks for protobuf deserialization that demonstrate that it is faster than json for the same structs? Protobuf seems much slower to decode due to the inability to know where an integer ends before one is done decoding it. Benchmarks I could find indicated protobuf decoding is maybe ~5x slower than json, but obviously this varies based on the payload and I'm not sure how much larger json encodings of the same data tend to be.
- akvadrako 4y ago
  How would you know where an integer in JSON ends without looking ahead? You won't even know if it is an integer or not. It seems like a more expensive operation than protobuf.
  - anonymoushn 4y ago
    You use about 2 SIMD instructions to compare 16 or 32 or 64 characters to whitespace, commas, and closing brackets, and one of these will be the character after the end of the integer literal. Theres no dependency between decoding the integer and tokenizing or decoding the next thing.
    - pclmulqdq 4y ago
      You can also use SIMD to parse varints (used by protobuf format).
sjducb 4y ago
I would question if the serialisation code speed actually matters. RAM and CPU are much much faster than an SSD so almost all (98%) of the time will be spent writing to disk. To speed it up you have to reduce the file size so that it can be written to disk faster. I think the fastest way would be a gzipped JSON file. You can do all of the converting to JSON and compression while you're waiting for the hard disk to finish writing.
- pkolaczk 4y ago
  > RAM and CPU are much much faster than an SSD so almost all (98%) of the time will be spent writing to disk. I wouldn't be so sure about it. Current SSDs can do gigabytes per second read and write. Sure, this is order of magnitude slower than theoretical sequential memory throughput, but might not be slower if you need to do some data processing on CPU or if you do memory access in an inefficient way. There are many ways you can screw it up and processing becomes the bottleneck. For example if you need to compress data, you might be already not be able to saturate SSD (most compression algorithms don't go faster than 1 GB/s maybe except LZ4). If you're not careful enough with heap allocations and you happen to create lot of tiny objects when deserializing, or you do lot of pointer chasing when serializing you may as well end up very far from theoretical maximum performance. I've seen systems maxing out at 10 MB/s just due to inefficient serde. Using plain text formats makes it somewhat worse.
- anonymoushn 4y ago
  I think this comment assumes very slow disks compared to the disks that are available.
bluepizza 4y ago
Serialization/deserialization is still quite costly even with protobufs/flat buffers. This is not the last tiny bit. In some applications, I've seen it take more than 15% of the total CPU usage.
- pclmulqdq 4y ago
  In some applications, this is true. I think 15% is pretty extreme, though, unless what you're doing is something like parsing/ creating data feeds (where the serialization is the point). In those cases, it's probably a good idea to have your own format. Still, JSON would be a lot worse for these cases than protobufs or flatbuffers. Also, are you sure you aren't compressing them if you're using 15% CPU?
  - bluepizza 4y ago
    Yeah, you almost got it :) It's a simple backing store for a few jobs that create data feeds. Agreed that JSON would be much worse. We have a hard requirement that all services must be Java based, and protobuf outperforms Gson and Jackson by miles. I have been trying to find the cheapest serialization/deserialization I could find for cache purposes. So far, the best option is a Guava/Caffeine cache, because we skip the serialization completely in this scenario, but so much more costly than having a good external cache.
    - pclmulqdq 4y ago
      If you are willing to do some DIY-ing, using finance-like encodings may be for you: See "Simple Binary Encoding" (https://www.fixtrading.org/standards/sbe/ https://www.fixtrading.org/standards/sbe/). The encoding scheme basically just uses structs with version numbering and IDs. There may be some Java code out there for SBE if you want to pull it off-the-shelf.
      - bluepizza 4y ago
        Fantastic! Thank you, this is really useful!

anonymoushn 4y ago

This comes up a lot, do you have any benchmarks for protobuf deserialization that demonstrate that it is faster than json for the same structs? Protobuf seems much slower to decode due to the inability to know where an integer ends before one is done decoding it. Benchmarks I could find indicated protobuf decoding is maybe ~5x slower than json, but obviously this varies based on the payload and I'm not sure how much larger json encodings of the same data tend to be.

akvadrako 4y ago
How would you know where an integer in JSON ends without looking ahead? You won't even know if it is an integer or not. It seems like a more expensive operation than protobuf.
- anonymoushn 4y ago
  You use about 2 SIMD instructions to compare 16 or 32 or 64 characters to whitespace, commas, and closing brackets, and one of these will be the character after the end of the integer literal. Theres no dependency between decoding the integer and tokenizing or decoding the next thing.
  - pclmulqdq 4y ago
    You can also use SIMD to parse varints (used by protobuf format).

sjducb 4y ago

I would question if the serialisation code speed actually matters. RAM and CPU are much much faster than an SSD so almost all (98%) of the time will be spent writing to disk. To speed it up you have to reduce the file size so that it can be written to disk faster. I think the fastest way would be a gzipped JSON file. You can do all of the converting to JSON and compression while you're waiting for the hard disk to finish writing.

pkolaczk 4y ago
> RAM and CPU are much much faster than an SSD so almost all (98%) of the time will be spent writing to disk. I wouldn't be so sure about it. Current SSDs can do gigabytes per second read and write. Sure, this is order of magnitude slower than theoretical sequential memory throughput, but might not be slower if you need to do some data processing on CPU or if you do memory access in an inefficient way. There are many ways you can screw it up and processing becomes the bottleneck. For example if you need to compress data, you might be already not be able to saturate SSD (most compression algorithms don't go faster than 1 GB/s maybe except LZ4). If you're not careful enough with heap allocations and you happen to create lot of tiny objects when deserializing, or you do lot of pointer chasing when serializing you may as well end up very far from theoretical maximum performance. I've seen systems maxing out at 10 MB/s just due to inefficient serde. Using plain text formats makes it somewhat worse.
anonymoushn 4y ago
I think this comment assumes very slow disks compared to the disks that are available.

bluepizza 4y ago

Serialization/deserialization is still quite costly even with protobufs/flat buffers. This is not the last tiny bit. In some applications, I've seen it take more than 15% of the total CPU usage.

pclmulqdq 4y ago
In some applications, this is true. I think 15% is pretty extreme, though, unless what you're doing is something like parsing/ creating data feeds (where the serialization is the point). In those cases, it's probably a good idea to have your own format. Still, JSON would be a lot worse for these cases than protobufs or flatbuffers. Also, are you sure you aren't compressing them if you're using 15% CPU?
- bluepizza 4y ago
  Yeah, you almost got it :) It's a simple backing store for a few jobs that create data feeds. Agreed that JSON would be much worse. We have a hard requirement that all services must be Java based, and protobuf outperforms Gson and Jackson by miles. I have been trying to find the cheapest serialization/deserialization I could find for cache purposes. So far, the best option is a Guava/Caffeine cache, because we skip the serialization completely in this scenario, but so much more costly than having a good external cache.
  - pclmulqdq 4y ago
    If you are willing to do some DIY-ing, using finance-like encodings may be for you: See "Simple Binary Encoding" (https://www.fixtrading.org/standards/sbe/ https://www.fixtrading.org/standards/sbe/). The encoding scheme basically just uses structs with version numbering and IDs. There may be some Java code out there for SBE if you want to pull it off-the-shelf.
    - bluepizza 4y ago
      Fantastic! Thank you, this is really useful!