7 ms·

Agreed. The headline leads people towards a deceptive conclusion. I've worked with both JSON and Protocol Buffers and there is no way that merely changing from

Agreed. The headline leads people towards a deceptive conclusion. I've worked with both JSON and Protocol Buffers and there is no way that merely changing from JSON to PB would reduce latency by 60%; not on an average request. Clearly other refactorings were undertaken to achieve that speedup. I've done some raw performance comparison tests between JSON and PB and the speedup with PB was insignificant for most of the payloads I tried. Does any developer here on HN really believe that JSON parsing (plus schema validation) is what adds the most latency to a request? It just doesn't add up that just switching to PB would deliver that speedup. It reminds me of another headline I read a few years ago about a company thanking TypeScript for reducing bugs by 25% (I can't remember what the exact number was) after a rewrite... Common. You rewrote the whole thing from scratch. What actually cut the bugs by 25% was the rewrite itself. People don't usually write it more buggy the second time... It's a pattern with these big tech consultants. They keep pushing specific tech solutions as silver bullets then they deliver flawed analyses of the results to propagate the myth that they solved the problem in that particular way... When in fact, the problem was solved by pure coincidence due to completely different factors.

avidphantasm 3y ago
Devil’s advocate: I would generally expect that a rewrite would increase bugs, at least at first.
- HankB99 3y ago
  I would expect that the target for a rewrite would be better known than the during the first write and that the morphing to hit a moving target would provide more opportunities for bugs. OTOH if there was a lot of pressure to get the rewrite done, that would be conducive to producing buggy code. I think management would be a bigger factor than any technical issues.
Cthulhu_ 3y ago
Other sources say protobuf is 6x as fast than JSON though. I mean with REST / JSON you get the overhead of converting data to JSON objects, JSON objects to text representation, text to gzip or whichever, then the transfer over TCP / HTTP (establish connection, handshake, etc), the actual transfer, and then gzip -> text -> json -> data again. Protobuf is a lot more direct and compact, you get a lot of those things for free, and that's even before considering the schema and code generation which is still far from ideal in the REST/JSON world. I don't believe that for the purposes of inter-service communication, REST/JSON can compete with protobuf. Caveat: I only have experience with REST/JSON and a little GraphQL, I haven't had the opportunity to work with protobuf yet. I'm more of a front end developer unfortunately, and I try to talk people out of doing microservices as much as possible.
- jongjong 3y ago
  I don't buy these arguments about 'This cheap computation is 6x faster than this other alternative cheap computation'. For example, in JavaScript, using standard 'for loops' with index numbers is a LOT faster (over 16 times faster for basic use cases) than using Array.forEach(), yet all the linters and consultants recommend using Array.forEach() instead of the standard for loop... What about latency??? Suddenly nobody cares about latency nor performance. The reason is that these operations which use marginal amounts of resources are pointless to refactor. If a function call which uses 1% of CPU time* (to service a standard request) has its performance improved by 'a whooping' 50%, then the program as a whole will use only 0.5% less CPU time than it did before. * (where all the other operations use the remaining 99%)
  - brvsft 3y ago
    > For example, in JavaScript, using standard 'for loops' with index numbers is a LOT faster (over 16 times faster for basic use cases) than using Array.forEach(), yet all the linters and consultants recommend using Array.forEach() instead of the standard for loop... What about latency??? Do you have proof of this claim? It smells like bs.
    - jongjong 3y ago
      If you have Node.js installed, you can run it yourself. I did run the experiment, that's how I came up with the 16x number. I just had the two kinds of loops iterating about 10 million times and just assigned a variable inside them (in the same way). I recorded the start and end times before and after the 10 million iterations (in each case) and found that Array.forEach took 16x longer to execute. The difference could be even higher if the operation inside the loop was cheaper since much of the cost of the loop depends on what logic is executed inside it. I kept it light (variable assignment is a cheap operation) but it's still an overly generous comparison. The real difference in performance of just the two kinds of loops are almost certainly greater than 16x. Ideally you should compare running the loops without any logic inside them but I was worried that optimizations in the JavaScript engine would cause it to just skip over the loops if they performed no computations and without any memory side effects. Anyway this was beside the point I was trying to make. My point is already proven; it makes no sense to optimize cheap operations.
dropofwill 3y ago
60% was just one request on average, could have been a huge payload that tended to be between data centers or something.
jlouis 3y ago
Parsing overhead is sensitive to the amount of work the service is doing. If the service is doing relatively little, and there's a lot of junk in your request, you are forced to parse a lot of data which you then throw away. JSON is particularly nasty because there's no way to skip over data you don't care about. To parse a string, you have to look at every byte. This can be parallelized (via SIMD), but there's limits. In contrast, protobuf sprays length encoded tags all over the place which allows you to quickly handle large sequences of strings and bytes. If your cache is hot and has a good hit-rate, the majority of your overhead is likely parsing. If you microbatch 100 requests, you have to parse 100 requests before you can ship them to the database for lookup (or the machine learning inference service). If the service is good at batch-processing, then the parsing becomes the latency-sensitive part. Note the caveat: the 60% is for large payloads. JSON contains a lot of repetition in the data, so you often see people add compression to JSON unknowingly, because their webserver is doing it behind their back. A fairly small request on the wire deflates to a large request in-memory, and takes way more processing time. That said, the statistician in me would like to have a distribution or interval rather than a number like "60%" because it is likely to vary. It's entirely possible that 60% is on the better end of what they are seeing (it's plausible in my book), but there's likely services where the improvement in latency is more mellow. If you want to reduce latency in a system, you should sample the distribution of processing latency. At least track the maximal latency over the last minute or so, preferably a couple of percentiles as well (95, 99, 99.9, ...).
- jongjong 3y ago
  Either way, the conclusion made by the headline is cherry-picked and is misleading for anyone who is concerned about average situations. From the article: > The result of Protocol Buffers adoption was an average increase in throughput by 6.25% for responses and 1.77% for requests. The team also observed up to 60% latency reduction for large payloads. It's very sneaky to describe throughput improvements using average requests/responses (which is what most people are interested in) but then switch to the 'worst case' request/response when describing latency... And doubly sneaky to then use that as the headline of the article.
  - 3y ago
3cats-in-a-coat 3y ago
There's no good reason to use JSON for internal communication long-term. Although I'm not a big fan of Protobuffs as well (too fragile and inflexible).
- topicseed 3y ago
  So what's your third option, if any
  - 3cats-in-a-coat 3y ago
    There are many options, and I'd be uncomfortable to suggest any in specific without knowing the project's details (I'll list a few below). The reason why switching JSON to Protobuff make me raise an eyebrow is because it represents a switch from one extreme to another. A mainstream, flexible, text-based protocol, to a specialized, rigid, binary protocol. When people do sudden moves like these, it's often misguided. I can almost hear the evangelists bashing how everything LinkedIn did with JSON is wrong, or something. In terms of formats, you'd get an easier transition and more balance between flexibility and efficiency out of BSON, Avro, Thrift, MessagePack. There are also alternatives to Protobuff like FlatBuffers and Cap'n Proto. There's also CBOR, which is interesting. There are also other ways of looking at the problem. How does Erlang serialize messages? It doesn't because it messages itself, so the message format is native to itself. And in fact I mostly lean in that direction, but it's not for everyone. Erlang is also dynamically typed, not the kind of language Protobuff and Cap'n Proto is aimed at I suppose.
    - ramchip 3y ago
      > How does Erlang serialize messages? It doesn't because it messages itself, so the message format is native to itself. I don't get the difference you're drawing... the in-memory and on-the-wire representation of terms are different, so there's still serialization involved (term_to_binary/1). The format is documented and there are libraries for other languages.
      - 3y ago
akdor1154 3y ago
As per TFA, the 60% was for huge payloads, maybe p99 or something. The mean benefit was much lower, but it's arguable that it's the slow ones that you care about speeding up the most.
IanCal 3y ago
> Does any developer here on HN really believe that JSON parsing (plus schema validation) is what adds the most latency to a request? It just doesn't add up that just switching to PB would deliver that speedup. I've absolutely had times when json serialising/deserialising was the vast majority of the request time.
- felixgallo 3y ago
  100%. In fact, I've usually found in my career that serialization/deserialization is the largest latency contributor, except when the API call makes remote database requests or something equally expensive. For the critical path of, say, game state updates, it's best to keep as much as you can in-memory on-box, so ridiculously expensive operations like json deserialization really stand out by comparison. Even for enterprise web crud calls, it's quite important when you're part of a giant ensemble of web calls, some of which may be serially dependent, to worry about things like this.
lallysingh 3y ago
> Does any developer here on HN really believe that JSON parsing (plus schema validation) is what adds the most latency to a request? It just doesn't add up that just switching to PB would deliver that speedup. 50k API endpoints means that probably a lot of them are pretty simple. The simpler the API, the higher percentage of it's time is spent in call overhead, which with JSON is parsing the entire input.
EMM_386 3y ago
> I've worked with both JSON and Protocol Buffers and there is no way that merely changing from JSON to PB would reduce latency by 60%; not on an average request. Clearly other refactorings were undertaken to achieve that speedup. The article notes that this is only for "large payloads", likely an edge case, and the average performance improvement is 6.25% for responses and 1.77% for requests. 1.77%! I get that this is "at scale" but is the additional complexity of all this engineering worth that? How much more difficult is the code to work with now? If it's at all more difficult to reason about, that is going to add to more engineering hours down the road to work with it. I assume tradeoffs like this were taken into account, and it was deemed that a <7% response improvement was worth it.

avidphantasm 3y ago

Devil’s advocate: I would generally expect that a rewrite would increase bugs, at least at first.

HankB99 3y ago
I would expect that the target for a rewrite would be better known than the during the first write and that the morphing to hit a moving target would provide more opportunities for bugs. OTOH if there was a lot of pressure to get the rewrite done, that would be conducive to producing buggy code. I think management would be a bigger factor than any technical issues.

Cthulhu_ 3y ago

Other sources say protobuf is 6x as fast than JSON though. I mean with REST / JSON you get the overhead of converting data to JSON objects, JSON objects to text representation, text to gzip or whichever, then the transfer over TCP / HTTP (establish connection, handshake, etc), the actual transfer, and then gzip -> text -> json -> data again. Protobuf is a lot more direct and compact, you get a lot of those things for free, and that's even before considering the schema and code generation which is still far from ideal in the REST/JSON world. I don't believe that for the purposes of inter-service communication, REST/JSON can compete with protobuf. Caveat: I only have experience with REST/JSON and a little GraphQL, I haven't had the opportunity to work with protobuf yet. I'm more of a front end developer unfortunately, and I try to talk people out of doing microservices as much as possible.

jongjong 3y ago
I don't buy these arguments about 'This cheap computation is 6x faster than this other alternative cheap computation'. For example, in JavaScript, using standard 'for loops' with index numbers is a LOT faster (over 16 times faster for basic use cases) than using Array.forEach(), yet all the linters and consultants recommend using Array.forEach() instead of the standard for loop... What about latency??? Suddenly nobody cares about latency nor performance. The reason is that these operations which use marginal amounts of resources are pointless to refactor. If a function call which uses 1% of CPU time* (to service a standard request) has its performance improved by 'a whooping' 50%, then the program as a whole will use only 0.5% less CPU time than it did before. * (where all the other operations use the remaining 99%)
- brvsft 3y ago
  > For example, in JavaScript, using standard 'for loops' with index numbers is a LOT faster (over 16 times faster for basic use cases) than using Array.forEach(), yet all the linters and consultants recommend using Array.forEach() instead of the standard for loop... What about latency??? Do you have proof of this claim? It smells like bs.
  - jongjong 3y ago
    If you have Node.js installed, you can run it yourself. I did run the experiment, that's how I came up with the 16x number. I just had the two kinds of loops iterating about 10 million times and just assigned a variable inside them (in the same way). I recorded the start and end times before and after the 10 million iterations (in each case) and found that Array.forEach took 16x longer to execute. The difference could be even higher if the operation inside the loop was cheaper since much of the cost of the loop depends on what logic is executed inside it. I kept it light (variable assignment is a cheap operation) but it's still an overly generous comparison. The real difference in performance of just the two kinds of loops are almost certainly greater than 16x. Ideally you should compare running the loops without any logic inside them but I was worried that optimizations in the JavaScript engine would cause it to just skip over the loops if they performed no computations and without any memory side effects. Anyway this was beside the point I was trying to make. My point is already proven; it makes no sense to optimize cheap operations.

dropofwill 3y ago

60% was just one request on average, could have been a huge payload that tended to be between data centers or something.

jlouis 3y ago

Parsing overhead is sensitive to the amount of work the service is doing. If the service is doing relatively little, and there's a lot of junk in your request, you are forced to parse a lot of data which you then throw away. JSON is particularly nasty because there's no way to skip over data you don't care about. To parse a string, you have to look at every byte. This can be parallelized (via SIMD), but there's limits. In contrast, protobuf sprays length encoded tags all over the place which allows you to quickly handle large sequences of strings and bytes. If your cache is hot and has a good hit-rate, the majority of your overhead is likely parsing. If you microbatch 100 requests, you have to parse 100 requests before you can ship them to the database for lookup (or the machine learning inference service). If the service is good at batch-processing, then the parsing becomes the latency-sensitive part. Note the caveat: the 60% is for large payloads. JSON contains a lot of repetition in the data, so you often see people add compression to JSON unknowingly, because their webserver is doing it behind their back. A fairly small request on the wire deflates to a large request in-memory, and takes way more processing time. That said, the statistician in me would like to have a distribution or interval rather than a number like "60%" because it is likely to vary. It's entirely possible that 60% is on the better end of what they are seeing (it's plausible in my book), but there's likely services where the improvement in latency is more mellow. If you want to reduce latency in a system, you should sample the distribution of processing latency. At least track the maximal latency over the last minute or so, preferably a couple of percentiles as well (95, 99, 99.9, ...).

jongjong 3y ago
Either way, the conclusion made by the headline is cherry-picked and is misleading for anyone who is concerned about average situations. From the article: > The result of Protocol Buffers adoption was an average increase in throughput by 6.25% for responses and 1.77% for requests. The team also observed up to 60% latency reduction for large payloads. It's very sneaky to describe throughput improvements using average requests/responses (which is what most people are interested in) but then switch to the 'worst case' request/response when describing latency... And doubly sneaky to then use that as the headline of the article.
- 3y ago

3cats-in-a-coat 3y ago

There's no good reason to use JSON for internal communication long-term. Although I'm not a big fan of Protobuffs as well (too fragile and inflexible).

topicseed 3y ago
So what's your third option, if any
- 3cats-in-a-coat 3y ago
  There are many options, and I'd be uncomfortable to suggest any in specific without knowing the project's details (I'll list a few below). The reason why switching JSON to Protobuff make me raise an eyebrow is because it represents a switch from one extreme to another. A mainstream, flexible, text-based protocol, to a specialized, rigid, binary protocol. When people do sudden moves like these, it's often misguided. I can almost hear the evangelists bashing how everything LinkedIn did with JSON is wrong, or something. In terms of formats, you'd get an easier transition and more balance between flexibility and efficiency out of BSON, Avro, Thrift, MessagePack. There are also alternatives to Protobuff like FlatBuffers and Cap'n Proto. There's also CBOR, which is interesting. There are also other ways of looking at the problem. How does Erlang serialize messages? It doesn't because it messages itself, so the message format is native to itself. And in fact I mostly lean in that direction, but it's not for everyone. Erlang is also dynamically typed, not the kind of language Protobuff and Cap'n Proto is aimed at I suppose.
  - ramchip 3y ago
    > How does Erlang serialize messages? It doesn't because it messages itself, so the message format is native to itself. I don't get the difference you're drawing... the in-memory and on-the-wire representation of terms are different, so there's still serialization involved (term_to_binary/1). The format is documented and there are libraries for other languages.
    - 3y ago

akdor1154 3y ago

As per TFA, the 60% was for huge payloads, maybe p99 or something. The mean benefit was much lower, but it's arguable that it's the slow ones that you care about speeding up the most.

IanCal 3y ago

> Does any developer here on HN really believe that JSON parsing (plus schema validation) is what adds the most latency to a request? It just doesn't add up that just switching to PB would deliver that speedup. I've absolutely had times when json serialising/deserialising was the vast majority of the request time.

felixgallo 3y ago
100%. In fact, I've usually found in my career that serialization/deserialization is the largest latency contributor, except when the API call makes remote database requests or something equally expensive. For the critical path of, say, game state updates, it's best to keep as much as you can in-memory on-box, so ridiculously expensive operations like json deserialization really stand out by comparison. Even for enterprise web crud calls, it's quite important when you're part of a giant ensemble of web calls, some of which may be serially dependent, to worry about things like this.

lallysingh 3y ago

> Does any developer here on HN really believe that JSON parsing (plus schema validation) is what adds the most latency to a request? It just doesn't add up that just switching to PB would deliver that speedup. 50k API endpoints means that probably a lot of them are pretty simple. The simpler the API, the higher percentage of it's time is spent in call overhead, which with JSON is parsing the entire input.

EMM_386 3y ago

> I've worked with both JSON and Protocol Buffers and there is no way that merely changing from JSON to PB would reduce latency by 60%; not on an average request. Clearly other refactorings were undertaken to achieve that speedup. The article notes that this is only for "large payloads", likely an edge case, and the average performance improvement is 6.25% for responses and 1.77% for requests. 1.77%! I get that this is "at scale" but is the additional complexity of all this engineering worth that? How much more difficult is the code to work with now? If it's at all more difficult to reason about, that is going to add to more engineering hours down the road to work with it. I assume tradeoffs like this were taken into account, and it was deemed that a <7% response improvement was worth it.