3 ms·

With compression, payload size differences aren't that significant, and don't necessarily favour other formats (e.g. almost none of the formats have particularl

With compression, payload size differences aren't that significant, and don't necessarily favour other formats (e.g. almost none of the formats have particularly compact representations of strings, and single decimal digit integer values are common enough that integer fields are not that wasteful in JSON). I totally hear you on serialization/deserialization speed, though it is amazing how inefficient some of the "machine readible" implementations can be, and how efficient people have been able to get JSON parsing, its horrible design disadvantage in this area is lethal. The simplicity argument is an amusing one to me. I think a lot of people build simple client/server code for JSON, but these "simple" implementations have all kinds of rough edges that prove problematic long term. Once you start doing things right, JSON proves to be far more complex.

secondcoming 6y ago
Are you including the size of the typically useless HTTP headers when you talk about 'payload size', or just the actual payload in the HTTP request itself? I really mean 'typically useless' in the scenario where the communication is being done internally, not when you're actually communicating over the public internet. There can be quite a bit of wastage in this scenario. I really don't care about the 'Date' header or the 'Server' header, but the HTTP spec does, so they're there.
- cbsmith 6y ago
  Thanks to all that complexity in HTTP (particularly HTTP2), there is no need to pay that headers size cost in most cases, particularly ones where bandwidth tends to be a concern. Yes, naive and "simple" implementations totally waste a TON of space in headers, and even if you exploit all that complexity HTTP is still inefficient in terms of network bandwidth consumption, much as even compressed JSON is still inefficient with its use of bytes. All I'm saying is that while JSON is horribly inefficient for some payloads, the reality is that most serialization formats tend to be efficient for specific kinds of payloads, but still have plenty of common cases they don't attempt to be efficient with. For example, machine readable formats like protobuf, thrift, avro, etc. employ some variation on varint encoding to keep small integers compactly represented, they encode strings relatively inefficiently; They encode field type & number, followed by a varint for length, followed by uncompressed UTF-8 encoded payload... and I've found that if anything it seems once you go through the compression ringer it often isn't much different in terms of space consumption vs. JSON... and sometimes it can be less efficient. Given how often fields end up being encoded as strings (even though they should be enums, ints, etc.), this has resulted in a lot of surprising cases for teams who have tried switching away from JSON to reduce payload sizes. If you cared about efficiency, you'd probably have a special encoding mechanism for short strings, and then a reasonably compact (but fast to parse) representation for longer strings, probably using something like SCSU encoding instead of UTF-8.

secondcoming 6y ago

Are you including the size of the typically useless HTTP headers when you talk about 'payload size', or just the actual payload in the HTTP request itself? I really mean 'typically useless' in the scenario where the communication is being done internally, not when you're actually communicating over the public internet. There can be quite a bit of wastage in this scenario. I really don't care about the 'Date' header or the 'Server' header, but the HTTP spec does, so they're there.

cbsmith 6y ago
Thanks to all that complexity in HTTP (particularly HTTP2), there is no need to pay that headers size cost in most cases, particularly ones where bandwidth tends to be a concern. Yes, naive and "simple" implementations totally waste a TON of space in headers, and even if you exploit all that complexity HTTP is still inefficient in terms of network bandwidth consumption, much as even compressed JSON is still inefficient with its use of bytes. All I'm saying is that while JSON is horribly inefficient for some payloads, the reality is that most serialization formats tend to be efficient for specific kinds of payloads, but still have plenty of common cases they don't attempt to be efficient with. For example, machine readable formats like protobuf, thrift, avro, etc. employ some variation on varint encoding to keep small integers compactly represented, they encode strings relatively inefficiently; They encode field type & number, followed by a varint for length, followed by uncompressed UTF-8 encoded payload... and I've found that if anything it seems once you go through the compression ringer it often isn't much different in terms of space consumption vs. JSON... and sometimes it can be less efficient. Given how often fields end up being encoded as strings (even though they should be enums, ints, etc.), this has resulted in a lot of surprising cases for teams who have tried switching away from JSON to reduce payload sizes. If you cared about efficiency, you'd probably have a special encoding mechanism for short strings, and then a reasonably compact (but fast to parse) representation for longer strings, probably using something like SCSU encoding instead of UTF-8.