5 ms·

I'd hardly call a few bytes per key and field excessive. Especially compared to something like XML. The datastructure complexity being limited is also a pretty

I'd hardly call a few bytes per key and field excessive. Especially compared to something like XML. The datastructure complexity being limited is also a pretty significant key to its success. More complex datatypes means greater chances for JSON handling libraries to lack compatibility. The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's 99.99% of the time not read by a human.

cbsmith 6y ago
JSON looks good when compared to XML. That's literally how low you have to go. "The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's not 99.99% of the time not read by a human." There are a few other shortcomings that are reasonably substantial/significant, but yeah, that's the gist of the problem.
- msla 6y ago
  > JSON looks good when compared to XML. JSON looks good compared to what we'd be using instead of JSON, which is nothing so nice and structured as XML. The competition to JSON is something infinitely more ad-hoc, probably without a distinct parser, such that a generic library to generate or consume it is impossible, and getting usable error messages is equally impossible. > "The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's not 99.99% of the time not read by a human." I agree with this and disagree at the same time: Optimizing for human readability means optimizing for the weird case, the 0.01% (but it seems to be more often than that) of the time you need to go beyond the tools you have to fix something. Saying that's rare is true but inapt: Seatbelts are only used in rare cases, too.
  - nitrogen 6y ago
    If we didn't have JSON we'd have settled on something like MsgPack.
  - cbsmith 6y ago
    Your 0.01% case is perhaps true with a binary format, but by using JSON, that 0.01% case just became a whole lot larger. ;-) It's like, "hey, we came up with a simple way to do it, but to make it easier to deal with this little edge condition that creates complexity, let's significantly up the complexity and number of edge conditions so they're endemic to the space, and then we're all good go". The irony is, I invariably end up needing to use a computer to help me read JSON anyway.
- kerkeslager 6y ago
  Even if we agree with your made-up numbers, 0.01% of the time that it's read by a human costs orders of magnitude more than the other 99.99%.
  - hackcasual 6y ago
    If anything 99.99% is probably underestimating it, especially for larger companies. If you send 10 million JSON documents per day, have 100 devs, and devs on average inspect 1 JSON document on the wire once per week, you're looking at closer to 99.9999%. Let's say that a binary format saves on average 10ms over JSON, then those 10m documents represent slightly over a day of overhead. If you've got good built in tooling for payload visualization, then you might have minimal overhead to debug from a text like format. Both protobufs and flatbuffers (not to mention BSON), have good tools that spit out JSON equivalents.
    - kerkeslager 6y ago
      Sure, if you make up numbers, you can argue that the sun is going to crash into the earth tomorrow and we're all going to die, so nothing in this conversation matters. In reality however, there are some cases where protobufs, flatbuffers, or BSON are superior to JSON, but there are a lot of cases where they aren't. You'll have to weigh the pros and cons for each situation. And a lot of the time, there's not time to benchmark everything, so you kind of have to guess how the elements of the system are going to interact. The one element that every system has is humans, so it's a fairly safe bet that humans will have to read whatever format you use. I spend probably 15-45 minutes a day just in Postman, testing JSON calls. If something goes wrong, I'm inspecting requests/responses in Chrome. When we integrate a new team member, we don't have to have them install any tools--they're included in the browser they have installed. We don't have to write any schemas. When I start a new project, I don't have to install any libraries: they're included in my language(s). When we integrate with a partner company, we hand them sample requests/responses as text. How many JSON documents per day do we have to send to get the payoffs you're claiming? And my company is not unique: in fact, the stack I'm using is one of the most common stacks on the market.
- Nelson69 6y ago
  Has anyone calculated the carbon foot print of parsing JSON?
ryanar 6y ago
I read and work with JSON all the time, logs, responses, code generated from json data. The format suffers from not being readable because of the quotes issue. Especially when what you are putting in there has quotes, the amount of escaping required is ridiculous. {"time":"2020-07-22T10:59:14.95406-04:00","message":"{\"level\":\"debug\",\"module\":\"system\",\"time\":\"2020-07-22T10:59:14.953909-04:00\",\"message\":\"Running MetricCollector.Flush()\"}"} this is a very moderate example of what I deal with daily, all because JSON includes quotes around fields.
- rileymat2 6y ago
  It gets worse when you want to put it into a c-string and you need to escape the quotes and the slashes again.
  - ryanar 6y ago
    exactly, I am excited about new formats like Amazon's Ion though https://github.com/amzn/ion-js https://github.com/amzn/ion-js
- blackrock 6y ago
  They embedded a JSON string within the JSON itself. I wonder if it would’ve been better for them to Base64 encode their message. Of course, this itself presents other problems.
mercer 6y ago
Yeah but that 0.01% it's developers checking API output and whatnot, and being able to read the output without any tooling (or maybe just a JSON 'prettier' tool) is great.

cbsmith 6y ago

JSON looks good when compared to XML. That's literally how low you have to go. "The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's not 99.99% of the time not read by a human." There are a few other shortcomings that are reasonably substantial/significant, but yeah, that's the gist of the problem.

msla 6y ago
> JSON looks good when compared to XML. JSON looks good compared to what we'd be using instead of JSON, which is nothing so nice and structured as XML. The competition to JSON is something infinitely more ad-hoc, probably without a distinct parser, such that a generic library to generate or consume it is impossible, and getting usable error messages is equally impossible. > "The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's not 99.99% of the time not read by a human." I agree with this and disagree at the same time: Optimizing for human readability means optimizing for the weird case, the 0.01% (but it seems to be more often than that) of the time you need to go beyond the tools you have to fix something. Saying that's rare is true but inapt: Seatbelts are only used in rare cases, too.
- nitrogen 6y ago
  If we didn't have JSON we'd have settled on something like MsgPack.
- cbsmith 6y ago
  Your 0.01% case is perhaps true with a binary format, but by using JSON, that 0.01% case just became a whole lot larger. ;-) It's like, "hey, we came up with a simple way to do it, but to make it easier to deal with this little edge condition that creates complexity, let's significantly up the complexity and number of edge conditions so they're endemic to the space, and then we're all good go". The irony is, I invariably end up needing to use a computer to help me read JSON anyway.
kerkeslager 6y ago
Even if we agree with your made-up numbers, 0.01% of the time that it's read by a human costs orders of magnitude more than the other 99.99%.
- hackcasual 6y ago
  If anything 99.99% is probably underestimating it, especially for larger companies. If you send 10 million JSON documents per day, have 100 devs, and devs on average inspect 1 JSON document on the wire once per week, you're looking at closer to 99.9999%. Let's say that a binary format saves on average 10ms over JSON, then those 10m documents represent slightly over a day of overhead. If you've got good built in tooling for payload visualization, then you might have minimal overhead to debug from a text like format. Both protobufs and flatbuffers (not to mention BSON), have good tools that spit out JSON equivalents.
  - kerkeslager 6y ago
    Sure, if you make up numbers, you can argue that the sun is going to crash into the earth tomorrow and we're all going to die, so nothing in this conversation matters. In reality however, there are some cases where protobufs, flatbuffers, or BSON are superior to JSON, but there are a lot of cases where they aren't. You'll have to weigh the pros and cons for each situation. And a lot of the time, there's not time to benchmark everything, so you kind of have to guess how the elements of the system are going to interact. The one element that every system has is humans, so it's a fairly safe bet that humans will have to read whatever format you use. I spend probably 15-45 minutes a day just in Postman, testing JSON calls. If something goes wrong, I'm inspecting requests/responses in Chrome. When we integrate a new team member, we don't have to have them install any tools--they're included in the browser they have installed. We don't have to write any schemas. When I start a new project, I don't have to install any libraries: they're included in my language(s). When we integrate with a partner company, we hand them sample requests/responses as text. How many JSON documents per day do we have to send to get the payoffs you're claiming? And my company is not unique: in fact, the stack I'm using is one of the most common stacks on the market.
Nelson69 6y ago
Has anyone calculated the carbon foot print of parsing JSON?

ryanar 6y ago

I read and work with JSON all the time, logs, responses, code generated from json data. The format suffers from not being readable because of the quotes issue. Especially when what you are putting in there has quotes, the amount of escaping required is ridiculous. {"time":"2020-07-22T10:59:14.95406-04:00","message":"{\"level\":\"debug\",\"module\":\"system\",\"time\":\"2020-07-22T10:59:14.953909-04:00\",\"message\":\"Running MetricCollector.Flush()\"}"} this is a very moderate example of what I deal with daily, all because JSON includes quotes around fields.

rileymat2 6y ago
It gets worse when you want to put it into a c-string and you need to escape the quotes and the slashes again.
- ryanar 6y ago
  exactly, I am excited about new formats like Amazon's Ion though https://github.com/amzn/ion-js https://github.com/amzn/ion-js
blackrock 6y ago
They embedded a JSON string within the JSON itself. I wonder if it would’ve been better for them to Base64 encode their message. Of course, this itself presents other problems.

mercer 6y ago

Yeah but that 0.01% it's developers checking API output and whatnot, and being able to read the output without any tooling (or maybe just a JSON 'prettier' tool) is great.