6 ms·

Nobody gets fired for picking JSON, but maybe they should?

the_cramer 2y ago
I feel like many of the points are complaining about the parsing side of JSON. Not the format itself. You can argue that a format is useless when "everyone" parses it "wrong" but no specification on this earth is free of that. Using a lot of json in our API space and it working fine (so far) leads me to think that OP complains about something that does not fit their use-case. Firing people for choosing something that does not fit "your" use-case seems like a wild take.
- flohofwoe 2y ago
  It's just a dig at the old saying "Nobody ever got fired for buying IBM" (e.g. for picking the popular solution even if it is has obvious downsides)
  - solidasparagus 2y ago
    Sure. But then they added "maybe they should be". Yeah, it's a clickbait title not backed up by the content, but I think it's fair to criticize the words the author chose to write.
- cstrahan 2y ago
  To add onto the sibling comment: It is indeed a play on the saying “Nobody ever got fired for buying IBM”. What wasn’t explained, though, is the meaning behind the expression. It’s an expression used to mock use of the bandwagon fallacy and a general lack of critical thought in the decision process. If you’re responsible for choosing a vendor, the easy out is to pick one of the largest vendors available: if it’s what everyone else is doing, then you can waive away any personal accountability for any resulting failures by claiming that “it’s what everyone else is doing, so it must have been the right choice, and therefore the failure that occurred must have been inherent to the problem domain, rather than something that could have been avoided if I personally chose better”. The phrase now gets used to mock such bandwagon behaviors: a CTO completely unfamiliar with Kubernetes, yet choosing it “because everyone else is using it”; an engineer picking a serialization format based on popularity while never having read the spec; etc. The article isn’t suggesting that anyone actually be fired. It is, however, critical of people choosing JSON due to familiarity/popularity, without any critical thought involved in choice (and bandwagon fallacy does not count as critical thought).
solidasparagus 2y ago
I appreciate the blog post and I learned a bunch from it! But the quote that comes to mind is: "People who say it cannot be done should not interrupt those who are doing it" It's quite a stretch to say that JSON is broken and can't be fixed. JSON works exceptionally well in practice and any review of it that fails to acknowledge this is, IMO, very lacking in perspective.
- cstrahan 2y ago
  A similar argument: If you look at the statistics, you’ll find that casual, condomless sex is very common. Despite the risks of STI transmission, it often pans out okay: most STIs are readily curable, while HIV has a low transmission rate amongst heterosexual couples (about 0.08% to 0.2% per sexual act) while PrEP is pretty common for homosexual men, and HSV has fairly low odds of transmission outside of an active outbreak (annual odds of transmission amongst couples: about 11–17% for couples with a male source partner and 3–4% with a female partner). Consequently, many people who have casual, condomless sex with many, many partners will end up just fine. Is it reckless? Well, the connotation of “reckless” is fairly negative, and I don’t see why I should get to judge what two consenting adults decide is an acceptable risk vs reward for themselves (at least, so long as I’m not involved). However, when we pivot back to the domain of software/engineering: when software design choices are made via bandwagon fallacy, claiming that “it usually works out okay”, I do find that to be reckless. You may be fine with such a carefree approach, but it isn’t really fair to other engineers, users, and stake holders. It’s not that I believe JSON (or similarly pitfall-laden technology) should be strictly avoided, but rather that the risks and failure modes should be given serious consideration, rather than minimized or outright dismissed. In terms of the STI analogy, it’s perfectly reasonable for two individuals to be aware of the risks, and depending on their appetite for said risk, agree upon the inclusive/exclusive nature of their relationship, as well as whether they exchange test results; what would be ridiculous is to pretend that the risk of infection is zero. There would be zero value in the article minimizing the design flaws (and resulting footguns) in JSON. The insistence that the article should do so is about as bizarre as someone responding to an article on safer sex by insisting that the article be amended with a note that “… but raw-dogging strangers isn’t too risky anyway, so, like, YOLO”.
catlifeonmars 2y ago
Go can actually decode numbers into an “arbitrary precision” raw token (json.Number, which is a wrapper around []byte). You can turn this on at the decoder level or for a particular field. I often do this for more fine grained control of number decoding without needing to drop to the tokenizer.
AtlasBarfed 2y ago
Jason can't be streamed: wtf? If you have a data connection that can be broken, every data format is incomplete until the last ack There is absolutely nothing stopping you from parsing json as it comes in. An open array that hasn't reached the closing arrays character yet? How is that different than a stream of data elements? With the Jackson API, I was easily able to parse incoming text into Json tokens and extract data as it came in. I did this for certain corner cases where Json documents possibly were sent in that were greater than could be practically put into memory. Sure, you might want to have a rollback function in case there is a break in the data transfer, but how'd that be different with a different data format streamed? The guy is complaining a lot about number formats. If you care about it that much just put your numbers and strings, and make the parsing format part of the standard /api interface. This guy simply appears to be in love with binary formats and strong typing. If that floats his boat by all means do it. The fact that so much of the web relies on Json, and it pretty much seems to work. The guy hopefully indicated some corner cases on its use. But come on it's not fatally flawed.
- dekhn 2y ago
  I stream JSON as a series of closed JSON documents. Parsing incomplete JSON just seems like a minefield of errors.
buzzardbait 2y ago
The complaint is mostly about JSON parsers, not the JSON spec itself, which as other comments have said works quite well in practice. In .NET land the most popular JSON library is Newtonsoft which plays fast and loose with the JSON spec and makes a lot of default assumptions. A couple of years ago Microsoft rolled out their own first-party JSON library, which is a lot stricter and requires the developer to explicitly configure the parser instead of making assumptions. https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/migrate-from-newtonsoft?pivots=dotnet-9-0#table-of-differences https://learn.microsoft.com/en-us/dotnet/standard/serializat... But some developers still instinctively reach for Newtonsoft, even when working on brand new projects. In practice, this sort of ingrained habit is far greater of a problem than the JSON spec itself. The author says that Protobuf makes all of JSON's problems go away. I don't know enough about Protobuf to confirm or deny, but I know that it's not tested 10 trillion times a day like JSON is.
- theFco 2y ago
  The argument that it is tested 10 trillion times a day is a fallacy (because it can't be that only the most popular option should be the best forever). And furthermore, in this context "tested" is a bit of an overstatement, I would say "used" many gazillions of times a day.
  - buzzardbait 2y ago
    Using is testing. The whole point of automated UI tests is to mimic usage. A random click on a website might not be a great test, but it is a test nonetheless. And if it fails enough times, someone somewhere will be held accountable.
  - dogboat 2y ago
    It is used so much across the board for FE and BE and CI and config. Despite the heavy usage I it hasn't been canned and I have never heard anyone at work complain about JSON for any reason. You could say it is network effects but if it were crap you could replace it much more easily than say moving from Python to Java or whatever. Especially for internal microservice stuff and perhaps front end. JSON and the tooling is basically solid. It is a non-concern.
nijave 2y ago
s/JSON/CSV/
s6af7ygt 2y ago
I agree with all of the points, except the last. Protobuf is a nice transfer format but it's horrible once a human wants to inspect or copy/paste a payload.
- nightowl_games 2y ago
  Exactly. And this is an extremely important detail. It makes your development process go from using glass tubes where you see everything as it is sent/received to black boxes of binary data. JSON is self describing, you can mostly just call the API, read the JSON response and figure it out from there. Ignoring that detail is ignoring the elephant in the room.
  - solidasparagus 2y ago
    The author was a tech lead on protobuf. I suspect that "JSON is worse than protobuf" is an obvious truth in their eyes.
p4cmanus3r 2y ago
PHP sucks too
- ramon156 2y ago
  One million reddit gold to you
  - devoutsalsa 2y ago
    “I like Elixir.” “Elixir has been an absolute joy to work with…”
dainiusse 2y ago
s/JSON/YAML/ :)
exabrial 2y ago
I wish json docs had a schema you exchange for your objects, and evaluate compliance against the schema. XSDs one of the only redeeming qualities of XML.
- ivell 2y ago
  You can use JSONSchema, https://json-schema.org/ https://json-schema.org/
- halfcat 2y ago
  Does JSON Schema help here? https://json-schema.org/ https://json-schema.org/
- dekhn 2y ago
  I've worked with json schema and have mixed opinions- it was extremely cumbersome to do some things that should be simple, and very simple to do things that should be extremely cumbersome. I agree that XSD was one of the only redeeming qualities of XML- the schema schema itself was fairly well thought-through. Personally I prefer the IDL-style languages- that would include CORBA (just the schema and encoding), protobufs, and the ur-IDL that Sun RPC was based on, XDR. Protobufs have gone through some weird evolution, though- IIRC proto3 went tyhrough a full 180 on required/optional fields.
deleted 2y ago
[deleted]
dzaima 2y ago
Much as JSON is without a doubt far from perfect, I think most of the article is rather meh. Protobuf won't stop people from trying to put a license keys, card numbers, or other IDs in a 64-bit float or int. All it'll do is make someone be able to smugly point to a certain person and say "you're stupid, you did this incorrectly as per the spec". The extra assurance could even make things worse, e.g. using 64-bit ints for credit card numbers and having something break that interprets it as a signed int (or perhaps become unusable if 20-digit numbers happen), or have code that converts it to a float anyway, devolving back to 53-bit precision. The "fix" for both formats for normal people is still exactly the same - "please please please don't put things that aren't strictly counts of things or imprecision-tolerant real numbers in number types". Unpaired surrogates is just a question of which component fails. If any! An arbitrary user input containing an unpaired surrogate would likely be better off quietly transformed (or preserved) than become a DoS vector. Lack of NaNs, infinities, and compact ways to encode arbitrary bytes are the things I personally find the most unfortunate. The special constants are especially weird as null/false/true exist, but byte strings do have the problem that not all languages have a reasonable distinct thing for such to map to. A funky side-note is that JSON being so tolerant/imprecise (and as such most suppliers not pushing it to edge-cases) means that crappy parsers/producers aren't typically particularly problematic, though this perhaps isn't really a thing to judge data formats by.
scotty79 2y ago
I really love the minimap idea on this website. I think it's the first time ever that I see it.
jendjcneeb 2y ago
As many have said, the post is complaining about broken parsers and broken data structures, not a broken spec. An example for this is using a map instead of a key-value-list for object representation Or float instead of arbitrary precision decimals Are there more broken json libraries out there, then there are broken XML libraries? Maybe. But that's not a problem with the spec. Choosing JSON if your use case is well suited is fine nevertheless Just don't use obviously broken libtaries
- dented42 2y ago
  But the libraries aren’t broken, they follow the spec. The spec is vague enough that compliant implementations can be mutually incompatible, that’s the main criticism as far as I can see.
lakomen 2y ago
Or maybe they shouldn't
avmich 2y ago
> Protobuf, however, has none of these problems I like this optimism :) .
Glyptodon 2y ago
I think in many cases people are just using JSON because they are getting it for free, not because they actively picked it.
Sohcahtoa82 2y ago
This article reeks of being written by someone with maybe a year or two of experience that's read someone else's ill-informed rants about JSON and is rehashing it. IMO, the only valid complaint is that the JSON spec can't handle Inf or NaN, but I'd argue that if you're finding yourself dealing with Inf or NaN, then you likely have a bug elsewhere, and JSON is merely exposing it. If you're storing credit card numbers, license keys, barcode IDs, etc. as integers, you're already doing it wrong, and what serialization format you're using is completely irrelevant.
- CRConrad 2y ago
  Totally concur with your last paragraph. But, considering that the author is apparently one of the people behind the competing format mentioned, protobuf, your first paragraph feels way off.
- nijave 2y ago
  Yeah, I like the "if you can't do math on it, it probably shouldn't be a numeric data type" rule of thumb i.e. applying an average to a list of credit card numbers doesn't make any sense nor does adding commas
donnaalbert 2y ago
[dead]

I feel like many of the points are complaining about the parsing side of JSON. Not the format itself. You can argue that a format is useless when "everyone" parses it "wrong" but no specification on this earth is free of that. Using a lot of json in our API space and it working fine (so far) leads me to think that OP complains about something that does not fit their use-case. Firing people for choosing something that does not fit "your" use-case seems like a wild take.

flohofwoe 2y ago
It's just a dig at the old saying "Nobody ever got fired for buying IBM" (e.g. for picking the popular solution even if it is has obvious downsides)
- solidasparagus 2y ago
  Sure. But then they added "maybe they should be". Yeah, it's a clickbait title not backed up by the content, but I think it's fair to criticize the words the author chose to write.
cstrahan 2y ago
To add onto the sibling comment: It is indeed a play on the saying “Nobody ever got fired for buying IBM”. What wasn’t explained, though, is the meaning behind the expression. It’s an expression used to mock use of the bandwagon fallacy and a general lack of critical thought in the decision process. If you’re responsible for choosing a vendor, the easy out is to pick one of the largest vendors available: if it’s what everyone else is doing, then you can waive away any personal accountability for any resulting failures by claiming that “it’s what everyone else is doing, so it must have been the right choice, and therefore the failure that occurred must have been inherent to the problem domain, rather than something that could have been avoided if I personally chose better”. The phrase now gets used to mock such bandwagon behaviors: a CTO completely unfamiliar with Kubernetes, yet choosing it “because everyone else is using it”; an engineer picking a serialization format based on popularity while never having read the spec; etc. The article isn’t suggesting that anyone actually be fired. It is, however, critical of people choosing JSON due to familiarity/popularity, without any critical thought involved in choice (and bandwagon fallacy does not count as critical thought).

solidasparagus 2y ago

I appreciate the blog post and I learned a bunch from it! But the quote that comes to mind is: "People who say it cannot be done should not interrupt those who are doing it" It's quite a stretch to say that JSON is broken and can't be fixed. JSON works exceptionally well in practice and any review of it that fails to acknowledge this is, IMO, very lacking in perspective.

cstrahan 2y ago
A similar argument: If you look at the statistics, you’ll find that casual, condomless sex is very common. Despite the risks of STI transmission, it often pans out okay: most STIs are readily curable, while HIV has a low transmission rate amongst heterosexual couples (about 0.08% to 0.2% per sexual act) while PrEP is pretty common for homosexual men, and HSV has fairly low odds of transmission outside of an active outbreak (annual odds of transmission amongst couples: about 11–17% for couples with a male source partner and 3–4% with a female partner). Consequently, many people who have casual, condomless sex with many, many partners will end up just fine. Is it reckless? Well, the connotation of “reckless” is fairly negative, and I don’t see why I should get to judge what two consenting adults decide is an acceptable risk vs reward for themselves (at least, so long as I’m not involved). However, when we pivot back to the domain of software/engineering: when software design choices are made via bandwagon fallacy, claiming that “it usually works out okay”, I do find that to be reckless. You may be fine with such a carefree approach, but it isn’t really fair to other engineers, users, and stake holders. It’s not that I believe JSON (or similarly pitfall-laden technology) should be strictly avoided, but rather that the risks and failure modes should be given serious consideration, rather than minimized or outright dismissed. In terms of the STI analogy, it’s perfectly reasonable for two individuals to be aware of the risks, and depending on their appetite for said risk, agree upon the inclusive/exclusive nature of their relationship, as well as whether they exchange test results; what would be ridiculous is to pretend that the risk of infection is zero. There would be zero value in the article minimizing the design flaws (and resulting footguns) in JSON. The insistence that the article should do so is about as bizarre as someone responding to an article on safer sex by insisting that the article be amended with a note that “… but raw-dogging strangers isn’t too risky anyway, so, like, YOLO”.

catlifeonmars 2y ago

Go can actually decode numbers into an “arbitrary precision” raw token (json.Number, which is a wrapper around []byte). You can turn this on at the decoder level or for a particular field. I often do this for more fine grained control of number decoding without needing to drop to the tokenizer.

AtlasBarfed 2y ago

Jason can't be streamed: wtf? If you have a data connection that can be broken, every data format is incomplete until the last ack There is absolutely nothing stopping you from parsing json as it comes in. An open array that hasn't reached the closing arrays character yet? How is that different than a stream of data elements? With the Jackson API, I was easily able to parse incoming text into Json tokens and extract data as it came in. I did this for certain corner cases where Json documents possibly were sent in that were greater than could be practically put into memory. Sure, you might want to have a rollback function in case there is a break in the data transfer, but how'd that be different with a different data format streamed? The guy is complaining a lot about number formats. If you care about it that much just put your numbers and strings, and make the parsing format part of the standard /api interface. This guy simply appears to be in love with binary formats and strong typing. If that floats his boat by all means do it. The fact that so much of the web relies on Json, and it pretty much seems to work. The guy hopefully indicated some corner cases on its use. But come on it's not fatally flawed.

dekhn 2y ago
I stream JSON as a series of closed JSON documents. Parsing incomplete JSON just seems like a minefield of errors.

buzzardbait 2y ago

The complaint is mostly about JSON parsers, not the JSON spec itself, which as other comments have said works quite well in practice. In .NET land the most popular JSON library is Newtonsoft which plays fast and loose with the JSON spec and makes a lot of default assumptions. A couple of years ago Microsoft rolled out their own first-party JSON library, which is a lot stricter and requires the developer to explicitly configure the parser instead of making assumptions. https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/migrate-from-newtonsoft?pivots=dotnet-9-0#table-of-differences https://learn.microsoft.com/en-us/dotnet/standard/serializat... But some developers still instinctively reach for Newtonsoft, even when working on brand new projects. In practice, this sort of ingrained habit is far greater of a problem than the JSON spec itself. The author says that Protobuf makes all of JSON's problems go away. I don't know enough about Protobuf to confirm or deny, but I know that it's not tested 10 trillion times a day like JSON is.

theFco 2y ago
The argument that it is tested 10 trillion times a day is a fallacy (because it can't be that only the most popular option should be the best forever). And furthermore, in this context "tested" is a bit of an overstatement, I would say "used" many gazillions of times a day.
- buzzardbait 2y ago
  Using is testing. The whole point of automated UI tests is to mimic usage. A random click on a website might not be a great test, but it is a test nonetheless. And if it fails enough times, someone somewhere will be held accountable.
- dogboat 2y ago
  It is used so much across the board for FE and BE and CI and config. Despite the heavy usage I it hasn't been canned and I have never heard anyone at work complain about JSON for any reason. You could say it is network effects but if it were crap you could replace it much more easily than say moving from Python to Java or whatever. Especially for internal microservice stuff and perhaps front end. JSON and the tooling is basically solid. It is a non-concern.

nijave 2y ago

s/JSON/CSV/

s6af7ygt 2y ago

I agree with all of the points, except the last. Protobuf is a nice transfer format but it's horrible once a human wants to inspect or copy/paste a payload.

nightowl_games 2y ago
Exactly. And this is an extremely important detail. It makes your development process go from using glass tubes where you see everything as it is sent/received to black boxes of binary data. JSON is self describing, you can mostly just call the API, read the JSON response and figure it out from there. Ignoring that detail is ignoring the elephant in the room.
- solidasparagus 2y ago
  The author was a tech lead on protobuf. I suspect that "JSON is worse than protobuf" is an obvious truth in their eyes.

p4cmanus3r 2y ago

PHP sucks too

ramon156 2y ago
One million reddit gold to you
- devoutsalsa 2y ago
  “I like Elixir.” “Elixir has been an absolute joy to work with…”

dainiusse 2y ago

s/JSON/YAML/ :)

exabrial 2y ago

I wish json docs had a schema you exchange for your objects, and evaluate compliance against the schema. XSDs one of the only redeeming qualities of XML.

ivell 2y ago
You can use JSONSchema, https://json-schema.org/ https://json-schema.org/
halfcat 2y ago
Does JSON Schema help here? https://json-schema.org/ https://json-schema.org/
dekhn 2y ago
I've worked with json schema and have mixed opinions- it was extremely cumbersome to do some things that should be simple, and very simple to do things that should be extremely cumbersome. I agree that XSD was one of the only redeeming qualities of XML- the schema schema itself was fairly well thought-through. Personally I prefer the IDL-style languages- that would include CORBA (just the schema and encoding), protobufs, and the ur-IDL that Sun RPC was based on, XDR. Protobufs have gone through some weird evolution, though- IIRC proto3 went tyhrough a full 180 on required/optional fields.

deleted 2y ago

[deleted]

dzaima 2y ago

Much as JSON is without a doubt far from perfect, I think most of the article is rather meh. Protobuf won't stop people from trying to put a license keys, card numbers, or other IDs in a 64-bit float or int. All it'll do is make someone be able to smugly point to a certain person and say "you're stupid, you did this incorrectly as per the spec". The extra assurance could even make things worse, e.g. using 64-bit ints for credit card numbers and having something break that interprets it as a signed int (or perhaps become unusable if 20-digit numbers happen), or have code that converts it to a float anyway, devolving back to 53-bit precision. The "fix" for both formats for normal people is still exactly the same - "please please please don't put things that aren't strictly counts of things or imprecision-tolerant real numbers in number types". Unpaired surrogates is just a question of which component fails. If any! An arbitrary user input containing an unpaired surrogate would likely be better off quietly transformed (or preserved) than become a DoS vector. Lack of NaNs, infinities, and compact ways to encode arbitrary bytes are the things I personally find the most unfortunate. The special constants are especially weird as null/false/true exist, but byte strings do have the problem that not all languages have a reasonable distinct thing for such to map to. A funky side-note is that JSON being so tolerant/imprecise (and as such most suppliers not pushing it to edge-cases) means that crappy parsers/producers aren't typically particularly problematic, though this perhaps isn't really a thing to judge data formats by.

scotty79 2y ago

I really love the minimap idea on this website. I think it's the first time ever that I see it.

jendjcneeb 2y ago

As many have said, the post is complaining about broken parsers and broken data structures, not a broken spec. An example for this is using a map instead of a key-value-list for object representation Or float instead of arbitrary precision decimals Are there more broken json libraries out there, then there are broken XML libraries? Maybe. But that's not a problem with the spec. Choosing JSON if your use case is well suited is fine nevertheless Just don't use obviously broken libtaries

dented42 2y ago
But the libraries aren’t broken, they follow the spec. The spec is vague enough that compliant implementations can be mutually incompatible, that’s the main criticism as far as I can see.

lakomen 2y ago

Or maybe they shouldn't

avmich 2y ago

> Protobuf, however, has none of these problems I like this optimism :) .

Glyptodon 2y ago

I think in many cases people are just using JSON because they are getting it for free, not because they actively picked it.

Sohcahtoa82 2y ago

This article reeks of being written by someone with maybe a year or two of experience that's read someone else's ill-informed rants about JSON and is rehashing it. IMO, the only valid complaint is that the JSON spec can't handle Inf or NaN, but I'd argue that if you're finding yourself dealing with Inf or NaN, then you likely have a bug elsewhere, and JSON is merely exposing it. If you're storing credit card numbers, license keys, barcode IDs, etc. as integers, you're already doing it wrong, and what serialization format you're using is completely irrelevant.

CRConrad 2y ago
Totally concur with your last paragraph. But, considering that the author is apparently one of the people behind the competing format mentioned, protobuf, your first paragraph feels way off.
nijave 2y ago
Yeah, I like the "if you can't do math on it, it probably shouldn't be a numeric data type" rule of thumb i.e. applying an average to a list of credit card numbers doesn't make any sense nor does adding commas

donnaalbert 2y ago

[dead]