4 ms·

Most (all?) the complaints here appear to be that specific libraries fail to implement the JSON spec in the way that the author has interpreted it. Some librari

Most (all?) the complaints here appear to be that specific libraries fail to implement the JSON spec in the way that the author has interpreted it. Some libraries try to 'help' by parsing things that they shouldn't, and some fail to parse things they probably should. This is why we end up with so many JSON parsing libraries I guess, but it's not really a problem with the format itself, beyond the fact that clearer specs might disambiguate things and lead to less deviation.

q3k 5y ago
> but it's not really a problem with the format itself, beyond the fact that clearer specs might disambiguate things and lead to less deviation. It is a problem, because it's not a spec that can be implemented reliably. Different parsers behave differently on various corner cases not only because of implementation blunders, but also because the standard(s) just let them do whatever. This spectacularly breaks systems that use more than one parser implementation, each slightly implementing the standard slightly differently. One part of some processing/parsing pipeline will let some payload through, while another one will reject it, or even parse it differently.
- horsawlarway 5y ago
  I disagree (at least mostly). This is a case where the spec is intentionally loose to allow compatibility with a much larger number of machines and use cases. You'll notice most of the cases where the behavior is implementation defined have resource requirements. example: how deep you want to allow nesting depends a LOT on the capabilities of the machine running the code. A sane value for a modern browser is going to be unworkable on an arduino/ESP32/embedded other. Also... if these ambiguities bother you, you probably haven't read the full http spec either. It's riddled with cases where behavior is implementation defined, for exactly the same reasons (resources are required, and you can't assume everyone has the same amount available). Want to take a guess at the maximum length for a url?
  - q3k 5y ago
    > This is a case where the spec is intentionally loose to allow compatibility with a much larger number of machines and use cases. There's plenty that could've been specified with little detriment to small systems: strings are UTF-8 with a well-defined escape sequence set, numbers are always IEEE-754 doubles, messages cannot be nested by more than 128 levels (or some other arbitrary number in this range), repeated fields are not permitted, everything non-compliant must fail the entire parse. Then the only thing left to handle is a maximum serialized size (which can be explicitly implementation or user defined). Set the maximum string length to maximum payload length defined earlier and you're golden. That is then your only difference between implementations. This will work on your Ryzen server and on your ESP8266 or ESP32, and can even be handled on your washing machine microcontroller^W^W^WArduino (with a slowdown for dealing with floating point numbers, but you already have to deal with that). Finally, the spec isn't loose because of some design choice to allow interoperability with more machines: it's loose because it was historically loose (see: JSON business card 'specification' chutzpah, which itself is based on a mess of a programming language that is/was JS), and before it could be formalized to something sensible it got implemented haphazardly by different languages. That doomed the format to forever be underdefined, as anything more strict would render existing implementations non-compliant.
    - horsawlarway 5y ago
      But that attempt at strictness harms implementation value. Even your own requirement set that you've claimed will work on everything is... bad - Sure I can parse every number as a double, if I'm willing to spend at least 64bits on every number in the payload. I just finished building a PH Autodoser for a hydroponics system I run - it sends JSON payloads with sensor data and receives JSON commands to do things like dispense PHDown/PHUp solution, toggle on water cooling. I have very little spare working memory on the device doing the actual monitoring. having to hold 64 bits per number would push me into having to buy a more expensive microcontroller. Instead - I have an informal contract that almost all fields are plain unsigned bytes (0 to 255) which works fine for my use-case, requiring just 1/8th the space. And to go the other direction - I have a desktop running some financial software, I pass around json payloads there, but a double is NOT ENOUGH. I want a BigInt field for numbers there instead, because rounding errors that would be a-ok for a ph sensor are absolutely not ok for calculating financial data. ---- Basically - I want the flexibility to chose the correct interpretation for my data. And this: "everything non-compliant must fail the entire parse." Is just fucking insanity. It's the literal antithesis of the robustness principle: "be conservative in what you send, be liberal in what you accept"
deleted 5y ago
[deleted]

q3k 5y ago

> but it's not really a problem with the format itself, beyond the fact that clearer specs might disambiguate things and lead to less deviation. It is a problem, because it's not a spec that can be implemented reliably. Different parsers behave differently on various corner cases not only because of implementation blunders, but also because the standard(s) just let them do whatever. This spectacularly breaks systems that use more than one parser implementation, each slightly implementing the standard slightly differently. One part of some processing/parsing pipeline will let some payload through, while another one will reject it, or even parse it differently.

horsawlarway 5y ago
I disagree (at least mostly). This is a case where the spec is intentionally loose to allow compatibility with a much larger number of machines and use cases. You'll notice most of the cases where the behavior is implementation defined have resource requirements. example: how deep you want to allow nesting depends a LOT on the capabilities of the machine running the code. A sane value for a modern browser is going to be unworkable on an arduino/ESP32/embedded other. Also... if these ambiguities bother you, you probably haven't read the full http spec either. It's riddled with cases where behavior is implementation defined, for exactly the same reasons (resources are required, and you can't assume everyone has the same amount available). Want to take a guess at the maximum length for a url?
- q3k 5y ago
  > This is a case where the spec is intentionally loose to allow compatibility with a much larger number of machines and use cases. There's plenty that could've been specified with little detriment to small systems: strings are UTF-8 with a well-defined escape sequence set, numbers are always IEEE-754 doubles, messages cannot be nested by more than 128 levels (or some other arbitrary number in this range), repeated fields are not permitted, everything non-compliant must fail the entire parse. Then the only thing left to handle is a maximum serialized size (which can be explicitly implementation or user defined). Set the maximum string length to maximum payload length defined earlier and you're golden. That is then your only difference between implementations. This will work on your Ryzen server and on your ESP8266 or ESP32, and can even be handled on your washing machine microcontroller^W^W^WArduino (with a slowdown for dealing with floating point numbers, but you already have to deal with that). Finally, the spec isn't loose because of some design choice to allow interoperability with more machines: it's loose because it was historically loose (see: JSON business card 'specification' chutzpah, which itself is based on a mess of a programming language that is/was JS), and before it could be formalized to something sensible it got implemented haphazardly by different languages. That doomed the format to forever be underdefined, as anything more strict would render existing implementations non-compliant.
  - horsawlarway 5y ago
    But that attempt at strictness harms implementation value. Even your own requirement set that you've claimed will work on everything is... bad - Sure I can parse every number as a double, if I'm willing to spend at least 64bits on every number in the payload. I just finished building a PH Autodoser for a hydroponics system I run - it sends JSON payloads with sensor data and receives JSON commands to do things like dispense PHDown/PHUp solution, toggle on water cooling. I have very little spare working memory on the device doing the actual monitoring. having to hold 64 bits per number would push me into having to buy a more expensive microcontroller. Instead - I have an informal contract that almost all fields are plain unsigned bytes (0 to 255) which works fine for my use-case, requiring just 1/8th the space. And to go the other direction - I have a desktop running some financial software, I pass around json payloads there, but a double is NOT ENOUGH. I want a BigInt field for numbers there instead, because rounding errors that would be a-ok for a ph sensor are absolutely not ok for calculating financial data. ---- Basically - I want the flexibility to chose the correct interpretation for my data. And this: "everything non-compliant must fail the entire parse." Is just fucking insanity. It's the literal antithesis of the robustness principle: "be conservative in what you send, be liberal in what you accept"

deleted 5y ago

[deleted]