3 ms·

But that attempt at strictness harms implementation value. Even your own requirement set that you've claimed will work on everything is... bad - Sure I can par

But that attempt at strictness harms implementation value. Even your own requirement set that you've claimed will work on everything is... bad - Sure I can parse every number as a double, if I'm willing to spend at least 64bits on every number in the payload. I just finished building a PH Autodoser for a hydroponics system I run - it sends JSON payloads with sensor data and receives JSON commands to do things like dispense PHDown/PHUp solution, toggle on water cooling. I have very little spare working memory on the device doing the actual monitoring. having to hold 64 bits per number would push me into having to buy a more expensive microcontroller. Instead - I have an informal contract that almost all fields are plain unsigned bytes (0 to 255) which works fine for my use-case, requiring just 1/8th the space. And to go the other direction - I have a desktop running some financial software, I pass around json payloads there, but a double is NOT ENOUGH. I want a BigInt field for numbers there instead, because rounding errors that would be a-ok for a ph sensor are absolutely not ok for calculating financial data. ---- Basically - I want the flexibility to chose the correct interpretation for my data. And this: "everything non-compliant must fail the entire parse." Is just fucking insanity. It's the literal antithesis of the robustness principle: "be conservative in what you send, be liberal in what you accept"

q3k 5y ago
> Instead - I have an informal contract that almost all fields are plain unsigned bytes (0 to 255) which works fine for my use-case, requiring just 1/8th the space. Right, but that informal contract is at the detriment of everyone else having to also specify the expected limits of numbers they work with. It makes your particular usecase easier, but it doesn't make the standard better in the grand scheme of things. > And to go the other direction - I have a desktop running some financial software, I pass around json payloads there, but a double is NOT ENOUGH. I want a BigInt field for numbers there instead, because rounding errors that would be a-ok for a ph sensor are absolutely not ok for calculating financial data. And JSON doesn't guarantee you that, you have to shop around for languages and implementations that permit this. If you then have to make work with an implementation that always deserializes to doubles (which is a compliant behaviour) or bytes (which is a compliant behaviour), you're screwed. Again, this might work for the simple case of you controlling both ends of the serialization, but it's terrible for trying to work with an end that you don't control (ie. when actually using JSON as an interchange format). > And this: "everything non-compliant must fail the entire parse." Is just fucking insanity. It's the literal antithesis of the robustness principle: "be conservative in what you send, be liberal in what you accept" The Robustness Principle followed blindly is known to be harmful when dealing with long-term standards, evolving implementations and the human element of software engineering [1]. My opinion is that an interchange format's job is to transfer some data reliably and atomically: the deserialized data should be either be 100% correct or the deserialization should be rejected. Anything else can and will lead to bugs, and bugs that are then difficult to solve (as at that point it's difficult to agree whether the serialization was not conservative enough, or the deserialization not liberal enough). [1] - https://datatracker.ietf.org/doc/html/draft-iab-protocol-maintenance https://datatracker.ietf.org/doc/html/draft-iab-protocol-mai...
- horsawlarway 5y ago
  Ok - so now you have a very strict protocol, that never gains traction because the strictness you value hampers utility. And yes - I'm aware of the "Bug for bug compatibility" problems that draft tries to highlight, but it's fairly clear that utility is paramount: > As [SUCCESS] demonstrates, success or failure of a protocol depends far more on factors like usefulness than on on technical excellence. Timely publication of protocol specifications, even with the potential for flaws, likely contributed significantly to the eventual success of the Internet.
  - TeMPOraL 5y ago
    > Ok - so now you have a very strict protocol, that never gains traction because the strictness you value hampers utility. Well, things shouldn't be like that. This is the collective insanity of our industry - standardizing on tools that are easiest to work with when you don't understand them and are under time pressure to cut corners. It's the opposite of good engineering.
    - horsawlarway 5y ago
      > Well, things shouldn't be like that. Ain't that the story of this planet. That said - I'm not really sure you can call it bad engineering. It turns out that people will use whatever is available to suit their needs (and god help you if you try to predetermine all the strange edge-cases they'll use it for... https://xkcd.com/1172/ https://xkcd.com/1172/). For every one developer that's had to debug a json parser compatibility issue, there are probably a million other developers happily solving real world problems with JSON with no issues at all.
karmakaze 5y ago
What you're describing here is a schema-specific parser. Even if the parser succeeded, you would reject the input as the values are out of range. Making a custom parser for this is fine, but call it what it is a parser for a subset of JSON--it would fail for a value of 0.1 or -1.
- horsawlarway 5y ago
  Sure. The problem is that many things that a very strict spec might require have real resource requirements. There's a reason the URL length in http is undefined, it's because the machine accepting the request doesn't have infinite memory. Even the latest spec is a simple recommendation to accept a request line of at least 8k octets. You can say "We must support nesting depth of N" in json, but the reality of the situation is that parsers can and will just ignore you. Are they non-compliant? Sure. Are they useful? Sure. Will people still use them? Fuck yes they will. Because utility trumps strictness in most cases.
josefx 5y ago
> , be liberal in what you accept While some people are fine with garbage in / garbage out some of us prefer to filter out the garbage before we have to spend days debugging why something doesn't work as it should.
- horsawlarway 5y ago
  One man's garbage is another man's treasure. If you don't like how the current parsers work, feel free to go write your own, because frankly "doesn't work as it should" is presumptuous bullshit. They work exactly as their current authors like, if that happens to not be correct for you... either put up or shut up.
  - josefx 5y ago
    1) I wasn't trying to imply that the parsers themselves aren't working but that you might have to debug programs using these parsers more often as its not apparent that the issue lies in the input instead of the program having to handle it. 2) Pretentious would be if I liked the color blue and then claimed that "walls should be blue" is a robustness principle.

q3k 5y ago

> Instead - I have an informal contract that almost all fields are plain unsigned bytes (0 to 255) which works fine for my use-case, requiring just 1/8th the space. Right, but that informal contract is at the detriment of everyone else having to also specify the expected limits of numbers they work with. It makes your particular usecase easier, but it doesn't make the standard better in the grand scheme of things. > And to go the other direction - I have a desktop running some financial software, I pass around json payloads there, but a double is NOT ENOUGH. I want a BigInt field for numbers there instead, because rounding errors that would be a-ok for a ph sensor are absolutely not ok for calculating financial data. And JSON doesn't guarantee you that, you have to shop around for languages and implementations that permit this. If you then have to make work with an implementation that always deserializes to doubles (which is a compliant behaviour) or bytes (which is a compliant behaviour), you're screwed. Again, this might work for the simple case of you controlling both ends of the serialization, but it's terrible for trying to work with an end that you don't control (ie. when actually using JSON as an interchange format). > And this: "everything non-compliant must fail the entire parse." Is just fucking insanity. It's the literal antithesis of the robustness principle: "be conservative in what you send, be liberal in what you accept" The Robustness Principle followed blindly is known to be harmful when dealing with long-term standards, evolving implementations and the human element of software engineering [1]. My opinion is that an interchange format's job is to transfer some data reliably and atomically: the deserialized data should be either be 100% correct or the deserialization should be rejected. Anything else can and will lead to bugs, and bugs that are then difficult to solve (as at that point it's difficult to agree whether the serialization was not conservative enough, or the deserialization not liberal enough). [1] - https://datatracker.ietf.org/doc/html/draft-iab-protocol-maintenance https://datatracker.ietf.org/doc/html/draft-iab-protocol-mai...

horsawlarway 5y ago
Ok - so now you have a very strict protocol, that never gains traction because the strictness you value hampers utility. And yes - I'm aware of the "Bug for bug compatibility" problems that draft tries to highlight, but it's fairly clear that utility is paramount: > As [SUCCESS] demonstrates, success or failure of a protocol depends far more on factors like usefulness than on on technical excellence. Timely publication of protocol specifications, even with the potential for flaws, likely contributed significantly to the eventual success of the Internet.
- TeMPOraL 5y ago
  > Ok - so now you have a very strict protocol, that never gains traction because the strictness you value hampers utility. Well, things shouldn't be like that. This is the collective insanity of our industry - standardizing on tools that are easiest to work with when you don't understand them and are under time pressure to cut corners. It's the opposite of good engineering.
  - horsawlarway 5y ago
    > Well, things shouldn't be like that. Ain't that the story of this planet. That said - I'm not really sure you can call it bad engineering. It turns out that people will use whatever is available to suit their needs (and god help you if you try to predetermine all the strange edge-cases they'll use it for... https://xkcd.com/1172/ https://xkcd.com/1172/). For every one developer that's had to debug a json parser compatibility issue, there are probably a million other developers happily solving real world problems with JSON with no issues at all.

karmakaze 5y ago

What you're describing here is a schema-specific parser. Even if the parser succeeded, you would reject the input as the values are out of range. Making a custom parser for this is fine, but call it what it is a parser for a subset of JSON--it would fail for a value of 0.1 or -1.

horsawlarway 5y ago
Sure. The problem is that many things that a very strict spec might require have real resource requirements. There's a reason the URL length in http is undefined, it's because the machine accepting the request doesn't have infinite memory. Even the latest spec is a simple recommendation to accept a request line of at least 8k octets. You can say "We must support nesting depth of N" in json, but the reality of the situation is that parsers can and will just ignore you. Are they non-compliant? Sure. Are they useful? Sure. Will people still use them? Fuck yes they will. Because utility trumps strictness in most cases.

josefx 5y ago

> , be liberal in what you accept While some people are fine with garbage in / garbage out some of us prefer to filter out the garbage before we have to spend days debugging why something doesn't work as it should.

horsawlarway 5y ago
One man's garbage is another man's treasure. If you don't like how the current parsers work, feel free to go write your own, because frankly "doesn't work as it should" is presumptuous bullshit. They work exactly as their current authors like, if that happens to not be correct for you... either put up or shut up.
- josefx 5y ago
  1) I wasn't trying to imply that the parsers themselves aren't working but that you might have to debug programs using these parsers more often as its not apparent that the issue lies in the input instead of the program having to handle it. 2) Pretentious would be if I liked the color blue and then claimed that "walls should be blue" is a robustness principle.