14 ms·

JSON is not a YAML subset

5 points by jmillikin 4y ago

hnlmorg 4y ago
Unless I’m misunderstanding your argument (which is possible, I’ve not had my morning coffee yet) what you’re describing is really more about JSONs strictness as a feature rather than YAML not being a superset of JSON. You’re general point is correct, you shouldn’t use a YAML parser to read JSON; but you fail to raise the most important point why. That being the strictness of JSON means if someone submits incorrect JSON then it should fail rather than silently correct it (and in ways that might even corrupt the data). Of course, there might be some instances where someone opts to use a YAML parser as a lazy way of allowing more lenient JSON syntax but that is a risk the application authors accept and one which should be well documented to their users. So it feels like you’re making the right conclusion while still arguing the wrong points. As an aside, it might be worth testing your site on mobile. It doesn’t really render properly for me on my iPhone (Safari). Lots of text out of bounds on the sides and I couldn’t scroll to read them. Plus there were (I think) images of Stack Overflow that looked like part of the article paragraphs and was hard to tell what was reference content and what was your content.
- jmillikin 4y ago
  Thanks for the comment (/ reminder) about mobile CSS -- it's on my todo queue, but there's been a lot of stuff popped on top recently. If the iPhone has a "desktop mode" feature, that might make it easier to read for now. > That being the strictness of JSON means if someone submits incorrect > JSON then it should fail rather than silently correct it (and in ways > that might even corrupt the data). The JSON document {"a": 1e2} is valid JSON. If you put it into a regular JSON parser (for example browser devtools), the correct result will come out: >>> JSON.parse('{"a": 1e2}') {a: 100} The issue is that it's also a valid YAML document, and (critically) the correct YAML parse is different than the correct JSON parse.
  - hnlmorg 4y ago
    Indeed but that doesn’t mean that JSON isn’t a subset of YAML. That means that the strictness of JSON removes ambiguity of what is a string and what is not while the lazy strings in YAML allow for ambiguity (and this problem comes up all the time with proper YAML documents too). Which is why I said you had reached the right conclusion despite conflating syntax compatibility with syntax strictness. It’s also worth noting that JSON doesn’t actually have a standard for representing complex numbers. In theory JSON should support scientific notation but given JSONs numeric parser is basically based around what JavaScript supported, it means as JSON was ported to other languages discrepancies have crept in with regards to how numbers should be parsed. Some guides even (incorrectly) state that scientific notation are not supported. And frankly it’s worst advice given how JSON parsers can vary so significantly. To cloud the issue further, plenty of other “standards” have emerged which are JSON supersets which do describe how numeric data should be handed. Some even support date fields too. Often these parsers are just labelled “JSON” and they can corrupt your data too but developers might think they’re using a proper JSON parser. So the issues isn’t that YAML doesn’t support JSON. It’s that the JSON spec was basically written on the back of a napkin and doesn’t support any form of versioning (so we can move away from this madness) whilst YAML also supports lazy strings and thus allow data to be ambiguous.
    - jmillikin 4y ago
      One of us is extraordinarily confused and I hope it isn't me. > Indeed but that doesn’t mean that JSON isn’t a subset of YAML. That > means that the strictness of JSON removes ambiguity of what is a > string and what is not while the lazy strings in YAML allow for ambiguity I ... don't think that's true. This isn't an issue of "strictness", and the document is unambiguous in both JSON and YAML. There's no question about correctness -- both parses appear to be correct for their respective languages. It's just that JSON and YAML disagree about whether the token `1e2` is an integer or a string. If LANG-X says that `1e2` is an integer and LANG-Y says it's a string, then axiomatically LANG-X cannot be considered a subset of LANG-Y. > It’s also worth noting that JSON doesn’t actually have a standard for > representing complex numbers. I don't see how that matters? `1e2` isn't a complex number, it's an integer. > In theory JSON should support scientific notation but [...] There's no "in theory" and no "but" here. Exponents are a part of the JSON spec, have been since the beginning, and are thoroughly documented in each iteration of the JSON RFC. A parser that can't parse {"a": 1e2} is not a JSON parser.
soraminazuki 4y ago
I find this claim misleading. First, the article starts off by picking examples from YAML 1.1. But the YAML 1.2 spec officially made YAML a superset of JSON. It was published in 2009, more than a decade ago. The article later does mention YAML 1.2, but it claims that it isn't compatible with JSON either because YAML 1.2 documents are required to have a %YAML directive stating its version. Except, that's not what the spec says. The actual spec states that documents without a %YAML directive should be treated as YAML 1.2 [1]. [1]: https://yaml.org/spec/1.2.1/#id2781553 https://yaml.org/spec/1.2.1/#id2781553
- jmillikin 4y ago
  That part of the YAML 1.2 spec is in conflict with reality, though. The base of YAML 1.1 documents is large enough that a backwards-incompatible change to default behavior is for practical purposes impossible. YAML 1.1 was released in 2005, and 1.2 in 2009 -- only four years later. But here we are, in 2022, and YAML 1.1 is still the default (in many cases, only) version supported. That's why the "Norway problem" persists -- it's not possible for the parser to know whether the author of an un-versioned YAML document containing "a: no" intended it to parse the same as {"a": false} or {"a": "no"}. Python (PyYAML) doesn't support 1.2 yet: https://github.com/yaml/pyyaml/issues/116 https://github.com/yaml/pyyaml/issues/116 Ruby (Psych) ditto -- I can't even find a tracking issue to enable it. Go (go-yaml) is a mixture of YAML 1.1 and 1.2, depending on the author's preferences. Also, as a rough guideline, you can't have a backwards-incompatible revision of a versioned spec declare that it's the new default version, because that breaks all existing users.

5 points by jmillikin 4y ago

Unless I’m misunderstanding your argument (which is possible, I’ve not had my morning coffee yet) what you’re describing is really more about JSONs strictness as a feature rather than YAML not being a superset of JSON. You’re general point is correct, you shouldn’t use a YAML parser to read JSON; but you fail to raise the most important point why. That being the strictness of JSON means if someone submits incorrect JSON then it should fail rather than silently correct it (and in ways that might even corrupt the data). Of course, there might be some instances where someone opts to use a YAML parser as a lazy way of allowing more lenient JSON syntax but that is a risk the application authors accept and one which should be well documented to their users. So it feels like you’re making the right conclusion while still arguing the wrong points. As an aside, it might be worth testing your site on mobile. It doesn’t really render properly for me on my iPhone (Safari). Lots of text out of bounds on the sides and I couldn’t scroll to read them. Plus there were (I think) images of Stack Overflow that looked like part of the article paragraphs and was hard to tell what was reference content and what was your content.

jmillikin 4y ago
Thanks for the comment (/ reminder) about mobile CSS -- it's on my todo queue, but there's been a lot of stuff popped on top recently. If the iPhone has a "desktop mode" feature, that might make it easier to read for now. > That being the strictness of JSON means if someone submits incorrect > JSON then it should fail rather than silently correct it (and in ways > that might even corrupt the data). The JSON document {"a": 1e2} is valid JSON. If you put it into a regular JSON parser (for example browser devtools), the correct result will come out: >>> JSON.parse('{"a": 1e2}') {a: 100} The issue is that it's also a valid YAML document, and (critically) the correct YAML parse is different than the correct JSON parse.
- hnlmorg 4y ago
  Indeed but that doesn’t mean that JSON isn’t a subset of YAML. That means that the strictness of JSON removes ambiguity of what is a string and what is not while the lazy strings in YAML allow for ambiguity (and this problem comes up all the time with proper YAML documents too). Which is why I said you had reached the right conclusion despite conflating syntax compatibility with syntax strictness. It’s also worth noting that JSON doesn’t actually have a standard for representing complex numbers. In theory JSON should support scientific notation but given JSONs numeric parser is basically based around what JavaScript supported, it means as JSON was ported to other languages discrepancies have crept in with regards to how numbers should be parsed. Some guides even (incorrectly) state that scientific notation are not supported. And frankly it’s worst advice given how JSON parsers can vary so significantly. To cloud the issue further, plenty of other “standards” have emerged which are JSON supersets which do describe how numeric data should be handed. Some even support date fields too. Often these parsers are just labelled “JSON” and they can corrupt your data too but developers might think they’re using a proper JSON parser. So the issues isn’t that YAML doesn’t support JSON. It’s that the JSON spec was basically written on the back of a napkin and doesn’t support any form of versioning (so we can move away from this madness) whilst YAML also supports lazy strings and thus allow data to be ambiguous.
  - jmillikin 4y ago
    One of us is extraordinarily confused and I hope it isn't me. > Indeed but that doesn’t mean that JSON isn’t a subset of YAML. That > means that the strictness of JSON removes ambiguity of what is a > string and what is not while the lazy strings in YAML allow for ambiguity I ... don't think that's true. This isn't an issue of "strictness", and the document is unambiguous in both JSON and YAML. There's no question about correctness -- both parses appear to be correct for their respective languages. It's just that JSON and YAML disagree about whether the token `1e2` is an integer or a string. If LANG-X says that `1e2` is an integer and LANG-Y says it's a string, then axiomatically LANG-X cannot be considered a subset of LANG-Y. > It’s also worth noting that JSON doesn’t actually have a standard for > representing complex numbers. I don't see how that matters? `1e2` isn't a complex number, it's an integer. > In theory JSON should support scientific notation but [...] There's no "in theory" and no "but" here. Exponents are a part of the JSON spec, have been since the beginning, and are thoroughly documented in each iteration of the JSON RFC. A parser that can't parse {"a": 1e2} is not a JSON parser.

soraminazuki 4y ago

I find this claim misleading. First, the article starts off by picking examples from YAML 1.1. But the YAML 1.2 spec officially made YAML a superset of JSON. It was published in 2009, more than a decade ago. The article later does mention YAML 1.2, but it claims that it isn't compatible with JSON either because YAML 1.2 documents are required to have a %YAML directive stating its version. Except, that's not what the spec says. The actual spec states that documents without a %YAML directive should be treated as YAML 1.2 [1]. [1]: https://yaml.org/spec/1.2.1/#id2781553 https://yaml.org/spec/1.2.1/#id2781553

jmillikin 4y ago
That part of the YAML 1.2 spec is in conflict with reality, though. The base of YAML 1.1 documents is large enough that a backwards-incompatible change to default behavior is for practical purposes impossible. YAML 1.1 was released in 2005, and 1.2 in 2009 -- only four years later. But here we are, in 2022, and YAML 1.1 is still the default (in many cases, only) version supported. That's why the "Norway problem" persists -- it's not possible for the parser to know whether the author of an un-versioned YAML document containing "a: no" intended it to parse the same as {"a": false} or {"a": "no"}. Python (PyYAML) doesn't support 1.2 yet: https://github.com/yaml/pyyaml/issues/116 https://github.com/yaml/pyyaml/issues/116 Ruby (Psych) ditto -- I can't even find a tracking issue to enable it. Go (go-yaml) is a mixture of YAML 1.1 and 1.2, depending on the author's preferences. Also, as a rough guideline, you can't have a backwards-incompatible revision of a versioned spec declare that it's the new default version, because that breaks all existing users.