3 ms·

{"this_article": "// would be better", "this_article": "// if it werent", "this_article": "// immediately", "this_article": "// right off the bat",

{"this_article": "// would be better", "this_article": "// if it werent", "this_article": "// immediately", "this_article": "// right off the bat", "this_article": "// wrong", "this_article": "meh"} //=> {"this_article": "meh"} Hacker spirit represent. Where there's a will, there's a way. (Death to all oppressors!) I actually agree, JSON is kind of a pain in the ass to work with. It's shocking to think of a pre-'jq' world, given how saturated we all are with JSON, but we lived like that for a long time. (We also evidently didn't even have Firebug until 2006?[1] Holy shit!) But is it a problem? I dunno. Is it worth doing something else? Ooof, scary proposition. I think one of the missing questions we don't ask, don't know is: what tools help us fix our json fast. What tools can see, oh there's a comma missing between these two lines, and just fix it for us. [1] https://en.wikipedia.org/wiki/Firebug_(software) https://en.wikipedia.org/wiki/Firebug_(software)

tikhonj 4y ago
As far as I can tell, you're relying on non-standard behavior. This is what [the standard][1] has to say about keys being unique: > * The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.* This means that different tools can—and, if experience is any guide, will—handle your example inconsistently. This can be especially troubling when JSON processing tools are used internally by some other system like a database, queue or server; most of the time these tools parse and serialize JSON without changing its meaning, but they can change or lose information when people rely on patterns like the one that you used. I would honestly be surprised if there isn't at least one realistically used tool that interprets your example differently, like producing {"this_article": "// would be better"} instead. [1]: https://www.ecma-international.org/publications-and-standards/standards/ecma-404/ https://www.ecma-international.org/publications-and-standard...
- rektide 4y ago
  I'd love to see the counter-evidence! Works on every browser, every server-side JS engine of note. Edit: I believe a number of these non-determinism did get formally resolved in EcmaScript itself, which is yes, not the same, but is an active, alive, & notable specification with some bearing.
  - vsnf 4y ago
    Go will serialize keys to maps in random order, meaning that unless you take special and extraordinary precautions while serializing your json structs, you will get keys that will not be where you expect them to be.
    - rektide 4y ago
      This does not appear to be true[1][2]. [1] https://github.com/golang/go/issues/24415 https://github.com/golang/go/issues/24415 [2] https://go.dev/play/p/nXLs0hd8Uj3 https://go.dev/play/p/nXLs0hd8Uj3
  - tikhonj 4y ago
    First thing I tried was the Haskell Aeson library which keeps the first value of the key: λ> decode f :: Maybe Value Just (Object (fromList [("this_article",String "// would be better")])) There's a more flexible jsonWith function that lets you control how duplicates (and, for that matter, field order) is handled, but it's not the default. The handful of other tools I spot-checked all keep the last pair, so it seems like almost but not quite standard behavior. I was curious about other programs and, searching around, found a discussion[1] about standardizing the behavior to "either reject objects with duplicate keys or keep the last value"; however, from a quick scan, that wasn't added to any of the standards documents I found. Apart from the standard I linked earlier there are a couple of JSON RFCs[2][3], but they basically say "different systems do different things and if you depend on the behavior, your JSON won't be universally interoperable" > When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates. > JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences. If anything, the fact that it is almost but not quite standard behavior probably makes it worse—you can rely on it for quite a while until it unexpectedly breaks and the figuring out why it broke will be a real headache. [1]: https://esdiscuss.org/topic/json-duplicate-keys https://esdiscuss.org/topic/json-duplicate-keys [2]: https://datatracker.ietf.org/doc/html/rfc7159 https://datatracker.ietf.org/doc/html/rfc7159 [3]: https://datatracker.ietf.org/doc/html/rfc8259 https://datatracker.ietf.org/doc/html/rfc8259
valbaca 4y ago
You just highlighted one of the issues with JSON. You think you can write comments like that (hell, I've done it too) but it's both undefined behavior and at worst breaks when your config comes across something that expects a strict adherence to some schema some poor intern or sr. dev was saddled with writing.
phendrenad2 4y ago
Sure, just throw away 50 years of syntax highlighting technology and code readability. <insert this is fine meme>

tikhonj 4y ago

As far as I can tell, you're relying on non-standard behavior. This is what [the standard][1] has to say about keys being unique: > * The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.* This means that different tools can—and, if experience is any guide, will—handle your example inconsistently. This can be especially troubling when JSON processing tools are used internally by some other system like a database, queue or server; most of the time these tools parse and serialize JSON without changing its meaning, but they can change or lose information when people rely on patterns like the one that you used. I would honestly be surprised if there isn't at least one realistically used tool that interprets your example differently, like producing {"this_article": "// would be better"} instead. [1]: https://www.ecma-international.org/publications-and-standards/standards/ecma-404/ https://www.ecma-international.org/publications-and-standard...

rektide 4y ago
I'd love to see the counter-evidence! Works on every browser, every server-side JS engine of note. Edit: I believe a number of these non-determinism did get formally resolved in EcmaScript itself, which is yes, not the same, but is an active, alive, & notable specification with some bearing.
- vsnf 4y ago
  Go will serialize keys to maps in random order, meaning that unless you take special and extraordinary precautions while serializing your json structs, you will get keys that will not be where you expect them to be.
  - rektide 4y ago
    This does not appear to be true[1][2]. [1] https://github.com/golang/go/issues/24415 https://github.com/golang/go/issues/24415 [2] https://go.dev/play/p/nXLs0hd8Uj3 https://go.dev/play/p/nXLs0hd8Uj3
- tikhonj 4y ago
  First thing I tried was the Haskell Aeson library which keeps the first value of the key: λ> decode f :: Maybe Value Just (Object (fromList [("this_article",String "// would be better")])) There's a more flexible jsonWith function that lets you control how duplicates (and, for that matter, field order) is handled, but it's not the default. The handful of other tools I spot-checked all keep the last pair, so it seems like almost but not quite standard behavior. I was curious about other programs and, searching around, found a discussion[1] about standardizing the behavior to "either reject objects with duplicate keys or keep the last value"; however, from a quick scan, that wasn't added to any of the standards documents I found. Apart from the standard I linked earlier there are a couple of JSON RFCs[2][3], but they basically say "different systems do different things and if you depend on the behavior, your JSON won't be universally interoperable" > When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates. > JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences. If anything, the fact that it is almost but not quite standard behavior probably makes it worse—you can rely on it for quite a while until it unexpectedly breaks and the figuring out why it broke will be a real headache. [1]: https://esdiscuss.org/topic/json-duplicate-keys https://esdiscuss.org/topic/json-duplicate-keys [2]: https://datatracker.ietf.org/doc/html/rfc7159 https://datatracker.ietf.org/doc/html/rfc7159 [3]: https://datatracker.ietf.org/doc/html/rfc8259 https://datatracker.ietf.org/doc/html/rfc8259

valbaca 4y ago

You just highlighted one of the issues with JSON. You think you can write comments like that (hell, I've done it too) but it's both undefined behavior and at worst breaks when your config comes across something that expects a strict adherence to some schema some poor intern or sr. dev was saddled with writing.

phendrenad2 4y ago

Sure, just throw away 50 years of syntax highlighting technology and code readability. <insert this is fine meme>