6 ms·

> JSON can be made prettier by sorting the JSON object members by element keys. This seems to be a bad idea. The JSON language spec has ORDERED object members.

by Garlef 6y ago

> JSON can be made prettier by sorting the JSON object members by element keys. This seems to be a bad idea. The JSON language spec has ORDERED object members. But the order is arbitrary (precisely the one given in the JSON string) and does not have to be the lexicographic. Sorting the object members by default would introduce problems whenever the order matters to the consumer of the JSON.

Garlef 6y ago
I checked the spec again: It's unclear: At one point it says "An object is an unordered set of name/value pairs." while in the actual grammar it is ordered: object '{' ws '}' '{' members '}' members member member ',' members
- gmfawcett 6y ago
  I recommend reading the ECMA-404 spec instead. It's less ambiguous, and basically says that you can treat the pairs as ordered if you want, as the syntax itself doesn't imbue the order with any meaning: https://www.ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf https://www.ecma-international.org/wp-content/uploads/ECMA-4...
- dragonwriter 6y ago
  Grammar is inherently ordered, semantics is different from syntax.
peterohler 6y ago
I think you will find the JSON object members are not ordered while JSON array members are ordered. Since the JSON object members are not ordered, changing the order for display purposes does not change the data in any material way.
Latty 6y ago
The spec[1] says: > An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array. "whenever the order matters to the consumer of the JSON" should be never. More pragmatically, regardless of what the spec says, a ton of JSON tooling assumes the order doesn't matter and relying on it would be a big mistake. [1]: https://tools.ietf.org/html/rfc7159#section-1 https://tools.ietf.org/html/rfc7159#section-1
- gmfawcett 6y ago
  The other spec (ECMA-404) disagrees with RFC 7159 on this point. That's the great thing about specs -- if you don't like what one says, there's always another to support your position. :)
  - Latty 6y ago
    Well, falling back to the pragmatist position: a lot of JSON tooling assumes the order isn't relevant semantically. Not taking that position will cause you a lot of headaches.
    - gmfawcett 6y ago
      Agreed, and if you want to write tools that are conformant with both specs, you really don't have a choice. The RFC pretty much spells out this point: "JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences." (This kind of text is why I prefer the ECMA document -- it's clearly written to be a normative standard, rather than as a field-report and Request for Comments.)
    - stickfigure 6y ago
      ...and a lot of other tooling assumes the order is relevant. Quite a lot of application logic in the real world, too. Which is why browsers preserve iteration order. JSON is a serialization format. Its components inherently have a serial order. You can't change this any more than you can legislate the value of pi to be 3.
      - gmfawcett 6y ago
        Nitpicking, but JSON was designed as a data-interchange format, it only happens to be used for serialization. I agree that any kind of pipeline component (i.e., not a source, and not a sink) should preserve ordering where possible, for the sake of robustness.
msluyter 6y ago
In theory, order shouldn't matter, right? But I recall seeing this trick for adding comments to json: { "foo": "this is a comment about foo", "foo": "actual value of foo that overwrites the comment" } The trick is that the second value value of foo overwrites the first. But, clearly, sorting would would wreak havoc here (if the value was used in the sort key). ;)
- Buttons840 6y ago
  Yes. That is valid JSON. A fun fact about MongoDB is it will actually store that JSON, both duplicate keys. The implication is that whatever MongoDB client you're using, that maps Mongo data to dictionaries/maps, is not capable of representing all valid MongoDB documents. It's important to recognize that Mongo may be storing data your client will not be able to access. I learned this when the Python client was showing one value for a key, and the Ruby client was showing another value for the same key, and neither client was showing the whole document.
  - jd_mongodb 6y ago
    I can't reproduce that behaviour on MongoDB 4.2.x using Python and the Shell client.
    - Buttons840 6y ago
      It was the case some years ago. I'm not sure if it still is the case.
dragonwriter 6y ago
> The JSON language spec has ORDERED object members. False. “An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.” [emphasis added][0] [0] https://tools.ietf.org/html/rfc8259 https://tools.ietf.org/html/rfc8259
- ucarion 6y ago
  Not exactly, unfortunately. The text you cite is in the introduction, which is non-normative. That text talks about the conceptual data model, but that's just to frame the reader's thinking. The normative text has the "real" answer, and the real answer is that it's basically undefined behavior. It starts by saying "The names within an object SHOULD be unique", and then elaborates: An object whose names are all unique is interoperable in the sense that all software implementations receiving that object will agree on the name-value mappings. When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates. JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences. https://tools.ietf.org/html/rfc8259#section-4 https://tools.ietf.org/html/rfc8259#section-4
  - hyperion2010 6y ago
    This reads like multiple instances of the same name in a json object is undefined behavior and the user will get what they deserve if they try to rely on that behavior.
  - dragonwriter 6y ago
    > The normative text Unlike some RFCs, which clearly and explicitly delineate normative from informative material, RFC 8259 does not, but the text you cite is on its face informative rather than normative: it does not specify what an implementation MUST or SHOULD do, or what the object model IS, it describes the variety of preexisting implementations (based, correctly or not, on prior specifications) that are in the wild.

by Garlef 6y ago

Garlef 6y ago

I checked the spec again: It's unclear: At one point it says "An object is an unordered set of name/value pairs." while in the actual grammar it is ordered: object '{' ws '}' '{' members '}' members member member ',' members

gmfawcett 6y ago
I recommend reading the ECMA-404 spec instead. It's less ambiguous, and basically says that you can treat the pairs as ordered if you want, as the syntax itself doesn't imbue the order with any meaning: https://www.ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf https://www.ecma-international.org/wp-content/uploads/ECMA-4...
dragonwriter 6y ago
Grammar is inherently ordered, semantics is different from syntax.

peterohler 6y ago

I think you will find the JSON object members are not ordered while JSON array members are ordered. Since the JSON object members are not ordered, changing the order for display purposes does not change the data in any material way.

Latty 6y ago

The spec[1] says: > An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array. "whenever the order matters to the consumer of the JSON" should be never. More pragmatically, regardless of what the spec says, a ton of JSON tooling assumes the order doesn't matter and relying on it would be a big mistake. [1]: https://tools.ietf.org/html/rfc7159#section-1 https://tools.ietf.org/html/rfc7159#section-1

gmfawcett 6y ago
The other spec (ECMA-404) disagrees with RFC 7159 on this point. That's the great thing about specs -- if you don't like what one says, there's always another to support your position. :)
- Latty 6y ago
  Well, falling back to the pragmatist position: a lot of JSON tooling assumes the order isn't relevant semantically. Not taking that position will cause you a lot of headaches.
  - gmfawcett 6y ago
    Agreed, and if you want to write tools that are conformant with both specs, you really don't have a choice. The RFC pretty much spells out this point: "JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences." (This kind of text is why I prefer the ECMA document -- it's clearly written to be a normative standard, rather than as a field-report and Request for Comments.)
  - stickfigure 6y ago
    ...and a lot of other tooling assumes the order is relevant. Quite a lot of application logic in the real world, too. Which is why browsers preserve iteration order. JSON is a serialization format. Its components inherently have a serial order. You can't change this any more than you can legislate the value of pi to be 3.
    - gmfawcett 6y ago
      Nitpicking, but JSON was designed as a data-interchange format, it only happens to be used for serialization. I agree that any kind of pipeline component (i.e., not a source, and not a sink) should preserve ordering where possible, for the sake of robustness.

msluyter 6y ago

In theory, order shouldn't matter, right? But I recall seeing this trick for adding comments to json: { "foo": "this is a comment about foo", "foo": "actual value of foo that overwrites the comment" } The trick is that the second value value of foo overwrites the first. But, clearly, sorting would would wreak havoc here (if the value was used in the sort key). ;)

Buttons840 6y ago
Yes. That is valid JSON. A fun fact about MongoDB is it will actually store that JSON, both duplicate keys. The implication is that whatever MongoDB client you're using, that maps Mongo data to dictionaries/maps, is not capable of representing all valid MongoDB documents. It's important to recognize that Mongo may be storing data your client will not be able to access. I learned this when the Python client was showing one value for a key, and the Ruby client was showing another value for the same key, and neither client was showing the whole document.
- jd_mongodb 6y ago
  I can't reproduce that behaviour on MongoDB 4.2.x using Python and the Shell client.
  - Buttons840 6y ago
    It was the case some years ago. I'm not sure if it still is the case.

dragonwriter 6y ago

> The JSON language spec has ORDERED object members. False. “An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.” [emphasis added][0] [0] https://tools.ietf.org/html/rfc8259 https://tools.ietf.org/html/rfc8259

ucarion 6y ago
Not exactly, unfortunately. The text you cite is in the introduction, which is non-normative. That text talks about the conceptual data model, but that's just to frame the reader's thinking. The normative text has the "real" answer, and the real answer is that it's basically undefined behavior. It starts by saying "The names within an object SHOULD be unique", and then elaborates: An object whose names are all unique is interoperable in the sense that all software implementations receiving that object will agree on the name-value mappings. When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates. JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences. https://tools.ietf.org/html/rfc8259#section-4 https://tools.ietf.org/html/rfc8259#section-4
- hyperion2010 6y ago
  This reads like multiple instances of the same name in a json object is undefined behavior and the user will get what they deserve if they try to rely on that behavior.
- dragonwriter 6y ago
  > The normative text Unlike some RFCs, which clearly and explicitly delineate normative from informative material, RFC 8259 does not, but the text you cite is on its face informative rather than normative: it does not specify what an implementation MUST or SHOULD do, or what the object model IS, it describes the variety of preexisting implementations (based, correctly or not, on prior specifications) that are in the wild.