7 ms·

Son – A minimal subset of JSON for machine-to-machine communication

47 points by seagreen 10y ago

nerdponx 10y ago
Cute name, but 1) it's totally un-Googleable, and 2) it doesn't mean anything.
- seagreen 10y ago
  I wanted to name it something like "mson" for "minimal-JSON", but that's already taken. EDIT: Not meaning anything is a plus! But I see your point about being un-googleable.
  - pacaro 10y ago
    ASN.1 uses the name DER Distinguished Encoding Rules for this purpose. There are some other edgecases that you aren't considering (or that I missed) Min and Max integer values - JavaScript has some pretty tight limits here Keys should be in lexicographic order — you might want to be more specific. Is there a permitted subset of Unicode? Which normalization rule must be used?
    - seagreen 10y ago
      There's an issue for min and max number values here: https://github.com/seagreen/Son/issues/3 https://github.com/seagreen/Son/issues/3 Son seems like a natural place to put restrictions on them, but I'm not sure if there's way to do it that (A) still makes for a clean spec and (B) still allows everything floats can encode to to be used.
      - pacaro 10y ago
        Yeah, as with the trailing zero, you're being burned by JavaScript's decision to just have a Number type. As far as js is concerned 1 and 1.0 are the same number.
      - pacaro 10y ago
        On an unrelated note. If human readability isn't a primary concern, I'd recommend requiring that < > & all be escaped, this prevents some crappy attacks Also, no top level lists!
        seagreen 10y ago
        Why no top level lists?
        pacaro 10y ago
        Ha! I guess I'm showing my age. There used to be an associated vulnerability, which is why some frameworks won't let you do it. It appears that the vulnerability was fixed in what are now fairly old versions of browsers. More info at http://stackoverflow.com/questions/16289894/is-json-hijacking-still-an-issue-in-modern-browsers http://stackoverflow.com/questions/16289894/is-json-hijackin...
  - drewnoakes 10y ago
    JSCON: JavaScript Canonical Object Notation
    - seagreen 10y ago
      This is a great name. I'll probably stick with Son because it's already released, but honestly this would have been better.
- shujito 10y ago
  How about sson?
- jasonkostempski 10y ago
  How about "System Object Notation"?
seagreen 10y ago
Author here. I originally started this project because I wanted a consistent way to serialize JSON so that the serialized bytes would hash the same way every time. As I worked on it though I realized it might be of general interested to people. Thus the example in the README of piping JSON through multiple tools without generating trivial changes that mess up diffs. Most of the decisions I made were clear: no insignificant whitespace, object keys must be ordered, etc. There are two things I'm still not sure about: + Son doesn't provide escape sequences for any Unicode character that JSON allows to be written unescaped. This includes U+007f (ASCII "delete"). Will that cause a problem for many programs? All the other ASCII control characters are required to be escaped by JSON, U+007f is the only one left out. + Son doesn't allow trailing zeros in fractions. This means you can't serialize `1.0`, you have to serialize it as `1`. I was confident in the decision to take out scientific notation (it would be cool if JSON parsers actually treated numbers as being in scientific notation and tracked significant digits, but they don't so I feel like that ship has sailed). Trailing zeros are different though because some JSON generators do use them to distinguish integers from fractions. The problem is that many parsers don't care about them, so you end up in a situation where parsers are tossing out information about documents, meaning they can't serialize them faithfully again which is the whole point of Son.
- mst 10y ago
  I am tempted to suggest either: 1. Keep 1.0 as a special case to maintain the int/float distinction (it's a float; calling it a fraction is kinda-of-a-lie). 2. Refuse to handle floats at all, at which point people can pass [ <mantissa>, <exponent> ] for reals or [ <numerator>, <denominator> ] for rationals. There is of course (3), "build a compliance suite and claim the parsers that toss out information are Incorrect", but that doesn't seem compatible with your postel-ish goals.
  - seagreen 10y ago
    > it's a float; calling it a fraction is kinda-of-a-lie This happens not to be correct. By specification JSON numbers are just a series of characters, arranged in a certain way: https://tools.ietf.org/html/rfc7159#section-6 https://tools.ietf.org/html/rfc7159#section-6 In practice though many JSON parsers will parse non-integer numbers to floats. > 2. Refuse to handle floats at all, at which point people can pass [ <mantissa>, <exponent> ] for reals or [ <numerator>, <denominator> ] for rationals. This is a really interesting idea. If you're writing something that's super important like medical software it would probably be worth considering. However, my goals are just to make minimal changes that improve JSON some while still keeping it fairly readable, so I think that means I should stick with allowing `123.456` or whatever. I'd like to try to keep an open mind on this though.
mixedCase 10y ago
Why not use something like Protobufs and Cap'n'Proto instead of yet another serialization format?
- kevin_thibedeau 10y ago
  They require a schema. A schemaless binary serialization like CBOR or BSON is more amenable to JSON interchange.
  - k__ 10y ago
    How about http://msgpack.org/ http://msgpack.org/?
    - seagreen 10y ago
      Son's a niche tool to allow talking to JSON services in a more consistent way. It's not meant as an competitor to any other data format. EDIT: s/alternative/competitor
- CJefferson 10y ago
  I always find myself finding myself back with JSON. The advantage it can be parsed easily in any language -- everything has a JSON parser.
  - hnbroseph 10y ago
    protobufs are fairly wide spread... from action script to erlang to haskell to visual basic.
    - CJefferson 10y ago
      Last time I tried (6 months ago) I couldn't find a proto3 implementation for Haskell that was complete ( https://github.com/google/proto-lens https://github.com/google/proto-lens doesn't do Any, https://github.com/alphaHeavy/protobuf https://github.com/alphaHeavy/protobuf doesn't do proto3). Maybe one exists, or I could use proto2 and rewrite the other system I was working with. I decided it was easier in my case to just output JSON.
nly 10y ago
I've implemented similarly constrained string representations for numerics for a config file format, but I ended up allowing both "1" and "1.0" so I could distinguish integer and floating point literals ("1.00" was still forbidden, as was "0.0"... zero was stored in a specially tagged way). The exponent syntax in JSON is fun because it allows numerous representations of 0 and 1 using e.g. 0e1 and 7e0
- seagreen 10y ago
  > I ended up allowing both "1" and "1.0" so I could distinguish integer and floating point literals Definitely the toughest call when making a JSON subset like this.
pwdisswordfish 10y ago
Umm... why? The only rationale for this project I can see is this: > Piping JSON through multiple programs creates lots of trivial changes, which makes it hard to do things like take meaningful diffs. But you can always put JSON through a pretty-printer which puts values into canonical form before diffing. I wouldn't bother turning it into a formal specification. And you mention 'No insignificant whitespace' in the README, so it's not like your format makes line-by-line diffs any clearer. It's as if IT hasn't had enough solutions in search of a problem...
- seagreen 10y ago
  > I wouldn't bother turning it into a formal specification. Too late! > But you can always put JSON through a pretty-printer which puts values into canonical form before diffing. Son is a starting point for building such a pretty printer. It takes care of messy details like eliminating the redundancies in string and number encoding, so all you have to do to specify the pretty-printer format is say where you want your newlines and how much to indent by.
  - kornish 10y ago
    > Son is a starting point for building such a pretty printer. I don't understand: such pretty printers already exist (e.g. jq, which does a whole lot more [0]). If you're transmitting JSON and want to diff two documents, just pipe them into jq or another pretty-printer with key ordering, then use one of many existing line-by-line diff tools. [0]: github.com/stedolan/jq
    - seagreen 10y ago
      jq is great. You can actually see how it's used to test the reference implementation of Son here (https://github.com/seagreen/Son/blob/master/implementation/test/JQ.hs https://github.com/seagreen/Son/blob/master/implementation/t...) using `jq --compact-output --sort-keys .` Unfortunately, jq doesn't provide flags to control scientific/non-scientific notation or which characters are escaped, meaning if you want very tight control over the JSON generated it's not a full option. (Consider that the motivating example in the Son README isn't all you might want to use Son for. For instance some people need consistent hashing of serialized JSON documents).
dsp1234 10y ago
"Object members must be sorted by ascending lexicographic order of their keys." How should the following be serialized? {"öp":1, "op":0}
- kbenson 10y ago
  Isn't that invalid? You can't have two object members with the same name, can you? I would assume that should be impossible to emit.
  - nocman 10y ago
    Um, the first character of each key is different -- they are not the same.
    - kbenson 10y ago
      Ah, I see. That wasn't obvious to me when I first looked. Isn't sorting of Unicode characters defined under Unicode? I think the correct answer for "how do I sort my unicode strings" is "defer to unicode". That may be quite expensive in the end, depending on object size and content.
      - cwyers 10y ago
        Alphabetical order changes from language to language. If you don't specify a collation, you can run into edge cases.
        wrs 10y ago
        And you can't specify "a collation" in a way that will be guaranteed to mean exactly the same thing next year. I don't believe there's a way to do a permanently stable sort if you also want it to reflect the current state of Unicode. (An obvious example is that an entire language might have been added to Unicode since you serialized something with those code points in it.) NTFS, which has to build a b-tree of filenames in a permanently stable sort order, solved this by writing the collation table to the disk when it's formatted, and never changing it again. Which means the Windows shell still has to re-sort the filenames because the currently "correct" collation may be different.
zeveb 10y ago
> Piping JSON through multiple programs creates lots of trivial changes, which makes it hard to do things like take meaningful diffs. That's the virtue of having a format which offers a canonical representation for data. > No insignificant whitespace. I.e., it's not human-readable. It also has no decent way to exchange binary data (there is no byte-sequence type: one must either use Base64 or an array of integers, neither of which is space-efficient). It'd be nice to have a format which is both human-readable and has a canonical representation. Fortunately, such a thing already exists (and I've even linked to it once already today), and has since 1997: http://people.csail.mit.edu/rivest/Sexp.txt http://people.csail.mit.edu/rivest/Sexp.txt Here's a JSON example (from http://json.org/example.html http://json.org/example.html): { "glossary": { "title": "example glossary", "GlossDiv": { "title": "S", "GlossList": { "GlossEntry": { "ID": "SGML", "SortAs": "SGML", "GlossTerm": "Standard Generalized Markup Language", "Acronym": "SGML", "Abbrev": "ISO 8879:1986", "GlossDef": { "para": "A meta-markup language, used to create markup languages such as DocBook.", "GlossSeeAlso": ["GML", "XML"] }, "GlossSee": "markup" } } } } } Here it is in SON: {"glossary":{"GlossDiv":{"GlossList":{"GlossEntry":{"Abbrev":"ISO 8879:1986","Acronym":"SGML","ID":"SGML","GlossDef":{"GlossSeeAlso":["GML","XML"],"para":"A meta-markup language, used to create markup languages such as DocBook."},"GlossSee":"markup","GlossTerm":"Standard Generalized Markup Language","SortAs":"SGML"}},"title":"S"},"title":"example glossary"}} Here it is in an advanced S-expression representation: (glossary "example glossary" (div S (entry SGML (sort SGML) (term "Standard Generalized Markup Language") (acronym SGML) (abbrev "ISO 8879:1986") (def "A meta-markup language, used to create markup languages such as DocBook." (see-also GML XML)) (see markup)))) And here it is in its canonical representation: (8:glossary16:example glossary(3:div1:S(5:entry4:SGML(4:sort4:SGML)(4:term36:Standard Generalized Markup Language)(7:acronym4:SGML)(6:abbrev13:ISO 8879:1986)(3:def72:A meta-markup language, used to create markup languages such as DocBook.(8:see-also3:GML3:XML))(3:see6:markup)))) What, you'd like something which is immune to 7-bit/8-bit or email mangling? Here's the same data in transport format: {KDg6Z2xvc3NhcnkxNjpleGFtcGxlIGdsb3NzYXJ5KDM6ZGl2MTpTKDU6ZW50cnk0OlNHTUwoNDpz b3J0NDpTR01MKSg0OnRlcm0zNjpTdGFuZGFyZCBHZW5lcmFsaXplZCBNYXJrdXAgTGFuZ3VhZ2Up KDc6YWNyb255bTQ6U0dNTCkoNjphYmJyZXYxMzpJU08gODg3OToxOTg2KSgzOmRlZjcyOkEgbWV0 YS1tYXJrdXAgbGFuZ3VhZ2UsIHVzZWQgdG8gY3JlYXRlIG1hcmt1cCBsYW5ndWFnZXMgc3VjaCBh cyBEb2NCb29rLig4OnNlZS1hbHNvMzpHTUwzOlhNTCkpKDM6c2VlNjptYXJrdXApKSkp} All three of those S-expression formats can be losslessly converted to one another. Ordering is exactly as specified (they are lists, not unordered or ordered dicts — although one can understand them as dicts, if desired).
bgamari 10y ago
Isn't this just a reinvention of Canonical JSON [1]? There were efforts [2] to standardize it, although it seems this fizzled out. [1] http://wiki.laptop.org/go/Canonical_JSON http://wiki.laptop.org/go/Canonical_JSON [2] https://datatracker.ietf.org/doc/draft-staykov-hu-json-canonical-form/ https://datatracker.ietf.org/doc/draft-staykov-hu-json-canon...
- seagreen 10y ago
  Thanks for brining Canonical JSON up. There's an issue to investigate it here: https://github.com/seagreen/Son/issues/7 https://github.com/seagreen/Son/issues/7 It doesn't appear that Canonical JSON does anything about redundant escape sequences in JSON. I'm still looking into it to be sure. This is a big part of the motivation behind the Son spec, and represents about half the EBNF: https://github.com/seagreen/Son/blob/master/son.ebnf#L19 https://github.com/seagreen/Son/blob/master/son.ebnf#L19 For anyone else who knows of more JSON subsets, if you report them here: https://housejeffries.com/page/7 https://housejeffries.com/page/7 it would be really appreciated. I definitely don't want to do duplicate work.
slowmovintarget 10y ago
You could use JSON as a host: https://github.com/cognitect/transit-format https://github.com/cognitect/transit-format I do like the comment about having the S-expression format for data interchange defined since 1997.
rc_bhg 10y ago
Whyyyyyyy????????????
dsp1234 10y ago
"Object keys must be unique."" How should the following be serialized (if at all)? {"a":0,"\u0073\u0323\u0307":1,"\u1E69":2,"Z":3} Note that I'm using the \u forms even though they aren't allowed via the specification because, HN doesn't show unicode combining properly sometimes, and so that's it's clear to see that it's a different byte pattern.
- seagreen 10y ago
  The plan is to go codepoint by codepoint in ascending order: `{"Z":3,"a":0,"\u0073\u0323\u0307":1,"\u1E69":2}` See this issue for the discussion: https://github.com/seagreen/Son/issues/1 https://github.com/seagreen/Son/issues/1 My current understanding is that RFC 7159 doesn't require Unicode normalization to be performed, if it does we're in trouble: https://tools.ietf.org/html/rfc7159 https://tools.ietf.org/html/rfc7159

47 points by seagreen 10y ago

nerdponx 10y ago

Cute name, but 1) it's totally un-Googleable, and 2) it doesn't mean anything.

seagreen 10y ago
I wanted to name it something like "mson" for "minimal-JSON", but that's already taken. EDIT: Not meaning anything is a plus! But I see your point about being un-googleable.
- pacaro 10y ago
  ASN.1 uses the name DER Distinguished Encoding Rules for this purpose. There are some other edgecases that you aren't considering (or that I missed) Min and Max integer values - JavaScript has some pretty tight limits here Keys should be in lexicographic order — you might want to be more specific. Is there a permitted subset of Unicode? Which normalization rule must be used?
  - seagreen 10y ago
    There's an issue for min and max number values here: https://github.com/seagreen/Son/issues/3 https://github.com/seagreen/Son/issues/3 Son seems like a natural place to put restrictions on them, but I'm not sure if there's way to do it that (A) still makes for a clean spec and (B) still allows everything floats can encode to to be used.
    - pacaro 10y ago
      Yeah, as with the trailing zero, you're being burned by JavaScript's decision to just have a Number type. As far as js is concerned 1 and 1.0 are the same number.
    - pacaro 10y ago
      On an unrelated note. If human readability isn't a primary concern, I'd recommend requiring that < > & all be escaped, this prevents some crappy attacks Also, no top level lists!
      - seagreen 10y ago
        Why no top level lists?
        pacaro 10y ago
        Ha! I guess I'm showing my age. There used to be an associated vulnerability, which is why some frameworks won't let you do it. It appears that the vulnerability was fixed in what are now fairly old versions of browsers. More info at http://stackoverflow.com/questions/16289894/is-json-hijacking-still-an-issue-in-modern-browsers http://stackoverflow.com/questions/16289894/is-json-hijackin...
- drewnoakes 10y ago
  JSCON: JavaScript Canonical Object Notation
  - seagreen 10y ago
    This is a great name. I'll probably stick with Son because it's already released, but honestly this would have been better.
shujito 10y ago
How about sson?
jasonkostempski 10y ago
How about "System Object Notation"?

seagreen 10y ago

Author here. I originally started this project because I wanted a consistent way to serialize JSON so that the serialized bytes would hash the same way every time. As I worked on it though I realized it might be of general interested to people. Thus the example in the README of piping JSON through multiple tools without generating trivial changes that mess up diffs. Most of the decisions I made were clear: no insignificant whitespace, object keys must be ordered, etc. There are two things I'm still not sure about: + Son doesn't provide escape sequences for any Unicode character that JSON allows to be written unescaped. This includes U+007f (ASCII "delete"). Will that cause a problem for many programs? All the other ASCII control characters are required to be escaped by JSON, U+007f is the only one left out. + Son doesn't allow trailing zeros in fractions. This means you can't serialize `1.0`, you have to serialize it as `1`. I was confident in the decision to take out scientific notation (it would be cool if JSON parsers actually treated numbers as being in scientific notation and tracked significant digits, but they don't so I feel like that ship has sailed). Trailing zeros are different though because some JSON generators do use them to distinguish integers from fractions. The problem is that many parsers don't care about them, so you end up in a situation where parsers are tossing out information about documents, meaning they can't serialize them faithfully again which is the whole point of Son.

mst 10y ago
I am tempted to suggest either: 1. Keep 1.0 as a special case to maintain the int/float distinction (it's a float; calling it a fraction is kinda-of-a-lie). 2. Refuse to handle floats at all, at which point people can pass [ <mantissa>, <exponent> ] for reals or [ <numerator>, <denominator> ] for rationals. There is of course (3), "build a compliance suite and claim the parsers that toss out information are Incorrect", but that doesn't seem compatible with your postel-ish goals.
- seagreen 10y ago
  > it's a float; calling it a fraction is kinda-of-a-lie This happens not to be correct. By specification JSON numbers are just a series of characters, arranged in a certain way: https://tools.ietf.org/html/rfc7159#section-6 https://tools.ietf.org/html/rfc7159#section-6 In practice though many JSON parsers will parse non-integer numbers to floats. > 2. Refuse to handle floats at all, at which point people can pass [ <mantissa>, <exponent> ] for reals or [ <numerator>, <denominator> ] for rationals. This is a really interesting idea. If you're writing something that's super important like medical software it would probably be worth considering. However, my goals are just to make minimal changes that improve JSON some while still keeping it fairly readable, so I think that means I should stick with allowing `123.456` or whatever. I'd like to try to keep an open mind on this though.

mixedCase 10y ago

Why not use something like Protobufs and Cap'n'Proto instead of yet another serialization format?

kevin_thibedeau 10y ago
They require a schema. A schemaless binary serialization like CBOR or BSON is more amenable to JSON interchange.
- k__ 10y ago
  How about http://msgpack.org/ http://msgpack.org/?
  - seagreen 10y ago
    Son's a niche tool to allow talking to JSON services in a more consistent way. It's not meant as an competitor to any other data format. EDIT: s/alternative/competitor
CJefferson 10y ago
I always find myself finding myself back with JSON. The advantage it can be parsed easily in any language -- everything has a JSON parser.
- hnbroseph 10y ago
  protobufs are fairly wide spread... from action script to erlang to haskell to visual basic.
  - CJefferson 10y ago
    Last time I tried (6 months ago) I couldn't find a proto3 implementation for Haskell that was complete ( https://github.com/google/proto-lens https://github.com/google/proto-lens doesn't do Any, https://github.com/alphaHeavy/protobuf https://github.com/alphaHeavy/protobuf doesn't do proto3). Maybe one exists, or I could use proto2 and rewrite the other system I was working with. I decided it was easier in my case to just output JSON.

nly 10y ago

I've implemented similarly constrained string representations for numerics for a config file format, but I ended up allowing both "1" and "1.0" so I could distinguish integer and floating point literals ("1.00" was still forbidden, as was "0.0"... zero was stored in a specially tagged way). The exponent syntax in JSON is fun because it allows numerous representations of 0 and 1 using e.g. 0e1 and 7e0

seagreen 10y ago
> I ended up allowing both "1" and "1.0" so I could distinguish integer and floating point literals Definitely the toughest call when making a JSON subset like this.

pwdisswordfish 10y ago

Umm... why? The only rationale for this project I can see is this: > Piping JSON through multiple programs creates lots of trivial changes, which makes it hard to do things like take meaningful diffs. But you can always put JSON through a pretty-printer which puts values into canonical form before diffing. I wouldn't bother turning it into a formal specification. And you mention 'No insignificant whitespace' in the README, so it's not like your format makes line-by-line diffs any clearer. It's as if IT hasn't had enough solutions in search of a problem...

seagreen 10y ago
> I wouldn't bother turning it into a formal specification. Too late! > But you can always put JSON through a pretty-printer which puts values into canonical form before diffing. Son is a starting point for building such a pretty printer. It takes care of messy details like eliminating the redundancies in string and number encoding, so all you have to do to specify the pretty-printer format is say where you want your newlines and how much to indent by.
- kornish 10y ago
  > Son is a starting point for building such a pretty printer. I don't understand: such pretty printers already exist (e.g. jq, which does a whole lot more [0]). If you're transmitting JSON and want to diff two documents, just pipe them into jq or another pretty-printer with key ordering, then use one of many existing line-by-line diff tools. [0]: github.com/stedolan/jq
  - seagreen 10y ago
    jq is great. You can actually see how it's used to test the reference implementation of Son here (https://github.com/seagreen/Son/blob/master/implementation/test/JQ.hs https://github.com/seagreen/Son/blob/master/implementation/t...) using `jq --compact-output --sort-keys .` Unfortunately, jq doesn't provide flags to control scientific/non-scientific notation or which characters are escaped, meaning if you want very tight control over the JSON generated it's not a full option. (Consider that the motivating example in the Son README isn't all you might want to use Son for. For instance some people need consistent hashing of serialized JSON documents).

dsp1234 10y ago

"Object members must be sorted by ascending lexicographic order of their keys." How should the following be serialized? {"öp":1, "op":0}

kbenson 10y ago
Isn't that invalid? You can't have two object members with the same name, can you? I would assume that should be impossible to emit.
- nocman 10y ago
  Um, the first character of each key is different -- they are not the same.
  - kbenson 10y ago
    Ah, I see. That wasn't obvious to me when I first looked. Isn't sorting of Unicode characters defined under Unicode? I think the correct answer for "how do I sort my unicode strings" is "defer to unicode". That may be quite expensive in the end, depending on object size and content.
    - cwyers 10y ago
      Alphabetical order changes from language to language. If you don't specify a collation, you can run into edge cases.
      - wrs 10y ago
        And you can't specify "a collation" in a way that will be guaranteed to mean exactly the same thing next year. I don't believe there's a way to do a permanently stable sort if you also want it to reflect the current state of Unicode. (An obvious example is that an entire language might have been added to Unicode since you serialized something with those code points in it.) NTFS, which has to build a b-tree of filenames in a permanently stable sort order, solved this by writing the collation table to the disk when it's formatted, and never changing it again. Which means the Windows shell still has to re-sort the filenames because the currently "correct" collation may be different.

zeveb 10y ago

> Piping JSON through multiple programs creates lots of trivial changes, which makes it hard to do things like take meaningful diffs. That's the virtue of having a format which offers a canonical representation for data. > No insignificant whitespace. I.e., it's not human-readable. It also has no decent way to exchange binary data (there is no byte-sequence type: one must either use Base64 or an array of integers, neither of which is space-efficient). It'd be nice to have a format which is both human-readable and has a canonical representation. Fortunately, such a thing already exists (and I've even linked to it once already today), and has since 1997: http://people.csail.mit.edu/rivest/Sexp.txt http://people.csail.mit.edu/rivest/Sexp.txt Here's a JSON example (from http://json.org/example.html http://json.org/example.html): { "glossary": { "title": "example glossary", "GlossDiv": { "title": "S", "GlossList": { "GlossEntry": { "ID": "SGML", "SortAs": "SGML", "GlossTerm": "Standard Generalized Markup Language", "Acronym": "SGML", "Abbrev": "ISO 8879:1986", "GlossDef": { "para": "A meta-markup language, used to create markup languages such as DocBook.", "GlossSeeAlso": ["GML", "XML"] }, "GlossSee": "markup" } } } } } Here it is in SON: {"glossary":{"GlossDiv":{"GlossList":{"GlossEntry":{"Abbrev":"ISO 8879:1986","Acronym":"SGML","ID":"SGML","GlossDef":{"GlossSeeAlso":["GML","XML"],"para":"A meta-markup language, used to create markup languages such as DocBook."},"GlossSee":"markup","GlossTerm":"Standard Generalized Markup Language","SortAs":"SGML"}},"title":"S"},"title":"example glossary"}} Here it is in an advanced S-expression representation: (glossary "example glossary" (div S (entry SGML (sort SGML) (term "Standard Generalized Markup Language") (acronym SGML) (abbrev "ISO 8879:1986") (def "A meta-markup language, used to create markup languages such as DocBook." (see-also GML XML)) (see markup)))) And here it is in its canonical representation: (8:glossary16:example glossary(3:div1:S(5:entry4:SGML(4:sort4:SGML)(4:term36:Standard Generalized Markup Language)(7:acronym4:SGML)(6:abbrev13:ISO 8879:1986)(3:def72:A meta-markup language, used to create markup languages such as DocBook.(8:see-also3:GML3:XML))(3:see6:markup)))) What, you'd like something which is immune to 7-bit/8-bit or email mangling? Here's the same data in transport format: {KDg6Z2xvc3NhcnkxNjpleGFtcGxlIGdsb3NzYXJ5KDM6ZGl2MTpTKDU6ZW50cnk0OlNHTUwoNDpz b3J0NDpTR01MKSg0OnRlcm0zNjpTdGFuZGFyZCBHZW5lcmFsaXplZCBNYXJrdXAgTGFuZ3VhZ2Up KDc6YWNyb255bTQ6U0dNTCkoNjphYmJyZXYxMzpJU08gODg3OToxOTg2KSgzOmRlZjcyOkEgbWV0 YS1tYXJrdXAgbGFuZ3VhZ2UsIHVzZWQgdG8gY3JlYXRlIG1hcmt1cCBsYW5ndWFnZXMgc3VjaCBh cyBEb2NCb29rLig4OnNlZS1hbHNvMzpHTUwzOlhNTCkpKDM6c2VlNjptYXJrdXApKSkp} All three of those S-expression formats can be losslessly converted to one another. Ordering is exactly as specified (they are lists, not unordered or ordered dicts — although one can understand them as dicts, if desired).

bgamari 10y ago

Isn't this just a reinvention of Canonical JSON [1]? There were efforts [2] to standardize it, although it seems this fizzled out. [1] http://wiki.laptop.org/go/Canonical_JSON http://wiki.laptop.org/go/Canonical_JSON [2] https://datatracker.ietf.org/doc/draft-staykov-hu-json-canonical-form/ https://datatracker.ietf.org/doc/draft-staykov-hu-json-canon...

seagreen 10y ago
Thanks for brining Canonical JSON up. There's an issue to investigate it here: https://github.com/seagreen/Son/issues/7 https://github.com/seagreen/Son/issues/7 It doesn't appear that Canonical JSON does anything about redundant escape sequences in JSON. I'm still looking into it to be sure. This is a big part of the motivation behind the Son spec, and represents about half the EBNF: https://github.com/seagreen/Son/blob/master/son.ebnf#L19 https://github.com/seagreen/Son/blob/master/son.ebnf#L19 For anyone else who knows of more JSON subsets, if you report them here: https://housejeffries.com/page/7 https://housejeffries.com/page/7 it would be really appreciated. I definitely don't want to do duplicate work.

slowmovintarget 10y ago

You could use JSON as a host: https://github.com/cognitect/transit-format https://github.com/cognitect/transit-format I do like the comment about having the S-expression format for data interchange defined since 1997.

rc_bhg 10y ago

Whyyyyyyy????????????

dsp1234 10y ago

"Object keys must be unique."" How should the following be serialized (if at all)? {"a":0,"\u0073\u0323\u0307":1,"\u1E69":2,"Z":3} Note that I'm using the \u forms even though they aren't allowed via the specification because, HN doesn't show unicode combining properly sometimes, and so that's it's clear to see that it's a different byte pattern.

seagreen 10y ago
The plan is to go codepoint by codepoint in ascending order: `{"Z":3,"a":0,"\u0073\u0323\u0307":1,"\u1E69":2}` See this issue for the discussion: https://github.com/seagreen/Son/issues/1 https://github.com/seagreen/Son/issues/1 My current understanding is that RFC 7159 doesn't require Unicode normalization to be performed, if it does we're in trouble: https://tools.ietf.org/html/rfc7159 https://tools.ietf.org/html/rfc7159