6 ms·

Rather than keep making new variants of JSON, it'd be nice if somebody could convince some mainstream language maintainers to just update their built-in JSON pa

Rather than keep making new variants of JSON, it'd be nice if somebody could convince some mainstream language maintainers to just update their built-in JSON parser to add optional features like skipping over comments and not caring about trailing commas. Most parsers support various flags already to configure things, so there could just be an ALLOW_COMMENTS flag and ALLOW_TRAILING_COMMAS flag. As a case in point, the python parser breaks the JSON spec already with regards to Infinity/NaN, but then has flags to configure this. See https://docs.python.org/3/library/json.html#module-json https://docs.python.org/3/library/json.html#module-json > The RFC does not permit the representation of infinite or NaN number values. Despite that, by default, this module accepts and outputs Infinity, -Infinity, and NaN as if they were valid JSON number literal values:

xpe 6y ago
I can see why this example appeals to you. However, asking implementors to go outside of a spec is _in effect_ making a new JSON variant. Once you see this, you can understand why people want a spec with a name.
spankalee 6y ago
It's going to have to be a different mode and mimetype because this format would break all kinds of parsers. I wish something would done in this area relatively soon though because JS has added a number of features that would make a new JSON much nicer like multi-line strings and BigInts. I think JSON + comments, commas, template literals, BigInt, NaN, Infinity, and BigDecimals (if/when those land in JS) would be very useful. (It'd be nice to include dates, but that's tricky w/o a literal and because dates)
- fiddlerwoaroof 6y ago
  JSON doesn’t specify its numeric types: the mapping of a string of digits to concrete numeric types is implementation-defined: so, JSON doesn’t need specific syntax for BigInts or arbitrary-precision decimals.
  - spankalee 6y ago
    Current parsers cannot start returning BigInts instead of numbers without that being a breaking change. And I'm not sure anyone wants a format where the result may change types based on the size of the number, especially for languages where arbitrary precision types are not compatible with other numbers.
    - skissane 6y ago
      > Current parsers cannot start returning BigInts instead of numbers without that being a breaking change. Current parsers aren't uniform here. Since the JSON spec is silent on what post-parsing format is used for numbers, each parser is free to do whatever makes sense in the context of the host language. I reckon you'll find some JSON parsers use bigints already, especially in languages with first-class bigint support (such as various Lisp dialects) > And I'm not sure anyone wants a format where the result may change types based on the size of the number That's nothing to do with the format, that's to do with the parser. Some parsers already do exactly that – use an integer type for numbers that are integers, use a floating point type for numbers that contain decimal points
      - banana_giraffe 6y ago
        > I reckon you'll find some JSON parsers already use bigints already Yep. Python is one such language. I've seen this catch people by surprise when they discover their serial number (which granted, should have been a string in the first place) doesn't survive a trip from Python to JSON to Javascript, among other languages.
        colejohnson66 6y ago
        It doesn’t help that it’s called a serial number, and is sequential (hence, serial, but some manufacturers don’t care). num++ is easier than implementing increment_number_string that works on ASCII digits.
        6y ago
dmitryminkovsky 6y ago
JSON parsing is already a minefield. Please see [0], specifically this chart [1]. As mentioned in a sibling comment, I think a new mimetype makes a lot more sense sense than stirring this pot further. [0] http://seriot.ch/parsing_json.php http://seriot.ch/parsing_json.php [1] http://seriot.ch/json/pruned_results.png http://seriot.ch/json/pruned_results.png
- zeroimpl 6y ago
  It is a minefield, but we all walk it all the time. A new mime type doesn’t really help because the parser doesn’t check the mime type, it assumes the programmer did that. I’m not against making a new mime type or specification, but there are already so many and making more doesn’t seem to help. Yes it would be nice if PHP and Python implement the logic for ignoring trailing commas the same way, but 99% of the time that isn’t all that important. Yes there will be cases where it matters and bugs are introduced, but since there already significant differences between JSON parsers I don’t see these kinds of things as making it much worse.
thayne 6y ago
ruby's JSON.parse accepts comments (but not trailing commas)
k_sze 6y ago
> Rather than keep making new variants of JSON, it'd be nice if somebody could convince some mainstream language maintainers to just update their built-in JSON parser to add optional features like skipping over comments and not caring about trailing commas. I think the most correct way to deal with this problem is to get IETF and ECMA to update the JSON standard first. Honestly, comma-after-final-element and comments are such important quality-of-life features that I never understood why they weren't part of the original spec. On the other hand, it might be easier to just adopt TOML and forget about JSON.
- ma2rten 6y ago
  I think that json is mostly a lightweight, human-readable data exchange format for machines to communicate. It's generally not meant to be written by humans.
  - k_sze 6y ago
    There are already better formats if we just want machines to communicate though; e.g. protobuf, msgpack. The whole reason JSON is text-based is to make it human-readable while also enabling data exchange, but the lack of comments works against that goal.
    - BiteCode_dev 6y ago
      Not in the browser or in many stalin.
      - oblio 6y ago
        > many stalin. Wasn't there just one Stalin, though?
        bregma 6y ago
        One was certainly more than enough. Unfortunately, many have tried to pluralize.
acje 6y ago
Or just do RSON? https://github.com/rson-rs/rson https://github.com/rson-rs/rson I honestly don't want to bash JS, but typing is not it's strength. (Not affiliated)
- saghm 6y ago
  I think that's the opposite of what GP is advocating; they're arguing that a better approach than defining entire new formats, we should just augment existing JSON implementations. I'd guess that the rationale is that it would be a tougher uphill battle to get people to switch to an entirely new format, but adding new features to implementations they're already using wouldn't require making anyone switch.
  - acje 6y ago
    And I generally think good constraints are more empowering than good options when it comes to any form of systems design. It took me 20 years in the industry and 20 minutes with cynefin to understand this.
lifthrasiir 6y ago
Seriously, I believe the omission of Infinity and NaN from JSON is a huge mistake, if not Crockford's bad joke. It is commonly said that JSON was originally going to be `eval`ed and Infinity and NaN could have been redefined, but that eval had to be already preceded by filtering anyway so you can put a few lines to ensure that Infinity and NaN are expected values. Or use `Function` instead. Or use `1/0` or `0/0` as pseudo-literals. Not every JS value is present in the JSON data model, but it's absurd that not every JS number is present in it.
bshimmin 6y ago
Wouldn't you just end up with (for example) "Python-flavoured JSON", just like we have "GitHub-flavoured Markdown"? On the other hand, there is an RFC for JSON where there isn't for Markdown, so it ought to be possible to make use of the usual standards process to formalise a successor to the original JSON which might include some of the very common additions some people always want (trailing commas, comments, proper dates) - that this hasn't happened over the last 15 years suggests really that the problem is actually due to lack of agreement.
marcosdumay 6y ago
While we are at it, how about making the parsers accept non-ascii space, so that things don't break if somebody using Windows edit a file somewhere and adds a BOM.

xpe 6y ago

I can see why this example appeals to you. However, asking implementors to go outside of a spec is _in effect_ making a new JSON variant. Once you see this, you can understand why people want a spec with a name.

spankalee 6y ago

It's going to have to be a different mode and mimetype because this format would break all kinds of parsers. I wish something would done in this area relatively soon though because JS has added a number of features that would make a new JSON much nicer like multi-line strings and BigInts. I think JSON + comments, commas, template literals, BigInt, NaN, Infinity, and BigDecimals (if/when those land in JS) would be very useful. (It'd be nice to include dates, but that's tricky w/o a literal and because dates)

fiddlerwoaroof 6y ago
JSON doesn’t specify its numeric types: the mapping of a string of digits to concrete numeric types is implementation-defined: so, JSON doesn’t need specific syntax for BigInts or arbitrary-precision decimals.
- spankalee 6y ago
  Current parsers cannot start returning BigInts instead of numbers without that being a breaking change. And I'm not sure anyone wants a format where the result may change types based on the size of the number, especially for languages where arbitrary precision types are not compatible with other numbers.
  - skissane 6y ago
    > Current parsers cannot start returning BigInts instead of numbers without that being a breaking change. Current parsers aren't uniform here. Since the JSON spec is silent on what post-parsing format is used for numbers, each parser is free to do whatever makes sense in the context of the host language. I reckon you'll find some JSON parsers use bigints already, especially in languages with first-class bigint support (such as various Lisp dialects) > And I'm not sure anyone wants a format where the result may change types based on the size of the number That's nothing to do with the format, that's to do with the parser. Some parsers already do exactly that – use an integer type for numbers that are integers, use a floating point type for numbers that contain decimal points
    - banana_giraffe 6y ago
      > I reckon you'll find some JSON parsers already use bigints already Yep. Python is one such language. I've seen this catch people by surprise when they discover their serial number (which granted, should have been a string in the first place) doesn't survive a trip from Python to JSON to Javascript, among other languages.
      - colejohnson66 6y ago
        It doesn’t help that it’s called a serial number, and is sequential (hence, serial, but some manufacturers don’t care). num++ is easier than implementing increment_number_string that works on ASCII digits.
        6y ago

dmitryminkovsky 6y ago

JSON parsing is already a minefield. Please see [0], specifically this chart [1]. As mentioned in a sibling comment, I think a new mimetype makes a lot more sense sense than stirring this pot further. [0] http://seriot.ch/parsing_json.php http://seriot.ch/parsing_json.php [1] http://seriot.ch/json/pruned_results.png http://seriot.ch/json/pruned_results.png

zeroimpl 6y ago
It is a minefield, but we all walk it all the time. A new mime type doesn’t really help because the parser doesn’t check the mime type, it assumes the programmer did that. I’m not against making a new mime type or specification, but there are already so many and making more doesn’t seem to help. Yes it would be nice if PHP and Python implement the logic for ignoring trailing commas the same way, but 99% of the time that isn’t all that important. Yes there will be cases where it matters and bugs are introduced, but since there already significant differences between JSON parsers I don’t see these kinds of things as making it much worse.

thayne 6y ago

ruby's JSON.parse accepts comments (but not trailing commas)

k_sze 6y ago

> Rather than keep making new variants of JSON, it'd be nice if somebody could convince some mainstream language maintainers to just update their built-in JSON parser to add optional features like skipping over comments and not caring about trailing commas. I think the most correct way to deal with this problem is to get IETF and ECMA to update the JSON standard first. Honestly, comma-after-final-element and comments are such important quality-of-life features that I never understood why they weren't part of the original spec. On the other hand, it might be easier to just adopt TOML and forget about JSON.

ma2rten 6y ago
I think that json is mostly a lightweight, human-readable data exchange format for machines to communicate. It's generally not meant to be written by humans.
- k_sze 6y ago
  There are already better formats if we just want machines to communicate though; e.g. protobuf, msgpack. The whole reason JSON is text-based is to make it human-readable while also enabling data exchange, but the lack of comments works against that goal.
  - BiteCode_dev 6y ago
    Not in the browser or in many stalin.
    - oblio 6y ago
      > many stalin. Wasn't there just one Stalin, though?
      - bregma 6y ago
        One was certainly more than enough. Unfortunately, many have tried to pluralize.

acje 6y ago

Or just do RSON? https://github.com/rson-rs/rson https://github.com/rson-rs/rson I honestly don't want to bash JS, but typing is not it's strength. (Not affiliated)

saghm 6y ago
I think that's the opposite of what GP is advocating; they're arguing that a better approach than defining entire new formats, we should just augment existing JSON implementations. I'd guess that the rationale is that it would be a tougher uphill battle to get people to switch to an entirely new format, but adding new features to implementations they're already using wouldn't require making anyone switch.
- acje 6y ago
  And I generally think good constraints are more empowering than good options when it comes to any form of systems design. It took me 20 years in the industry and 20 minutes with cynefin to understand this.

lifthrasiir 6y ago

Seriously, I believe the omission of Infinity and NaN from JSON is a huge mistake, if not Crockford's bad joke. It is commonly said that JSON was originally going to be `eval`ed and Infinity and NaN could have been redefined, but that eval had to be already preceded by filtering anyway so you can put a few lines to ensure that Infinity and NaN are expected values. Or use `Function` instead. Or use `1/0` or `0/0` as pseudo-literals. Not every JS value is present in the JSON data model, but it's absurd that not every JS number is present in it.

bshimmin 6y ago

Wouldn't you just end up with (for example) "Python-flavoured JSON", just like we have "GitHub-flavoured Markdown"? On the other hand, there is an RFC for JSON where there isn't for Markdown, so it ought to be possible to make use of the usual standards process to formalise a successor to the original JSON which might include some of the very common additions some people always want (trailing commas, comments, proper dates) - that this hasn't happened over the last 15 years suggests really that the problem is actually due to lack of agreement.

marcosdumay 6y ago

While we are at it, how about making the parsers accept non-ascii space, so that things don't break if somebody using Windows edit a file somewhere and adds a BOM.