9 ms·

"Fortunately we have JSON now." Which doesn't support big ints. JSON isn't a silver bullet.

NelsonMinar 12y ago
Fair enough. Then again XML doesn't support any kind of int and yet we managed. (Unless you count XML Schema, in which case now you have N problems.)
- 1ris 12y ago
  As a Peano-player I'd represent the interger 3 in xml as <succ> <succ> <succ> <zero></zero> </succ> </succ> </succ>
  - raverbashing 12y ago
    Don't give them ideas...
- the_why_of_y 12y ago
  Actually they split things up nicely so you can use XMLSchema part 2 (which contains the useful data types) without part 1 (which contains the N problems you mention).
mike_hearn 12y ago
So far the only serialization format I've found which isn't stuffed with surprising quirks is protobufs.
- cbsmith 12y ago
  One could make an argument that its zigzag format is mildly quirky, but in general, I'd agree. Protobufs have some surprising limitations, but their simplicity has limited their quirkiness.
  - mike_hearn 12y ago
    That's true. There are some annoying limitations, though usually they're easy to work around at the app layer (e.g. no flexible type system).
benjaminjackman 12y ago
The common approach I have seen/used is to encode big ints as a string instead. The lack of support for NaN and +/-Infininty means that doubles won't encode in JSON either. On other trick is that because comments aren't supported is to make a keys named "//SomeComment":"", this will still be syntactically correct json and you can simply ignore them inside your program. JSON is missing a lot of little things, but once you add them in you end up with something that is a lot harder to parse, which ultimately hurts the ubiquitous appeal of json.
- schoen 12y ago
  The JSON parser in the Python standard library does support big ints, NaN, and +/-Infinity. >>> json.loads("NaN") nan >>> json.loads("Infinity") inf >>> json.loads("-Infinity") -inf >>> json.loads("1234567890"*20) 123456789012345678901234567890123456789012345678901234567890 123456789012345678901234567890123456789012345678901234567890 123456789012345678901234567890123456789012345678901234567890 12345678901234567890L I haven't read the JSON spec, so I'm wondering whether this is actually standard JSON, whether it's an official extension of some kind to JSON, or whether the Python JSON parser is just being too clever.
  - comex 12y ago
    Not standard. > Numeric values that cannot be represented as sequences of digits (such as Infinity and NaN) are not permitted. - http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf http://www.ecma-international.org/publications/files/ECMA-ST...
    - schoen 12y ago
      Huh, the Python json module documentation admits that the Python behavior isn't standards-compliant. https://docs.python.org/2/library/json.html#standard-compliance-and-interoperability https://docs.python.org/2/library/json.html#standard-complia... It does claim that the support for arbitrary precision numbers is standards-compliant. Looking at the ECMA standard briefly, that seems to be correct to me: the official JSON grammar does allow arbitrary-precision, and disclaims restrictions on how a particular programming language interprets numbers. I have to admit that I'm kind of concerned about the NaN and Infinity issue because I thought there was more uniformity among JSON parsers about what is or isn't legitimate JSON.
contingencies 12y ago
Ahh, grasshopper. Truth is, there is more than one "truth".
- contingencies 12y ago
  Meh, since so many downvotes, that was a smart-ass way of saying "put the integer in quotes so it transfers".
kazinator 12y ago
Sure it does: { "mybigint" : -123434580239458203948203982345723458 } You can throw the spec out the window and put as many digits as you want into a JSON integer. Any half decent parser in a half decent language will accumulate the token and spit out an integer object. There is no reason to write a JSON parser that doesn't accept bignum integers, in a language that has them. There are only two reasons some JSON implementation doesn't accept them. One is that the underlying software doesn't handle them. In that case JSON is moot; it's a limitation of that system. You cannot input bignums into that system through any means; they are not representable. The other is that the JSON implementation was written by obtuse blockheads: bignums could easily be supported since the underlying language has them, but aren't simply for compliance with the JSON specification. Compliance can be taken to counterproductive extremes.
- xxgreg 12y ago
  For example: import 'dart:convert'; main() { print(JSON.decode('{ "mybigint" : -123434580239458203948203982345723458 }')['mybigint']); } Prints -123434580239458203948203982345723458
  - dalke 12y ago
    What does it do for large floats? In Python: >>> import json >>> json.loads('{ "mybigfloat" : -1' + "8"*310 + '.0}') {u'mybigfloat': -inf}
- cbsmith 12y ago
  > There is no reason to write a JSON parser that doesn't accept bignum integers, in a language that has them. Couldn't you write that about about almost any data type in JSON? The entire point of having a standard for such things is so that you don't have to worry about the differences between individual parsers/serializers.
  - thaumasiotes 12y ago
    > The entire point of having a standard for such things is so that you don't have to worry about the differences between individual parsers/serializers. That makes this an interesting case. As pointed out elsewhere, adamtulinius is randomly passing on an easily-falsified urban legend; the JSON standard explicitly allows bignums. That doesn't leave much room for not having to worry about the differences between individual parsers; if the language doesn't allow bignums, it's not going to parse them no matter how much it wants to be standards-compliant.
colanderman 12y ago
Where in the spec does it say that? http://json.org/ http://json.org/ I see number, which can be an int, which can be digit1-9 digits, and digits is digit digits. No size limitation there. In fact numbers have no semantic meaning applied to them in JSON. Probably the only reason to include them is to standardize their representation as a debugging aid.
thaumasiotes 12y ago
I can't resist pointing out for a third time what colanderman and schoen have already said: JSON supports bigints exactly the same as small ints. Where did you get the idea that it didn't?

Fair enough. Then again XML doesn't support any kind of int and yet we managed. (Unless you count XML Schema, in which case now you have N problems.)

1ris 12y ago
As a Peano-player I'd represent the interger 3 in xml as <succ> <succ> <succ> <zero></zero> </succ> </succ> </succ>
- raverbashing 12y ago
  Don't give them ideas...
the_why_of_y 12y ago
Actually they split things up nicely so you can use XMLSchema part 2 (which contains the useful data types) without part 1 (which contains the N problems you mention).

mike_hearn 12y ago

So far the only serialization format I've found which isn't stuffed with surprising quirks is protobufs.

cbsmith 12y ago
One could make an argument that its zigzag format is mildly quirky, but in general, I'd agree. Protobufs have some surprising limitations, but their simplicity has limited their quirkiness.
- mike_hearn 12y ago
  That's true. There are some annoying limitations, though usually they're easy to work around at the app layer (e.g. no flexible type system).

benjaminjackman 12y ago

The common approach I have seen/used is to encode big ints as a string instead. The lack of support for NaN and +/-Infininty means that doubles won't encode in JSON either. On other trick is that because comments aren't supported is to make a keys named "//SomeComment":"", this will still be syntactically correct json and you can simply ignore them inside your program. JSON is missing a lot of little things, but once you add them in you end up with something that is a lot harder to parse, which ultimately hurts the ubiquitous appeal of json.

schoen 12y ago
The JSON parser in the Python standard library does support big ints, NaN, and +/-Infinity. >>> json.loads("NaN") nan >>> json.loads("Infinity") inf >>> json.loads("-Infinity") -inf >>> json.loads("1234567890"*20) 123456789012345678901234567890123456789012345678901234567890 123456789012345678901234567890123456789012345678901234567890 123456789012345678901234567890123456789012345678901234567890 12345678901234567890L I haven't read the JSON spec, so I'm wondering whether this is actually standard JSON, whether it's an official extension of some kind to JSON, or whether the Python JSON parser is just being too clever.
- comex 12y ago
  Not standard. > Numeric values that cannot be represented as sequences of digits (such as Infinity and NaN) are not permitted. - http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf http://www.ecma-international.org/publications/files/ECMA-ST...
  - schoen 12y ago
    Huh, the Python json module documentation admits that the Python behavior isn't standards-compliant. https://docs.python.org/2/library/json.html#standard-compliance-and-interoperability https://docs.python.org/2/library/json.html#standard-complia... It does claim that the support for arbitrary precision numbers is standards-compliant. Looking at the ECMA standard briefly, that seems to be correct to me: the official JSON grammar does allow arbitrary-precision, and disclaims restrictions on how a particular programming language interprets numbers. I have to admit that I'm kind of concerned about the NaN and Infinity issue because I thought there was more uniformity among JSON parsers about what is or isn't legitimate JSON.

contingencies 12y ago

Ahh, grasshopper. Truth is, there is more than one "truth".

contingencies 12y ago
Meh, since so many downvotes, that was a smart-ass way of saying "put the integer in quotes so it transfers".

kazinator 12y ago

Sure it does: { "mybigint" : -123434580239458203948203982345723458 } You can throw the spec out the window and put as many digits as you want into a JSON integer. Any half decent parser in a half decent language will accumulate the token and spit out an integer object. There is no reason to write a JSON parser that doesn't accept bignum integers, in a language that has them. There are only two reasons some JSON implementation doesn't accept them. One is that the underlying software doesn't handle them. In that case JSON is moot; it's a limitation of that system. You cannot input bignums into that system through any means; they are not representable. The other is that the JSON implementation was written by obtuse blockheads: bignums could easily be supported since the underlying language has them, but aren't simply for compliance with the JSON specification. Compliance can be taken to counterproductive extremes.

xxgreg 12y ago
For example: import 'dart:convert'; main() { print(JSON.decode('{ "mybigint" : -123434580239458203948203982345723458 }')['mybigint']); } Prints -123434580239458203948203982345723458
- dalke 12y ago
  What does it do for large floats? In Python: >>> import json >>> json.loads('{ "mybigfloat" : -1' + "8"*310 + '.0}') {u'mybigfloat': -inf}
cbsmith 12y ago
> There is no reason to write a JSON parser that doesn't accept bignum integers, in a language that has them. Couldn't you write that about about almost any data type in JSON? The entire point of having a standard for such things is so that you don't have to worry about the differences between individual parsers/serializers.
- thaumasiotes 12y ago
  > The entire point of having a standard for such things is so that you don't have to worry about the differences between individual parsers/serializers. That makes this an interesting case. As pointed out elsewhere, adamtulinius is randomly passing on an easily-falsified urban legend; the JSON standard explicitly allows bignums. That doesn't leave much room for not having to worry about the differences between individual parsers; if the language doesn't allow bignums, it's not going to parse them no matter how much it wants to be standards-compliant.

colanderman 12y ago

Where in the spec does it say that? http://json.org/ http://json.org/ I see number, which can be an int, which can be digit1-9 digits, and digits is digit digits. No size limitation there. In fact numbers have no semantic meaning applied to them in JSON. Probably the only reason to include them is to standardize their representation as a debugging aid.

thaumasiotes 12y ago

I can't resist pointing out for a third time what colanderman and schoen have already said: JSON supports bigints exactly the same as small ints. Where did you get the idea that it didn't?