3 ms·

I've mentioned it before, but JSON Numbers are arbitrary-precision floating-point. They can't be naively stored as JavaScript numbers in all cases, since JS num

I've mentioned it before, but JSON Numbers are arbitrary-precision floating-point. They can't be naively stored as JavaScript numbers in all cases, since JS numbers are IEEE754 double-precision floating point. Likewise, you can't store arbitrary JS number values in JSON Numbers, since it doesn't handle IEEE754 exception values (qNaN, sNaN, ±INF). So if you're trying to publish data from a GPS chip that reports fix failure by setting the latitude and longitude to a NaN (possibly with a failure code in the exception value) you can handle that in JS but only through some intermediate representation in JSON.

deleted 6y ago
[deleted]
vlovich123 6y ago
Honestly I think it's the biggest misfeature of JSON. Arbitrary precision numbers should have been handled differently & I've repeatedly seen JSON parsers incorrectly handle those. We're now in a situation where you can't serialize numbers you encounter daily into JSON and you can't trust that numbers you correctly serialized into JSON will be properly deserialized by the peer. I spent a lot of time writing the number handling code in PBNJSON for WebOS back in the day (IIRC there were 0 C/C++ parsers that offered correct number parsing at the time). It also makes sure to properly report overflow/underflow/failure to convert whenever you encounter situations that aren't possible (e.g. reading 32-bit number of a 64-bit int JSON, reading 64-bit integer for doubles that are out of range, etc). I'm sure I missed some corner cases though as it's insanely difficult to get every single corner case (especially with floating point).
mormegil 6y ago
> I've mentioned it before, but JSON Numbers are arbitrary-precision floating-point. They can't be naively stored as JavaScript numbers in all cases, since JS numbers are IEEE754 double-precision floating point. Well, RFC 8259 says "This specification allows implementations to set limits on the range and precision of numbers accepted." and goes on to explain interoperability with IEEE doubles.
- e12e 6y ago
  Which implementation though? If you're only talking to yourself - you don't really need to (de)serialize to/from json - in many (most) use cases the json serializer will be different from the deserializer, and the data structure will be in a different memory model and/or runtime... And I don't think there's a standard way to tag a json document hinting that 32bits is enough, or here comes a 128 bit float?
  - mormegil 6y ago
    Well, that's the point, isn't it. The RFC basically says "while the JSON grammar allows for arbitrary-precision numbers, don't expect anything larger or more precise than a double, unless you know the recipient uses a more capable implementation". (In other words, JSON numbers are _not_ arbitrary-precision in practice.)
dan-robertson 6y ago
This is true in theory but not in practice. Most implementations don’t want to pay the cost of converting everything to bignums (or the convenience (and possibly speed) cost of leaving the numbers as strings). Generally the json-parsing is separate from the rest of the deserialisation so there’s no great way for the parser to know when it sees a number if it should convert to float or int or do something clever. As soon as you have something in your pipeline that converts to floats, it is poisoned. Lots of implementations you may use do this conversation though (e.g. basically anything JavaScript, jq) so it is pretty easy to get float poisoning. The actual solution to representing big bums in practice is to wrap them in double quotes. It only costs two bytes (and if you care about those bytes then you should probably not be using json.
- thayne 6y ago
  > It only costs two bytes And having to have special code to handle getting it as a string instead of a number.
  - Spivak 6y ago
    But that would be the case anyway because we’re talking about ways to handle numbers that don’t fit in floats. If your JSON has to be put through a black-box pipeline where you don’t know if it will be reencoded at some point by a lossy parser then the only safe thing to do is pass as a string.

deleted 6y ago

[deleted]

vlovich123 6y ago

Honestly I think it's the biggest misfeature of JSON. Arbitrary precision numbers should have been handled differently & I've repeatedly seen JSON parsers incorrectly handle those. We're now in a situation where you can't serialize numbers you encounter daily into JSON and you can't trust that numbers you correctly serialized into JSON will be properly deserialized by the peer. I spent a lot of time writing the number handling code in PBNJSON for WebOS back in the day (IIRC there were 0 C/C++ parsers that offered correct number parsing at the time). It also makes sure to properly report overflow/underflow/failure to convert whenever you encounter situations that aren't possible (e.g. reading 32-bit number of a 64-bit int JSON, reading 64-bit integer for doubles that are out of range, etc). I'm sure I missed some corner cases though as it's insanely difficult to get every single corner case (especially with floating point).

mormegil 6y ago

> I've mentioned it before, but JSON Numbers are arbitrary-precision floating-point. They can't be naively stored as JavaScript numbers in all cases, since JS numbers are IEEE754 double-precision floating point. Well, RFC 8259 says "This specification allows implementations to set limits on the range and precision of numbers accepted." and goes on to explain interoperability with IEEE doubles.

e12e 6y ago
Which implementation though? If you're only talking to yourself - you don't really need to (de)serialize to/from json - in many (most) use cases the json serializer will be different from the deserializer, and the data structure will be in a different memory model and/or runtime... And I don't think there's a standard way to tag a json document hinting that 32bits is enough, or here comes a 128 bit float?
- mormegil 6y ago
  Well, that's the point, isn't it. The RFC basically says "while the JSON grammar allows for arbitrary-precision numbers, don't expect anything larger or more precise than a double, unless you know the recipient uses a more capable implementation". (In other words, JSON numbers are _not_ arbitrary-precision in practice.)

dan-robertson 6y ago

This is true in theory but not in practice. Most implementations don’t want to pay the cost of converting everything to bignums (or the convenience (and possibly speed) cost of leaving the numbers as strings). Generally the json-parsing is separate from the rest of the deserialisation so there’s no great way for the parser to know when it sees a number if it should convert to float or int or do something clever. As soon as you have something in your pipeline that converts to floats, it is poisoned. Lots of implementations you may use do this conversation though (e.g. basically anything JavaScript, jq) so it is pretty easy to get float poisoning. The actual solution to representing big bums in practice is to wrap them in double quotes. It only costs two bytes (and if you care about those bytes then you should probably not be using json.

thayne 6y ago
> It only costs two bytes And having to have special code to handle getting it as a string instead of a number.
- Spivak 6y ago
  But that would be the case anyway because we’re talking about ways to handle numbers that don’t fit in floats. If your JSON has to be put through a black-box pipeline where you don’t know if it will be reencoded at some point by a lossy parser then the only safe thing to do is pass as a string.