6 ms·

What is the value of JSON.parse(JSON.stringify(BigInt(Number.MAX_SAFE_INTEGER) + 2n))? > Finally, BigInts cannot be serialized to JSON. Hmm, ok. I guess that'

by coke12 8y ago

What is the value of JSON.parse(JSON.stringify(BigInt(Number.MAX_SAFE_INTEGER) + 2n))? > Finally, BigInts cannot be serialized to JSON. Hmm, ok. I guess that's fine. But what about large numbers coming from over the network? Can we get a BigInt-aware JSON.parse() standardized ASAP?

olliej 8y ago
no. This is not related to parsing, it’s related to the JSON format. JSON is a stable interchange format, and for that reason we’ve got a huge base of tooling that uses and emits it. It’s all interoperable because there is just that one syntax. So if you say “add JSON.parse() API”, you’re saying “I want to transmit non-Json strings but claim they’re json “. That breaks: * old browsers - not supporting bigints would mean they couldn’t parse any of your “json” data, even if you weren’t using the bigints. Eg feature detection wouldn’t work. * all shipped products that read json, because as with the browsers they could not parse any “json” that contained invalid data. [edit: Ok, let's try to do something about the endless downvotes: Say you do JSON.parse("9007199254740994") - 2^53+2 Should this parse to a BigInt or a Number? In JavaScript today it will parse as a Number, without losing precision, but it is beyond the range of exactly representable integers in JS. You can see this by doing console.log(9007199254740995) and see that the output is 9007199254740996. So what happens if I do that following: JSON.parse("[9007199254740992, 9007199254740993, 9007199254740994, 9007199254740995]")? the first and third are exactly representable as a double, the second and forth are not. So should this parse as Number, BigInt, Number, BigInt? Or BigInt, BigInt, BigInt, BigInt? What value should be the trigger for treating an integer as a BigInt vs. a Number? What if we're doing JSON.parse("[9007199254740992, 9007199254740993, 9007199254740994, 9007199254740995]").map(x=>x+2) If any of the values get interpreted as a BigInt this will throw. But in existing browsers it won't. ]
- deathanatos 8y ago
  > It’s all interoperable because there is just that one syntax. And that syntax and specification leaves open the possibility of arbitrary precision integers. From the JSON RFC[1]: > This specification allows implementations to set limits on the range and precision of numbers accepted. Those limits vary by implementation. The subsequent paragraphs warn, of course, that not all implementations support sending or receiving numbers outside of certain ranges, and explicitly calls out the contiguous integer range of an IEEE double as a good range for good interoperability. > “I want to transmit non-Json strings but claim they’re json “. Those strings are already JSON. There are serializers out there that support arbitrary precision integers. Python, for example, will happily serialize such integers: In [6]: json.dumps(2 ** 512) Out[6]: '13407807929942597099574024998205846127479365820592393377723561443721764030073546976801874298166903427690031858186486050853753882811946569946433649006084096' (This will parse in today's browsers, too, but you'll get an imprecise result, as JavaScript's Number cannot represent that value exactly. Larger numbers result in Infinity.) [1]: https://tools.ietf.org/html/rfc8259#section-6 https://tools.ietf.org/html/rfc8259#section-6
  - coke12 8y ago
    Yup, this is exactly the problem. Currently JSON.parse() will silently truncate large numbers. This behavior made sense when there was only one numeric type in the language, but BigInt opens the door for arbitrary precision integers.
    - olliej 8y ago
      But you can't determine (without a suffix) whether a number you encounter is a BigInt or a floating point value that lost precision so doesn't have a decimal point.
    - testvox 8y ago
      What is the one language? json serialized data can be consumed in many different languages which have very different numeric data types. The json spec even says > JSON is agnostic about the semantics of numbers. In any programming language, there can be a variety of number types of various capacities and complements, fixed or floating, binary or decimal. That can make interchange between different programming languages difficult. JSON instead offers only the representation of numbers that humans use: a sequence of digits. All programming languages know how to make sense of digit sequences even if they disagree on internal representations. That is enough to allow interchange and later more precisely defines a number in a way that does not restrict its maximum size. > A number is a sequence of decimal digits with no superfluous leading zero. It may have a preceding minus sign (U+002D). It may have a fractional part prefixed by a decimal point (U+002E). It may have an exponent, prefixed by e (U+0065) or E (U+0045) and optionally + (U+002B) or – (U+002D). The digits are the code points U+0030 through U+0039. http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf http://www.ecma-international.org/publications/files/ECMA-ST...
  - olliej 8y ago
    in JS your number would drop precision, because it doesn't have the 'n' suffix. Because you can't mix and match numeric types, JSON.parse() cannot distinguish between a number, and a bigint. so take: "[{x:0,y:1},{x:2, y:3456789....}}]" What should JSON.parse() do? There's no reason for it to parse any of the values before the final extra precision value as a regular number, so if it parses the large one as a bigint, it would break. > (This will parse in today's browsers, too, but you'll get an imprecise result, as JavaScript's Number cannot represent that value exactly. Larger numbers result in Infinity.) Your example of python output is exactly the thing that would cause a problem. Your data will work in all existing browsers - it just loses precision. But in any browser that tries to parse it as a bigint it will result in failures, as now any subsequent arithmetic will fail because you can't (for good reasons) mix and match BigInt with non-BigInt values. You also can't actually make a decision about whether a value should be parsed into a BigInt, vs a Number. Basically, take something like 2^54 -- 18014398509481984 -- should this be parsed as a Number, or as a BigInt. It's exactly representable with a double, but it has the same value as 2^54+1 (which is not exactly representable). So should that be parsed as a BigInt or as a Number. Without a suffix (not compatible with existing spec) they cannot be distinguished, and so to retain backwards compatibility must be parsed as a Number.
- askmike 8y ago
  How about the backwards compatible way of: {"value": "90071992547409949007199254740994n"} Which would get parsed into a BigInt. This specifically means that value isn't a number but a bigint. All JSON consumers that don't support that get strings, all JSON consumers that do get BigInts (when parsing). The only downside I see is existing json that has strings containing a number and a "n" at the end.
  - rocqua 8y ago
    > The only downside I see is existing json that has strings containing a number and a "n" at the end. Imagine being on the receiving end of that bug!
  - thedufer 8y ago
    The other problem is that this would be a bizarre JavaScript-specific quirk, and would fail to interop with JSON implementations in other languages.
    - askmike 8y ago
      I'm not sure if I agree, I think the BigInt type as proposed does no exist in other languages.
      - thedufer 8y ago
        The JSON spec doesn't bound numbers in any way, and plenty of languages have support for arbitrarily large numbers (for example, Python's number type).
  - Spivak 8y ago
    This would require changing the JSON spec (good luck) as it doesn't fit the grammar for JSON numbers.
    - myfonj 8y ago
      Yes, it doesn't fit grammar for ordinary JSON Numbers, but since it is plain old standard JSON there is nothing wrong about it: it just puts some demand on consuming application. Instead of {"boring Number not greater than 2^53": 1234} you could send something like {"exciting huge BigInt": {"value": "12(…)89", "type": "BigInt"}} where you have all hints what to do with it. You can even invent some funny perks, like {"exciting huge BigInt serialized to base36": {"value": "12(…)yz", "type": "BigInt", "base": 36} } assuming you could create some parseBigInt(radix) function.
- BillinghamJ 8y ago
  You're getting downvotes because what you've said is incorrect. This wouldn't be changing/violating/breaking the JSON format - it already allows for arbitrary precision. However changing the behaviour of `JSON.parse` would indeed be a breaking change. That's an issue with the JS spec and web compatibility - not with JSON. The solution there is simple - either make a new function with new rules or add the ability to pass in options to `JSON.parse`.
  - olliej 8y ago
    The spec breakage is that you’d need the n suffix to indicate it’s a bigint.
    - BillinghamJ 8y ago
      No, you wouldn't. You'd just put the number in without the "n". In JSON, an integer is an integer - there's no limit to its size and no need to specify whether the integer is big or small. The "n" notation is just to ensure compatibility within JS - has nothing to do with representing the number in JSON.
- Spivak 8y ago
  This isn't about parsing the JSON format but the mapping between JSON datatypes and your language datatypes. JSON numbers are arbitrary precision, how you choose the represent those numbers in your language varies widely, and up to you. Adding hooks into JSON.parse would actually solve a whole class of problems related to datatype mappings. Right now they're statically defined by whoever implements the parse method but it turns out not to be very flexible. What if I want to represent my JSON array as an ArrayBuffer or a SortedArray? What if I want to represent my JSON numbers as strings or my libraries number type for working with currency. What if I want to map my hashes to my own HashMap? Being able to say JSON.parse({ array: SortedArray, number: BigInt }) would be very nice.
  - olliej 8y ago
    JSON.parse already supports that through the reviver parameter.
    - Spivak 8y ago
      It doesn't quite do what I'm looking for since it only allows you to transform data after it's already been mapped to a type. So if my integer can't fit into the Number type without loss of precision then my reviver function is going to get the lossy number so it's already to late to do a conversion to BigInt. I need to actually hook into the parser so I can say, "hey when you're about to parse a number pass the token to me and I'll take it from here."
sambeau 8y ago
VM143:1 Uncaught TypeError: Do not know how to serialize a BigInt at JSON.stringify (<anonymous>) at <anonymous>:1:17
mirekrusin 8y ago
Just pass your own serializer/reviver to JSON.parse/stringify to use something like `{ "$bigint": '0f0a0e...' }`.
tokyodude 8y ago
why does it matter? I can't serialize/deserialize Map, Set, Date, etc nor can I serialize/deserialize class Foo or Bar. If you want to serialize higher level concepts buold a high level lib. Don't clutter Json
dclowd9901 8y ago
I don’t see how BigInts can’t be serialized to JSON? Isn’t the “n” suffix exactly that — a serialization?
- thedufer 8y ago
  An integer suffixed with the letter "n" is not valid JSON and will not parse if you follow the spec. The "n" suffix is how you write BigInts in JavaScript, not JSON.
  - Spivak 8y ago
    How JS chooses to show the string interpretation of the datatype has nothing to do with whether you can map the type into an abstract JSON number, which you can. JSON.stringify(1n) === "1" JSON.stringify(100000000000000000000n) === "100000000000000000000"

by coke12 8y ago

olliej 8y ago

no. This is not related to parsing, it’s related to the JSON format. JSON is a stable interchange format, and for that reason we’ve got a huge base of tooling that uses and emits it. It’s all interoperable because there is just that one syntax. So if you say “add JSON.parse() API”, you’re saying “I want to transmit non-Json strings but claim they’re json “. That breaks: * old browsers - not supporting bigints would mean they couldn’t parse any of your “json” data, even if you weren’t using the bigints. Eg feature detection wouldn’t work. * all shipped products that read json, because as with the browsers they could not parse any “json” that contained invalid data. [edit: Ok, let's try to do something about the endless downvotes: Say you do JSON.parse("9007199254740994") - 2^53+2 Should this parse to a BigInt or a Number? In JavaScript today it will parse as a Number, without losing precision, but it is beyond the range of exactly representable integers in JS. You can see this by doing console.log(9007199254740995) and see that the output is 9007199254740996. So what happens if I do that following: JSON.parse("[9007199254740992, 9007199254740993, 9007199254740994, 9007199254740995]")? the first and third are exactly representable as a double, the second and forth are not. So should this parse as Number, BigInt, Number, BigInt? Or BigInt, BigInt, BigInt, BigInt? What value should be the trigger for treating an integer as a BigInt vs. a Number? What if we're doing JSON.parse("[9007199254740992, 9007199254740993, 9007199254740994, 9007199254740995]").map(x=>x+2) If any of the values get interpreted as a BigInt this will throw. But in existing browsers it won't. ]

deathanatos 8y ago
> It’s all interoperable because there is just that one syntax. And that syntax and specification leaves open the possibility of arbitrary precision integers. From the JSON RFC[1]: > This specification allows implementations to set limits on the range and precision of numbers accepted. Those limits vary by implementation. The subsequent paragraphs warn, of course, that not all implementations support sending or receiving numbers outside of certain ranges, and explicitly calls out the contiguous integer range of an IEEE double as a good range for good interoperability. > “I want to transmit non-Json strings but claim they’re json “. Those strings are already JSON. There are serializers out there that support arbitrary precision integers. Python, for example, will happily serialize such integers: In [6]: json.dumps(2 ** 512) Out[6]: '13407807929942597099574024998205846127479365820592393377723561443721764030073546976801874298166903427690031858186486050853753882811946569946433649006084096' (This will parse in today's browsers, too, but you'll get an imprecise result, as JavaScript's Number cannot represent that value exactly. Larger numbers result in Infinity.) [1]: https://tools.ietf.org/html/rfc8259#section-6 https://tools.ietf.org/html/rfc8259#section-6
- coke12 8y ago
  Yup, this is exactly the problem. Currently JSON.parse() will silently truncate large numbers. This behavior made sense when there was only one numeric type in the language, but BigInt opens the door for arbitrary precision integers.
  - olliej 8y ago
    But you can't determine (without a suffix) whether a number you encounter is a BigInt or a floating point value that lost precision so doesn't have a decimal point.
  - testvox 8y ago
    What is the one language? json serialized data can be consumed in many different languages which have very different numeric data types. The json spec even says > JSON is agnostic about the semantics of numbers. In any programming language, there can be a variety of number types of various capacities and complements, fixed or floating, binary or decimal. That can make interchange between different programming languages difficult. JSON instead offers only the representation of numbers that humans use: a sequence of digits. All programming languages know how to make sense of digit sequences even if they disagree on internal representations. That is enough to allow interchange and later more precisely defines a number in a way that does not restrict its maximum size. > A number is a sequence of decimal digits with no superfluous leading zero. It may have a preceding minus sign (U+002D). It may have a fractional part prefixed by a decimal point (U+002E). It may have an exponent, prefixed by e (U+0065) or E (U+0045) and optionally + (U+002B) or – (U+002D). The digits are the code points U+0030 through U+0039. http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf http://www.ecma-international.org/publications/files/ECMA-ST...
- olliej 8y ago
  in JS your number would drop precision, because it doesn't have the 'n' suffix. Because you can't mix and match numeric types, JSON.parse() cannot distinguish between a number, and a bigint. so take: "[{x:0,y:1},{x:2, y:3456789....}}]" What should JSON.parse() do? There's no reason for it to parse any of the values before the final extra precision value as a regular number, so if it parses the large one as a bigint, it would break. > (This will parse in today's browsers, too, but you'll get an imprecise result, as JavaScript's Number cannot represent that value exactly. Larger numbers result in Infinity.) Your example of python output is exactly the thing that would cause a problem. Your data will work in all existing browsers - it just loses precision. But in any browser that tries to parse it as a bigint it will result in failures, as now any subsequent arithmetic will fail because you can't (for good reasons) mix and match BigInt with non-BigInt values. You also can't actually make a decision about whether a value should be parsed into a BigInt, vs a Number. Basically, take something like 2^54 -- 18014398509481984 -- should this be parsed as a Number, or as a BigInt. It's exactly representable with a double, but it has the same value as 2^54+1 (which is not exactly representable). So should that be parsed as a BigInt or as a Number. Without a suffix (not compatible with existing spec) they cannot be distinguished, and so to retain backwards compatibility must be parsed as a Number.
askmike 8y ago
How about the backwards compatible way of: {"value": "90071992547409949007199254740994n"} Which would get parsed into a BigInt. This specifically means that value isn't a number but a bigint. All JSON consumers that don't support that get strings, all JSON consumers that do get BigInts (when parsing). The only downside I see is existing json that has strings containing a number and a "n" at the end.
- rocqua 8y ago
  > The only downside I see is existing json that has strings containing a number and a "n" at the end. Imagine being on the receiving end of that bug!
- thedufer 8y ago
  The other problem is that this would be a bizarre JavaScript-specific quirk, and would fail to interop with JSON implementations in other languages.
  - askmike 8y ago
    I'm not sure if I agree, I think the BigInt type as proposed does no exist in other languages.
    - thedufer 8y ago
      The JSON spec doesn't bound numbers in any way, and plenty of languages have support for arbitrarily large numbers (for example, Python's number type).
- Spivak 8y ago
  This would require changing the JSON spec (good luck) as it doesn't fit the grammar for JSON numbers.
  - myfonj 8y ago
    Yes, it doesn't fit grammar for ordinary JSON Numbers, but since it is plain old standard JSON there is nothing wrong about it: it just puts some demand on consuming application. Instead of {"boring Number not greater than 2^53": 1234} you could send something like {"exciting huge BigInt": {"value": "12(…)89", "type": "BigInt"}} where you have all hints what to do with it. You can even invent some funny perks, like {"exciting huge BigInt serialized to base36": {"value": "12(…)yz", "type": "BigInt", "base": 36} } assuming you could create some parseBigInt(radix) function.
BillinghamJ 8y ago
You're getting downvotes because what you've said is incorrect. This wouldn't be changing/violating/breaking the JSON format - it already allows for arbitrary precision. However changing the behaviour of `JSON.parse` would indeed be a breaking change. That's an issue with the JS spec and web compatibility - not with JSON. The solution there is simple - either make a new function with new rules or add the ability to pass in options to `JSON.parse`.
- olliej 8y ago
  The spec breakage is that you’d need the n suffix to indicate it’s a bigint.
  - BillinghamJ 8y ago
    No, you wouldn't. You'd just put the number in without the "n". In JSON, an integer is an integer - there's no limit to its size and no need to specify whether the integer is big or small. The "n" notation is just to ensure compatibility within JS - has nothing to do with representing the number in JSON.
Spivak 8y ago
This isn't about parsing the JSON format but the mapping between JSON datatypes and your language datatypes. JSON numbers are arbitrary precision, how you choose the represent those numbers in your language varies widely, and up to you. Adding hooks into JSON.parse would actually solve a whole class of problems related to datatype mappings. Right now they're statically defined by whoever implements the parse method but it turns out not to be very flexible. What if I want to represent my JSON array as an ArrayBuffer or a SortedArray? What if I want to represent my JSON numbers as strings or my libraries number type for working with currency. What if I want to map my hashes to my own HashMap? Being able to say JSON.parse({ array: SortedArray, number: BigInt }) would be very nice.
- olliej 8y ago
  JSON.parse already supports that through the reviver parameter.
  - Spivak 8y ago
    It doesn't quite do what I'm looking for since it only allows you to transform data after it's already been mapped to a type. So if my integer can't fit into the Number type without loss of precision then my reviver function is going to get the lossy number so it's already to late to do a conversion to BigInt. I need to actually hook into the parser so I can say, "hey when you're about to parse a number pass the token to me and I'll take it from here."

sambeau 8y ago

VM143:1 Uncaught TypeError: Do not know how to serialize a BigInt at JSON.stringify (<anonymous>) at <anonymous>:1:17

mirekrusin 8y ago

Just pass your own serializer/reviver to JSON.parse/stringify to use something like `{ "$bigint": '0f0a0e...' }`.

tokyodude 8y ago

why does it matter? I can't serialize/deserialize Map, Set, Date, etc nor can I serialize/deserialize class Foo or Bar. If you want to serialize higher level concepts buold a high level lib. Don't clutter Json

dclowd9901 8y ago

I don’t see how BigInts can’t be serialized to JSON? Isn’t the “n” suffix exactly that — a serialization?

thedufer 8y ago
An integer suffixed with the letter "n" is not valid JSON and will not parse if you follow the spec. The "n" suffix is how you write BigInts in JavaScript, not JSON.
- Spivak 8y ago
  How JS chooses to show the string interpretation of the datatype has nothing to do with whether you can map the type into an abstract JSON number, which you can. JSON.stringify(1n) === "1" JSON.stringify(100000000000000000000n) === "100000000000000000000"