5 ms·

I do not understand where the 64bit integers not working with JSON comes from. JSON the format had no limit on integer size. And all Java JSON libraries I know

by gmokki 2y ago

I do not understand where the 64bit integers not working with JSON comes from. JSON the format had no limit on integer size. And all Java JSON libraries I know can handle arbitrary prevsion integers (BigInt) and 32/64bit int/long types when serializing and deserializing. Quick googling shows that also JavaScript has proper support for 64bit integers with their BigInt type, and it can be used to deserialize incoming data correctly with the default parser, albeit requiring a bit more work to annotate the fields. Personally I often explicitly make sure that the integers I return in trust environments as identifiers in REST APIs are by default longer than 52bits so that buggy parser libraries are caught early.

GrayShade 2y ago
Qt refused for almost a decade to support deserializing 64-bit integers from JSON because of compatibility concerns.
- Spivak 2y ago
  jq couldn't handle them until relatively recently. This isn't a few bad parsers. You can't assume a json parser will handle bigints correctly and when you're angling to be low-level plumbing and work with every language and software that hasn't been recompiled in years you have to be pretty conservative in what you send.
  - capitainenemo 2y ago
    Yeah, and it isn't like jq is being incorrect in this, the JSON RFCs recommend not using values that can't be represented as a 64 bit float (pasted links to a few RFCs in another response). So if you want to represent a large number safely, better to put it in a string, where special handling can be done after the initial JSON parse without loss of information.
    - GrayShade 2y ago
      No, it doesn't. It's not a SHOULD. And why should Qt be concerned with what jq can or can't do? If you're writing and reading back from Qt, the limitations of other implementations don't apply.
capitainenemo 2y ago
The number type in JSON is 64 bit float, limiting integers without loss of precision to 2⁵³-1. BigInt is a new concept and not technically supported. So whether it works in your random library of choice is probably a potshoot. "Use within JSON: Using JSON.stringify() with any BigInt value will raise a TypeError, as BigInt values aren't serialized in JSON by default. " https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt#use_within_json https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
- IshKebab 2y ago
  That's not true. The number type in JSON is an arbitrary sized number. You're thinking of JavaScript which is not the same thing.
  - capitainenemo 2y ago
    https://datatracker.ietf.org/doc/html/rfc7493#section-2.2 https://datatracker.ietf.org/doc/html/rfc7493#section-2.2 No. (which expands on the slightly fluffier text in https://datatracker.ietf.org/doc/html/rfc7159#section-6 https://datatracker.ietf.org/doc/html/rfc7159#section-6 which broadly says the same thing... and https://datatracker.ietf.org/doc/html/rfc8259#section-6 https://datatracker.ietf.org/doc/html/rfc8259#section-6) If you look at a parser matrix, flouting this rule is asking for trouble. Which is why the recommendations on MDN for bigint recommend an explicit parsing from a string as needed. (I'm genuinely curious why this is being downvoted. Does someone out there feel so strongly that because an arbitrary string of characters technically can be parsed as any infinitely large number, that it's wise to ignore RFCs and dozens of existing implementations when making use of JSON as a common interchange format?)
    - skissane 2y ago
      You are mixing up two different standards here, JSON and I-JSON. I-JSON is a subset of JSON with stricter requirements. JSON recommends (for interoperability) that numbers be limited to those representable as IEEE 64-bit binary floats-but does not require that. A document which ignores that recommendation is still officially valid JSON. By contrast, I-JSON, as a subset of JSON, upgrades that recommendation to a requirement, so documents which ignore it are not valid I-JSON
theamk 2y ago
If you fully work within a trusted environment, why bother with JSON? Use your favorite binary serialization with codegen. The whole point of JSON is almost every programming language can read and write it - and if you want this to be the case, stringify anything unusual, like large integers.

by gmokki 2y ago

GrayShade 2y ago

Qt refused for almost a decade to support deserializing 64-bit integers from JSON because of compatibility concerns.

Spivak 2y ago
jq couldn't handle them until relatively recently. This isn't a few bad parsers. You can't assume a json parser will handle bigints correctly and when you're angling to be low-level plumbing and work with every language and software that hasn't been recompiled in years you have to be pretty conservative in what you send.
- capitainenemo 2y ago
  Yeah, and it isn't like jq is being incorrect in this, the JSON RFCs recommend not using values that can't be represented as a 64 bit float (pasted links to a few RFCs in another response). So if you want to represent a large number safely, better to put it in a string, where special handling can be done after the initial JSON parse without loss of information.
  - GrayShade 2y ago
    No, it doesn't. It's not a SHOULD. And why should Qt be concerned with what jq can or can't do? If you're writing and reading back from Qt, the limitations of other implementations don't apply.

capitainenemo 2y ago

The number type in JSON is 64 bit float, limiting integers without loss of precision to 2⁵³-1. BigInt is a new concept and not technically supported. So whether it works in your random library of choice is probably a potshoot. "Use within JSON: Using JSON.stringify() with any BigInt value will raise a TypeError, as BigInt values aren't serialized in JSON by default. " https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt#use_within_json https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

IshKebab 2y ago
That's not true. The number type in JSON is an arbitrary sized number. You're thinking of JavaScript which is not the same thing.
- capitainenemo 2y ago
  https://datatracker.ietf.org/doc/html/rfc7493#section-2.2 https://datatracker.ietf.org/doc/html/rfc7493#section-2.2 No. (which expands on the slightly fluffier text in https://datatracker.ietf.org/doc/html/rfc7159#section-6 https://datatracker.ietf.org/doc/html/rfc7159#section-6 which broadly says the same thing... and https://datatracker.ietf.org/doc/html/rfc8259#section-6 https://datatracker.ietf.org/doc/html/rfc8259#section-6) If you look at a parser matrix, flouting this rule is asking for trouble. Which is why the recommendations on MDN for bigint recommend an explicit parsing from a string as needed. (I'm genuinely curious why this is being downvoted. Does someone out there feel so strongly that because an arbitrary string of characters technically can be parsed as any infinitely large number, that it's wise to ignore RFCs and dozens of existing implementations when making use of JSON as a common interchange format?)
  - skissane 2y ago
    You are mixing up two different standards here, JSON and I-JSON. I-JSON is a subset of JSON with stricter requirements. JSON recommends (for interoperability) that numbers be limited to those representable as IEEE 64-bit binary floats-but does not require that. A document which ignores that recommendation is still officially valid JSON. By contrast, I-JSON, as a subset of JSON, upgrades that recommendation to a requirement, so documents which ignore it are not valid I-JSON

theamk 2y ago

If you fully work within a trusted environment, why bother with JSON? Use your favorite binary serialization with codegen. The whole point of JSON is almost every programming language can read and write it - and if you want this to be the case, stringify anything unusual, like large integers.