4 ms·

What about the 2^63 corner-case? Consider this JSON: {"key": 9223372036854775807}. With most parsers it never fails. But... some JSON parsers (include JS.eval

What about the 2^63 corner-case? Consider this JSON: {"key": 9223372036854775807}. With most parsers it never fails. But... some JSON parsers (include JS.eval) parse it to 9223372036854776000 and continue on their merry way. The problem isn't user-provided JSON here. The problem is user-provided data (or computer-provided data) that's inside the JSON. rachelbythebay's take (http://rachelbythebay.com/w/2019/07/21/reliability/ http://rachelbythebay.com/w/2019/07/21/reliability/): On the other hand, if you only need 53 bits of your 64 bit numbers, and enjoy blowing CPU on ridiculously inefficient marshaling and unmarshaling steps, hey, it's your funeral.

shawnz 7y ago
In what situation would that create a problem that isn't noticed immediately during testing?
- juliusmusseau 7y ago
  When the tests fail to 1.) experiment with numeric values greater than 2^53 or less than -2^53 and 2.) fail to carefully check the results even in the situations where they did experiment with such values.
  - shawnz 7y ago
    How often do you encounter situations where your values start off below 2^53 and then later grow above 2^53? How often do you deal with integers greater than 53 bits where the app doesn't immediately fail when the exact value is not correct (i.e. unique IDs or crypto keys)? I feel like these are edge cases which would rarely come into play in most software.
    - juliusmusseau 7y ago
      Here's some stuff where it would bite me in my work: - I sometimes use random 64 bit integers in my data (usually as cheaper synthetic guid-like keys). - I sometimes use CRC-64 hashes. - I sometimes use 2^63 as a sentinel.
    - mikeash 7y ago
      Edge cases which rarely come into play are the ones you should be most afraid of. Things that happen often get caught before they cause trouble. Things that never happen are fine. Things that happen rarely are what get you. For an made-up example of how this could get you, let’s imagine that you have a message service that gives each message a unique ID. Some bright soul decided to give this ID a nice structure and make it a 64-bit value where the top 32 bits are an incrementing integer per user, and the bottom 32 bits are the user’s ID, assigned by with a global incrementing integer. Everything works great in testing and you deploy and the VC money is rolling in and then some of your very prolific users go past 2 million messages and suddenly messages are getting mixed up and you’re leaking private info because your access checking code happens to get the real 64-bit value but your message retrieval code puts the ID in a JSON number. Now you might respond, but that ID scheme is dumb, don’t do that. And you may be right! But dumb things happen. It’s unwise to leave land mines lying around in your software just because they only detonate when someone does something dumb.
- overgard 7y ago
  You're writing the server for a procedural space game where the coordinates are stored as numbers. As the ships get far from (0,0,0) people start to report that the graphics get "jumpy" and gameplay rules break frequently. Or, your server keeps track of how long it's been running, in nanoseconds. When the server stays up for months, people start noticing weird precision issues and hard to replicate bugs around time.
jacobolus 7y ago
> some JSON parsers (include JS.eval) parse it to 9223372036854776000 and continue on their merry way This is correct behavior though...? Every number in JSON is implicitly a double-precision float. JSON doesn’t distinguish other number types. If you want that big a string of digits in JSON, put it in a string. Edit: let me make a more precise statement since several people seem to have a problem with the one above: Every number that you send to a typical JavaScript JSON parser is implicitly a double-precision float, and it is correct behavior for a JavaScript JSON parser to treat a long string of digits as a double-precision float, even if that results in lost precision. The JSON specification itself punts on the precise semantic meaning of numbers, leaving it up to producers and consumers of the JSON to coordinate their number interpretation.
- justincormack 7y ago
  Every number in JavaScript is, JSON does not specify.
  - jacobolus 7y ago
    “JavaScript Object” is right there in the name.
    - overgard 7y ago
      Java is also in the name JavaScript, but we know how much that has to do with it.
      - TheRealPomax 7y ago
        indeed, we've all read that history. And we all know how much JavasSript has to do with "JavaScript Object Notation", too right? Basically everything?
        overgard 7y ago
        Other than shared syntax, JSON is its own thing.

shawnz 7y ago

In what situation would that create a problem that isn't noticed immediately during testing?

juliusmusseau 7y ago
When the tests fail to 1.) experiment with numeric values greater than 2^53 or less than -2^53 and 2.) fail to carefully check the results even in the situations where they did experiment with such values.
- shawnz 7y ago
  How often do you encounter situations where your values start off below 2^53 and then later grow above 2^53? How often do you deal with integers greater than 53 bits where the app doesn't immediately fail when the exact value is not correct (i.e. unique IDs or crypto keys)? I feel like these are edge cases which would rarely come into play in most software.
  - juliusmusseau 7y ago
    Here's some stuff where it would bite me in my work: - I sometimes use random 64 bit integers in my data (usually as cheaper synthetic guid-like keys). - I sometimes use CRC-64 hashes. - I sometimes use 2^63 as a sentinel.
  - mikeash 7y ago
    Edge cases which rarely come into play are the ones you should be most afraid of. Things that happen often get caught before they cause trouble. Things that never happen are fine. Things that happen rarely are what get you. For an made-up example of how this could get you, let’s imagine that you have a message service that gives each message a unique ID. Some bright soul decided to give this ID a nice structure and make it a 64-bit value where the top 32 bits are an incrementing integer per user, and the bottom 32 bits are the user’s ID, assigned by with a global incrementing integer. Everything works great in testing and you deploy and the VC money is rolling in and then some of your very prolific users go past 2 million messages and suddenly messages are getting mixed up and you’re leaking private info because your access checking code happens to get the real 64-bit value but your message retrieval code puts the ID in a JSON number. Now you might respond, but that ID scheme is dumb, don’t do that. And you may be right! But dumb things happen. It’s unwise to leave land mines lying around in your software just because they only detonate when someone does something dumb.
overgard 7y ago
You're writing the server for a procedural space game where the coordinates are stored as numbers. As the ships get far from (0,0,0) people start to report that the graphics get "jumpy" and gameplay rules break frequently. Or, your server keeps track of how long it's been running, in nanoseconds. When the server stays up for months, people start noticing weird precision issues and hard to replicate bugs around time.

jacobolus 7y ago

> some JSON parsers (include JS.eval) parse it to 9223372036854776000 and continue on their merry way This is correct behavior though...? Every number in JSON is implicitly a double-precision float. JSON doesn’t distinguish other number types. If you want that big a string of digits in JSON, put it in a string. Edit: let me make a more precise statement since several people seem to have a problem with the one above: Every number that you send to a typical JavaScript JSON parser is implicitly a double-precision float, and it is correct behavior for a JavaScript JSON parser to treat a long string of digits as a double-precision float, even if that results in lost precision. The JSON specification itself punts on the precise semantic meaning of numbers, leaving it up to producers and consumers of the JSON to coordinate their number interpretation.

justincormack 7y ago
Every number in JavaScript is, JSON does not specify.
- jacobolus 7y ago
  “JavaScript Object” is right there in the name.
  - overgard 7y ago
    Java is also in the name JavaScript, but we know how much that has to do with it.
    - TheRealPomax 7y ago
      indeed, we've all read that history. And we all know how much JavasSript has to do with "JavaScript Object Notation", too right? Basically everything?
      - overgard 7y ago
        Other than shared syntax, JSON is its own thing.