4 ms·

I'm fine with a parser that doesn't get all of the corner cases as long as it fails gracefully. Really, the only time it would matter is if you are parsing use

by umvi 7y ago

I'm fine with a parser that doesn't get all of the corner cases as long as it fails gracefully. Really, the only time it would matter is if you are parsing user-provided JSON and said user was trying to exploit your parser somehow. But 99% of the time, I'm not parsing user-provided JSON, so I don't ever encounter these corner cases and parsing/serialization works great.

majewsky 7y ago
> 99% of the time, I'm not parsing user-provided JSON I take it you've never implemented a service with a REST API.
- umvi 7y ago
  I have, but I work
  - umvi 7y ago
    Oops, somehow the second half of the comment got truncated. I meant to say: "I have, but I work mainly with embedded systems, and so there aren't usually POST or PUT APIs, just GET. So JSON is nearly always generated by the device and consumed by whoever is querying the device, not the other way around."
- deleted 7y ago
  [deleted]
juliusmusseau 7y ago
What about the 2^63 corner-case? Consider this JSON: {"key": 9223372036854775807}. With most parsers it never fails. But... some JSON parsers (include JS.eval) parse it to 9223372036854776000 and continue on their merry way. The problem isn't user-provided JSON here. The problem is user-provided data (or computer-provided data) that's inside the JSON. rachelbythebay's take (http://rachelbythebay.com/w/2019/07/21/reliability/ http://rachelbythebay.com/w/2019/07/21/reliability/): On the other hand, if you only need 53 bits of your 64 bit numbers, and enjoy blowing CPU on ridiculously inefficient marshaling and unmarshaling steps, hey, it's your funeral.
- shawnz 7y ago
  In what situation would that create a problem that isn't noticed immediately during testing?
  - juliusmusseau 7y ago
    When the tests fail to 1.) experiment with numeric values greater than 2^53 or less than -2^53 and 2.) fail to carefully check the results even in the situations where they did experiment with such values.
    - shawnz 7y ago
      How often do you encounter situations where your values start off below 2^53 and then later grow above 2^53? How often do you deal with integers greater than 53 bits where the app doesn't immediately fail when the exact value is not correct (i.e. unique IDs or crypto keys)? I feel like these are edge cases which would rarely come into play in most software.
      - juliusmusseau 7y ago
        Here's some stuff where it would bite me in my work: - I sometimes use random 64 bit integers in my data (usually as cheaper synthetic guid-like keys). - I sometimes use CRC-64 hashes. - I sometimes use 2^63 as a sentinel.
      - mikeash 7y ago
        Edge cases which rarely come into play are the ones you should be most afraid of. Things that happen often get caught before they cause trouble. Things that never happen are fine. Things that happen rarely are what get you. For an made-up example of how this could get you, let’s imagine that you have a message service that gives each message a unique ID. Some bright soul decided to give this ID a nice structure and make it a 64-bit value where the top 32 bits are an incrementing integer per user, and the bottom 32 bits are the user’s ID, assigned by with a global incrementing integer. Everything works great in testing and you deploy and the VC money is rolling in and then some of your very prolific users go past 2 million messages and suddenly messages are getting mixed up and you’re leaking private info because your access checking code happens to get the real 64-bit value but your message retrieval code puts the ID in a JSON number. Now you might respond, but that ID scheme is dumb, don’t do that. And you may be right! But dumb things happen. It’s unwise to leave land mines lying around in your software just because they only detonate when someone does something dumb.

by umvi 7y ago

majewsky 7y ago

> 99% of the time, I'm not parsing user-provided JSON I take it you've never implemented a service with a REST API.

umvi 7y ago
I have, but I work
- umvi 7y ago
  Oops, somehow the second half of the comment got truncated. I meant to say: "I have, but I work mainly with embedded systems, and so there aren't usually POST or PUT APIs, just GET. So JSON is nearly always generated by the device and consumed by whoever is querying the device, not the other way around."
deleted 7y ago
[deleted]

juliusmusseau 7y ago

What about the 2^63 corner-case? Consider this JSON: {"key": 9223372036854775807}. With most parsers it never fails. But... some JSON parsers (include JS.eval) parse it to 9223372036854776000 and continue on their merry way. The problem isn't user-provided JSON here. The problem is user-provided data (or computer-provided data) that's inside the JSON. rachelbythebay's take (http://rachelbythebay.com/w/2019/07/21/reliability/ http://rachelbythebay.com/w/2019/07/21/reliability/): On the other hand, if you only need 53 bits of your 64 bit numbers, and enjoy blowing CPU on ridiculously inefficient marshaling and unmarshaling steps, hey, it's your funeral.

shawnz 7y ago
In what situation would that create a problem that isn't noticed immediately during testing?
- juliusmusseau 7y ago
  When the tests fail to 1.) experiment with numeric values greater than 2^53 or less than -2^53 and 2.) fail to carefully check the results even in the situations where they did experiment with such values.
  - shawnz 7y ago
    How often do you encounter situations where your values start off below 2^53 and then later grow above 2^53? How often do you deal with integers greater than 53 bits where the app doesn't immediately fail when the exact value is not correct (i.e. unique IDs or crypto keys)? I feel like these are edge cases which would rarely come into play in most software.
    - juliusmusseau 7y ago
      Here's some stuff where it would bite me in my work: - I sometimes use random 64 bit integers in my data (usually as cheaper synthetic guid-like keys). - I sometimes use CRC-64 hashes. - I sometimes use 2^63 as a sentinel.
    - mikeash 7y ago
      Edge cases which rarely come into play are the ones you should be most afraid of. Things that happen often get caught before they cause trouble. Things that never happen are fine. Things that happen rarely are what get you. For an made-up example of how this could get you, let’s imagine that you have a message service that gives each message a unique ID. Some bright soul decided to give this ID a nice structure and make it a 64-bit value where the top 32 bits are an incrementing integer per user, and the bottom 32 bits are the user’s ID, assigned by with a global incrementing integer. Everything works great in testing and you deploy and the VC money is rolling in and then some of your very prolific users go past 2 million messages and suddenly messages are getting mixed up and you’re leaking private info because your access checking code happens to get the real 64-bit value but your message retrieval code puts the ID in a JSON number. Now you might respond, but that ID scheme is dumb, don’t do that. And you may be right! But dumb things happen. It’s unwise to leave land mines lying around in your software just because they only detonate when someone does something dumb.