5 ms·

It's weird that any parser that loses digits is tolerated. A parser that forces strings into uppercase US-ASCII never would be.

msm_ 3y ago
That's true for every floating point number in every programming language you have ever used, though. $ python3 Python 3.10.13 (main, Aug 24 2023, 12:59:26) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 100000.000000000017 100000.00000000001
- Izkata 3y ago
  This is why Decimal exists: Python 3.8.10 (default, Nov 22 2023, 10:22:35) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from decimal import Decimal >>> Decimal('100000.000000000017') Decimal('100000.000000000017') For example: >>> import json >>> json.loads('{"a": 100000.000000000017}') {'a': 100000.00000000001} >>> json.loads('{"a": 100000.000000000017}', parse_float=Decimal) {'a': Decimal('100000.000000000017')}
  - cerved 3y ago
    but serializing/deserializing decimal using the json module is futile
    - yawaramin 3y ago
      Why is it futile? It can be serialized/deserialized perfectly through its string representation.
  - dhosek 3y ago
    And not every programming language offers a Decimal type and on most of those, there’s usually a performance penalty associated with it not to mention issues of interoperability and developer knowledge of its existence. For financial calculations, usually using integers with an implicit decimal offset (e.g., US currency amounts being expressed in cents rather than dollars), while other contexts will often determine that the inherent inaccuracy of IEEE floating types is a non-issue. The biggest potential problem lies in treating values that act kind of like numbers and look like numbers as numbers, e.g., Dewey Decimal classification numbers or the topic in a Library of Congress classification.¹ ⸻ 1. This is a bit on my mind lately as I discovered that LibraryThing’s sort by LoC classification seems to be broken so I exported my library (discovering that they export as ISO8859-1 with no option for UTF-8) and wrote a custom sorter for LOC classification codes for use in finally arranging the books on my shelves after my move last year.
  - Retr0id 3y ago
    Decimal is not arbitrary precision, though. It has many of the same issues, you'll just see them in different places. >>> Decimal('100000.00000000000000000000017') + Decimal('1') Decimal('100001.0000000000000000000002')
- enriquto 3y ago
  > That's true for every floating point number in every programming language you have ever used, though. Alright, if "you" have only ever used python. In C, for example, we have hexadecimal floating point literals that represent all floats and doubles exactly (including infinities and nans that make the json parser fail miserably).
  - legobmw99 3y ago
    If you use the same syntax as OP, C’s parser will also round that literal. The existence of a hex literal for floats is something orthogonal
  - codetrotter 3y ago
    > we have hexadecimal floating point literals that represent all floats and doubles exactly How do you do that? A couple of resources I found but which I’m not sure if are about exactly what you speak of https://stackoverflow.com/questions/65480947/is-ieee-754-representation-used-in-c/65481061 https://stackoverflow.com/questions/65480947/is-ieee-754-rep... https://gcc.gnu.org/onlinedocs/gcc/Hex-Floats.html https://gcc.gnu.org/onlinedocs/gcc/Hex-Floats.html Furthermore, what exactly do you mean by “all floats and doubles exactly”?
    - enriquto 3y ago
      Yes, I was talking about what is described in your resources. You can do this: // define a floating-point literal in hex and print it in decimal float x = 0x1p-8; // x = 1.0/256 printf("x = %g\n", x); // prints 0.00390625 // define a floating point literal in decimal and print it in various ways float y = 0.3; // non-representable, rounded to closest float printf("y = %g\n", y); // 0.3 (the %g format does some heuristics) printf("y = %.10f\n", y); // 0.3000000119 printf("y = %.20f\n", y); // 0.30000001192092895508 printf("y = %a\n", f); // 0x1.333334p-2
      - codetrotter 3y ago
        So for example if you make a variable that has the value parent commenter used 100000.000000000017 And then you print it. Does it preserve the exact value?
tpm 3y ago
https://0.30000000000000004.com/ https://0.30000000000000004.com/ Although it would be good to move in the direction of using a BigDecimal equivalent by default when ingesting unknown data.
int_19h 3y ago
It's tolerated because the JSON spec explicitly allows it: This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available. Note that when such software is used, numbers that are integers and are in the range [-(2**53)+1, (2**53)-1] are interoperable in the sense that implementations will agree exactly on their numeric values. And yes, this is completely insane for a format that supposed to be specifically for serialization and interop. Needless to say, the industry has enthusiastically adopted it to the point where it became the standard. I miss XML these days. Sure, it was verbose and had a bunch of different and probably excessive numeric types defined for XML Schema... but at least they were well-defined (https://www.w3.org/TR/xmlschema-2/#built-in-datatypes https://www.w3.org/TR/xmlschema-2/#built-in-datatypes). And, on the other hand, without a schema, all you had were strings. Either way, no mismatched expectations.

msm_ 3y ago

That's true for every floating point number in every programming language you have ever used, though. $ python3 Python 3.10.13 (main, Aug 24 2023, 12:59:26) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 100000.000000000017 100000.00000000001

Izkata 3y ago
This is why Decimal exists: Python 3.8.10 (default, Nov 22 2023, 10:22:35) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from decimal import Decimal >>> Decimal('100000.000000000017') Decimal('100000.000000000017') For example: >>> import json >>> json.loads('{"a": 100000.000000000017}') {'a': 100000.00000000001} >>> json.loads('{"a": 100000.000000000017}', parse_float=Decimal) {'a': Decimal('100000.000000000017')}
- cerved 3y ago
  but serializing/deserializing decimal using the json module is futile
  - yawaramin 3y ago
    Why is it futile? It can be serialized/deserialized perfectly through its string representation.
- dhosek 3y ago
  And not every programming language offers a Decimal type and on most of those, there’s usually a performance penalty associated with it not to mention issues of interoperability and developer knowledge of its existence. For financial calculations, usually using integers with an implicit decimal offset (e.g., US currency amounts being expressed in cents rather than dollars), while other contexts will often determine that the inherent inaccuracy of IEEE floating types is a non-issue. The biggest potential problem lies in treating values that act kind of like numbers and look like numbers as numbers, e.g., Dewey Decimal classification numbers or the topic in a Library of Congress classification.¹ ⸻ 1. This is a bit on my mind lately as I discovered that LibraryThing’s sort by LoC classification seems to be broken so I exported my library (discovering that they export as ISO8859-1 with no option for UTF-8) and wrote a custom sorter for LOC classification codes for use in finally arranging the books on my shelves after my move last year.
- Retr0id 3y ago
  Decimal is not arbitrary precision, though. It has many of the same issues, you'll just see them in different places. >>> Decimal('100000.00000000000000000000017') + Decimal('1') Decimal('100001.0000000000000000000002')
enriquto 3y ago
> That's true for every floating point number in every programming language you have ever used, though. Alright, if "you" have only ever used python. In C, for example, we have hexadecimal floating point literals that represent all floats and doubles exactly (including infinities and nans that make the json parser fail miserably).
- legobmw99 3y ago
  If you use the same syntax as OP, C’s parser will also round that literal. The existence of a hex literal for floats is something orthogonal
- codetrotter 3y ago
  > we have hexadecimal floating point literals that represent all floats and doubles exactly How do you do that? A couple of resources I found but which I’m not sure if are about exactly what you speak of https://stackoverflow.com/questions/65480947/is-ieee-754-representation-used-in-c/65481061 https://stackoverflow.com/questions/65480947/is-ieee-754-rep... https://gcc.gnu.org/onlinedocs/gcc/Hex-Floats.html https://gcc.gnu.org/onlinedocs/gcc/Hex-Floats.html Furthermore, what exactly do you mean by “all floats and doubles exactly”?
  - enriquto 3y ago
    Yes, I was talking about what is described in your resources. You can do this: // define a floating-point literal in hex and print it in decimal float x = 0x1p-8; // x = 1.0/256 printf("x = %g\n", x); // prints 0.00390625 // define a floating point literal in decimal and print it in various ways float y = 0.3; // non-representable, rounded to closest float printf("y = %g\n", y); // 0.3 (the %g format does some heuristics) printf("y = %.10f\n", y); // 0.3000000119 printf("y = %.20f\n", y); // 0.30000001192092895508 printf("y = %a\n", f); // 0x1.333334p-2
    - codetrotter 3y ago
      So for example if you make a variable that has the value parent commenter used 100000.000000000017 And then you print it. Does it preserve the exact value?

tpm 3y ago

https://0.30000000000000004.com/ https://0.30000000000000004.com/ Although it would be good to move in the direction of using a BigDecimal equivalent by default when ingesting unknown data.

int_19h 3y ago

It's tolerated because the JSON spec explicitly allows it: This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available. Note that when such software is used, numbers that are integers and are in the range [-(2**53)+1, (2**53)-1] are interoperable in the sense that implementations will agree exactly on their numeric values. And yes, this is completely insane for a format that supposed to be specifically for serialization and interop. Needless to say, the industry has enthusiastically adopted it to the point where it became the standard. I miss XML these days. Sure, it was verbose and had a bunch of different and probably excessive numeric types defined for XML Schema... but at least they were well-defined (https://www.w3.org/TR/xmlschema-2/#built-in-datatypes https://www.w3.org/TR/xmlschema-2/#built-in-datatypes). And, on the other hand, without a schema, all you had were strings. Either way, no mismatched expectations.