6 ms·

NaN Is Weird

35 points by freediver 7mo ago

paulddraper 7mo ago
NaN == NaN is truly a perversion of equality. It makes little sense that 1/0 is SIGFPE, but log(-5) is NaN in C. And the same is true for higher level languages, and their error facilities. What a mess.
- Aardwolf 7mo ago
  Python is extra annoying though with refusing to support division through zero the way other programming languages with IEEE floats do (i.e. output inf or nan instead of throwing an exception), even though it has no problem doing things like float('inf') / float('inf') --> nan. It specifically does it for division through zero as if it wants to be a junior grade calculator just for this one thing. They could at least have fixed this when breaking backwards incompatibility from python2 to 3...
  - pletnes 7mo ago
    Just use numpy if you want to do math. Seriously.
  - dgrunwald 7mo ago
    In most languages, `x: float = 0` involves an implicit conversion from int to float. In Python, type annotations have no impact on runtime behavior, so even though the type checker accepts this code, `type(x)` will be `int` -- python acts as if `int` was a subtype of `float`. It would be weird if the behavior of `1 / x` was different depending on whether `0` or `0.0` was passed to a `x: float` parameter -- if `int` is a subtype of `float`, then any operation allowed on `float` (e.g. division) should have the same behavior on both types. This means Python had to choose at least one: 1. division violates the liskov substitution principle 2. division by zero involving only integer inputs returns NaN 3. division by zero involving only float inputs throws exception 4. It's a type error to pass an int where a float is expected. They went with option 3, and I think I agree that this is the least harmful/surprising choice. Proper statically typed languages don't have to make this unfortunate tradeoff.
    - Aardwolf 7mo ago
      C does different things for 0.0 / 0.0 and 0 / 0 and it's not that weird to deal with (well it has other issues like it being platform dependent what happens with this). JS has no problem with it either (0.0 / 0.0 gives nan, 0n / 0n gives exception since it are integers). Python is the only language doing this (of the ones I use at least). I don't think the notation `x: float = 0` existed when it was new by the way so that can't be the design reason? since python seems to handle integer through integer divisions as float (e.g. 5 / 2 outputs 2.5), 0 / 0 giving nan would seem to be expected there > liskov substitution principle that would imply one is a subtype of another, is that really the case here? there are floats that can't be represented as an integer (e.g. 0.5) and integers that can't be represented as a double precision float (e.g. 18446744073709551615)
      - kbolino 7mo ago
        Python chose, quite some time ago, not to follow C's lead on division: PEP 238 – Changing the Division Operator (2001) [1] The rationale is basically that newcomers to Python should see the results that they would expect from grade school mathematics, not the results that an experienced programmer would expect from knowing C. While the PEP above doesn't touch on division by zero, it does point toward the objective being a cohesive, layman-friendly numeric system. [1]: https://peps.python.org/pep-0238/ https://peps.python.org/pep-0238/
      - xigoi 7mo ago
        C and JavaScript both treat integers and floats as separate types. In Python, ints and floats with the same numeric value are considered identical for almost all purposes.
      - dgrunwald 7mo ago
        Yes `int` acts as if it was a subtype of `float`: https://typing.python.org/en/latest/spec/special-types.html#special-cases-for-float-and-complex https://typing.python.org/en/latest/spec/special-types.html#...
    - paulddraper 7mo ago
      > In Python, type annotations have no impact on runtime behavior That's true of any "scripting" language types. E.g. TypeScript const x: number = 0n; // still is a bigint Granted, in both the Python and the TypeScript example, the type checker will complain at your incorrect type annotation.
  - caditinpiscinam 7mo ago
    Nah. Python gets it right; all high level languages should operate this way. Division by zero is a bug 90% of the time. Errors should never pass silently. Special cases aren't special enough to break the rules. IEEE floats should be a base on which more reasonable math semantics are built. Saying that Python should return NaN or inf instead of throwing an error is like saying that Python should return a random value from memory or segfault when reading an out-of-bounds list index.
    - paulddraper 7mo ago
      Exactly. Programming language design should be coherent and consistent, not just for floats, but arrays, classes, pointers, or anything else that it offers. And the sensible thing will depend on that language.
  - paulddraper 7mo ago
    > It specifically does it for division through zero as if it wants to be a junior grade calculator just for this one thing. Not just division by zero. math.sqrt(-1) math.log(-2) are errors as well. I agree that using both errors and float('nan') is odd. Tbf, C mixes errors (for integers) and NaN (for floats).
- adamzochowski 7mo ago
  Makes perfect sense . NaN is a special type indicating one can't reason about it normal way. It is an unknown or value that can't be represented. When comparing, think of it like comparing two bags of unknown amount of apples. One bag has NaN count of apples Other bag has NaN count of apples Do the two bags have equal number of apples? I wish all languages used nulls the way SQL does.
  - caditinpiscinam 7mo ago
    Respectfully, I disagree. If NaNs were meant to represent unknown quantities, then they would return false for all comparisons. But NaN != NaN is true. Assuming that two unknowns are always different is just as incorrect as assuming that they're always the same. I'd also push back on the idea that this behavior makes sense. In my experience it's a consistent source of confusion for anyone learning to program. It's one of the clearest violations of the principle of least astonishment in programming language design. As others have noted, it makes conscientious languages like Rust do all sorts of gymnastics to accommodate. It's a weird edge case, and imo a design mistake. "Special cases aren't special enough to break the rules." Also, I think high level languages should avoid exposing programmers to NaN whenever possible. Python gets this right: 0/0 should be an error, not a NaN.
    - glkindlmann 7mo ago
      As a standard for floating point representation and computation, IEEE 754 solved multiple long-standing serious problems better than anything that came before it. I don't think its sensible to judge it with a PL design lens like "principle of least astonishment"; certainly not as if IEEE754 is a peer to Rust or Python. Or, you could learn about the surprise, frustration, and expense involved in trying to write portable numeric code in the 1970s, prior to IEEE 754: https://people.eecs.berkeley.edu/~wkahan/ieee754status/754story.html https://people.eecs.berkeley.edu/~wkahan/ieee754status/754st...
    - happytoexplain 7mo ago
      `NaN == NaN` vs `NaN != NaN` forces us to pick which idea to violate: 1. A value of a given type, either a literal or a value in memory, should represent a single concrete value, semantically (`5` represents the number five). 2. Classical logic (if A = B is true, then A != B is false). I vote for violating #1, because we control those terms better. Is it weird that we invented a value that represents "unrepresentable" (as opposed to representing "no value")? Yes, but there are practical reasons for why we did it. It would be much more weird/surprising to violate classical logic.
- nitwit005 7mo ago
  There's no non-confusing option for comparisons. You have two invalid values, but they aren't necessarily the same invalid value. There are multiple operations that can produce NaN. It's a sentinel value for an error. Once you have an error, doing math with the error code isn't sensible.
  - caditinpiscinam 7mo ago
    There are no non-confusing options, but some of those are still clearly worse than others. What should sorted([3, nan, 2, 4, 1]) give you in Python? A) [1, 2, 3, 4, nan] is an good option B) [nan, 1, 2, 3, 4] is an good option C) An error is an good option D) [3, nan, 1, 2, 4] is a silly, bad option. It's definitely not what you want, and it's quiet enough to slip by unnoticed. This is what you get when Nan != NaN NaN == NaN is wrong. NaN != NaN is wrong, unintuitive, and breaks the rest of your code. If you want to signal that an operation is invalid, then throw an error. The silently nonsensical semantics of NaN are the worst possible response
    - paulddraper 7mo ago
      Yes. And to boot…it’s only floating point numbers In invalid math op with integers produces an error (or undefined behavior). Definitely not some self unequal abomination.
agwa 7mo ago
Fun fact - in C++ std::sort has undefined behavior, and can crash[1], if you try to sort a container with NaNs in it. [1] https://stackoverflow.com/questions/18291620/why-will-stdsort-crash-if-the-comparison-function-is-not-as-operator https://stackoverflow.com/questions/18291620/why-will-stdsor...
- tialaramex 7mo ago
  This particular defect is a Quality of Implementation problem that's cheerfully allowed by the standard. Rust's standard sorts not only lack the API footgun which causes so many C++ programmers to blow their feet off this way, but even if you go out of your way to provide a nonsensical comparison both sorts are safe anyway because it's a safe language and yet they're also faster than any popular implementation of std::sort / std::stable_sort as appropriate. It's kinda silly, as late as the Biden administration one of the popular C++ stdlib implementations wasn't even O(n log n) because apparently twenty years was not enough for them to have introduced an introsort yet... That one isn't the standard's fault, the C++ 11 standard does say you should provide an O(n log n) sort at least.
adampunk 7mo ago
NaN is weird? No, NaN is normal*, NaN PAYLOADS are weird: https://anniecherkaev.com/the-secret-life-of-nan https://anniecherkaev.com/the-secret-life-of-nan *This is false, NaN is weird, though maybe it needs to be. It is nowhere written arithmetic on computers must be straightforward.
- tialaramex 7mo ago
  Actually none of the floating point value are normal :D Although Almost All† Real Numbers are Normal, none of the floats are normal, and nor are most numbers any regular person would think of (possible exceptions Pi and Euler's Number which are conjectured to be Normal although it is unproven) † Almost All is a term of art in mathematics, remember there are uncountably many real numbers, so the fact the Normals are also uncountable puts them at least in the right ballpark, unlike the rationals.
kmeisthax 7mo ago
This is also why Rust has separate PartialEq and Eq traits - the latter is only available for types that don't have weird not-self-equal values like floating point NaNs or SQL NULLs. If you lie to Rust and create a wrapper type over f32 or f64 that has Eq, then you'd get unindexable NaN keys that just sit in your hashmap forever. The real surprise to me is that Python can index NaN keys sometimes, at least by reference to the original NaN. I knew CPython does some Weird Shit with primitive values, so I assume it's because the hashmap is comparing by reference first and then by value.
- tialaramex 7mo ago
  > unindexable NaN keys that just sit in your hashmap forever. Although you shouldn't put nonsense in a HashMap, HashMap::drain will give you (an iterator to get) back everything you put into the HashMap even if it's nonsense and HashMap::retain will let you provide a predicate to throw away the nonsense if you want to keep everything else, while HashMap::extract_if even allows you to get the nonsense back out again to put that somewhere more sensible. These work because they don't need to find your nonsense in particular using hashing, which may now be impossible, they're just looking at everything.
kubb 7mo ago
It would be more satisfying to learn why hash of nan is not guaranteed to be the same. It feels like a bug.
- Jaxan 7mo ago
  The hash is the same. But a hash set has to use == in case of equal hashes (to avoid collisions).
  - kubb 7mo ago
    It's not always the same: >>> hash(float('nan')) 271103401 >>> hash(float('nan')) 271103657
    - zahlman 7mo ago
      Yes. The CPython hash algorithm for floats (https://github.com/python/cpython/blob/main/Python/pyhash.c#L87 https://github.com/python/cpython/blob/main/Python/pyhash.c#...) special-cases the non-finite values: floating-point infinities hash to special values modeled on the digits of pi (seriously! See https://github.com/python/cpython/blob/main/Include/cpython/pyhash.h#L19 https://github.com/python/cpython/blob/main/Include/cpython/...), and NaNs fall through ultimately to https://github.com/python/cpython/blob/main/Include/internal/pycore_pyhash.h#L10 https://github.com/python/cpython/blob/main/Include/internal... which is based on object identity (the pointer to the object is used rather than its data).
    - 1-more 7mo ago
      maybe it's that multiple bit patterns can be NaN and these are two different ones? In IEEE-754, a number with all the exponent bits set to 1 is +/-infinity if the fraction bits are all zero, otherwise it's NaN. So these could be values where the fractions differ. Can you see what the actual bits it's setting are?
- AlotOfReading 7mo ago
  At the standards level, NaN payload propagation isn't guaranteed, regardless of any other issues.
AndriyKunitsyn 7mo ago
NaN that is not equal to itself _even if it's the same variable_ is not a Python oddity, it's an IEEE 754 oddity.
- riskassessment 7mo ago
  Nor is that inequality an oddity at all. If you were to think NaN should equal NaN, that thought would probably stem from the belief that NaN is a singular entity which is a misunderstanding of its purpose. NaN rather signifies a specific number that is not representable as a floating point. Two specific numbers that cannot be represented are not necessarily equal because they may have resulted from different calculations! I'll add that, if I recall correctly, in R, the statement NaN == NaN evaluates to NA which basicall means "it is not known whether these numbers equal each other" which is a more reasonable result than False.
  - AndriyKunitsyn 7mo ago
    It's the only "primitive type" that does that. If I deserialize data from wire, I'll be very surprised when the same bits deserialize as unequal variables. If it cannot be represented, then throwing makes more sense than trying to represent it.
    - adrian_b 7mo ago
      Other primitive types also do this, but this is not clearly visible from high-level programming languages, because most HLLs have only incomplete support for the CPU hardware. If you do a (signed) integer operation, the hardware does not fit the result in a register of the size expected in a HLL, but the result has some bits elsewhere, typically in a "flags" register. So the result of an integer arithmetic operation has an extra bit, usually named as the "overflow" bit. That bit is used to encode a not-a-number value, i.e. if the overflow bit is set, the result of the operation is an integer NaN. For correct results, one should check whether the result is a NaN, which is called checking for integer overflow (unlike for FP, the integer execution units do not distinguish between true overflow and undefined operations, i.e. there are no distinct encodings for infinity and for NaN). After checking that the result is not a NaN, the extra bit can be stripped from the result. If you serialize an integer number for sending it elsewhere, that implicitly assumes that wherever your number was produced, someone has tested for overflow, i.e. that the value is not a NaN, so the extra bit was correctly stripped from the value. If nobody has tested, your serialized value can be bogus, the same as when serializing a FP NaN and not checking later that it is a NaN, before using one of the 6 relational operators intended for total orders, which may be wrong for partial orders.
cmovq 7mo ago
> we had an unusual discussion about a Python oddity There are so many discussions about "X language is so weird about it handles numbers!" and it's just IEEE 754 floats.
- swiftcoder 7mo ago
  The oddity here is not the float itself, it's that Python provided a default hash implementation for floats
  - JuniperMesos 7mo ago
    Yeah IEEE 754 floating point numbers should probably not be hashable, and the weird (but standard-defined) behaviour with respect to NaN equality is one good reason for this.
    - deleted 7mo ago
      [deleted]
  - _cogg 7mo ago
    Python supports arithmetic on mixed numeric types, so it makes sense that floats and ints should have a hash function that behaves somewhat consistently. I don't write a lot of python, but having used other scripting languages it wouldn't surprise me if numeric types get mixed up by accident often enough. You probably want int(2) and float(2) to be considered the same key in a dictionary to avoid surprises. See: https://docs.python.org/3/library/stdtypes.html#hashing-of-numeric-types https://docs.python.org/3/library/stdtypes.html#hashing-of-n...
    - swiftcoder 7mo ago
      > You probably want int(2) and float(2) to be considered the same key in a dictionary to avoid surprises There are a variety of problems here: - floats can't represent whole numbers exactly as they get further from zero. - error accumulates in floating point arithmetic, so calculated keys may not match constant keys - Do you really want 2.0 and 2.00001 to refer to different objects?
munchler 7mo ago
>>> my_dict[nan] = 3 >>> my_dict[nan] 3 Wait, how does that work if nan isn't equal to itself?
- gravel7623 7mo ago
  I guess because the hash of an instance stays consistent (which is used to retrieve the value from the dict). The `__eq__` method must disregard the hash and return False for all nans.
  - munchler 7mo ago
    But the hash alone shouldn't be enough to match the key. Isn't an equality check also needed to avoid a false positive? That's the idea behind a hash table, as I understand it. (I'm not a Python programmer.)
    - zahlman 7mo ago
      That equality check also considers object identity first: >>> class Evil: ... def __init__(self, hash_value): self.hash_value = hash_value ... def __eq__(self, other): return False ... def __hash__(self): return self.hash_value ... >>> e1, e2 = Evil(1), Evil(2) >>> {e1:1, e2:2} {<__main__.Evil object at ...>: 1, <__main__.Evil object at ...>: 2} >>> {e1:1, e2:2}[Evil(1)] Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: <__main__.Evil object at ...> >>> {e1:1, e2:2}[e1] 1 I'm pretty sure that this is meant as an optimization. (But it does have to find the instance via the hash lookup first. This won't work if you e.g. return a random number from `__hash__`.)
zahlman 7mo ago
> Last week in the Python Discord we had an unusual discussion about a Python oddity. Oh, I missed it. But yes, this is more to do with NaN than Python. > But, of course, you can't actually get to those values by their keys: ... That is, unless you stored the specific instance of nan as a variable: Worth noting that sets work the same way here, although this was glossed over: you can store multiple NaNs in a set, but not the same NaN instance multiple times. Even though it isn't equal to itself, the insertion process (for both dicts and sets) will consider object identity before object equality: >>> x = float('nan') >>> {x for _ in range(10)} {nan} And, yes, the same is of course true of `collections.Counter`.
- cogman10 7mo ago
  If I could change one thing in computing, it'd be how SQL handles NULL. But if I got a second thing, it'd be how IEEE handles NaN. I probably wouldn't even allow NaN as a representation. If some mathematical operation results in what would be NaN, I'd rather force the programming language to throw some sort of interrupt or exception. Much like what happens when you divide an integer by 0. Heck, I'd probably even stop infinity from being represented with floats. If someone did 1/0 or 0/0, I'd interrupt rather than generating an INF or NaN. In my experience, INF and NaN are almost always an indicator of programming error. If someone want's to programmatically represent those concepts, they could do it on top of and to the side of the floating point specification, not inside it.
  - zahlman 7mo ago
    Floating-point infinity is actually really useful in Python because you can easily and efficiently compare it to Python's arbitrary-size integers.
    - cogman10 7mo ago
      For what purpose? When does this come up? I can also compare infinity to Java's BigDecimal values but I fail to see what I'd want to or need to.
wackych 7mo ago
Reminds me of this classic 4chan thread which started with an absurd-sounding comparison operator and ended with NaN semantic revelations. https://archive.tinychan.net/read/prog/1176222557 https://archive.tinychan.net/read/prog/1176222557
jeleh 7mo ago
...reminds me that object + object is NaN: > {} + {} NaN see https://www.destroyallsoftware.com/talks/wat https://www.destroyallsoftware.com/talks/wat PS: Wait for it ... Watman! =8-)
superjan 7mo ago
IEEE 754 prescripes, for better or worse, that any mathematical comparison operator (==, <, > ….) involving at least one NaN must always return false, including comparison against itself. This is annoying for something like dictionaries or hashtables. C# has a solution: if you call a.Equals(b) on two floats a and b, it will return true also if both are NaN. I think this is a cool solution: it keeps the meaning of math operators the same identical with other languages, but you still have sensible behavior for containers. I believe this behavior is copied from Java.
- adrian_b 7mo ago
  I consider this as a very bad solution, because it can lead to very subtle bugs. The correct solution for any programming language is to define all the 14 relational operators that are required by any partially-ordered set, instead of defining only the 6 of them that are sufficient for a totally-ordered set. If the programming language fails to define all 14 operators, then you must always test the operands for NaNs, before using any of the 6 ALGOL relational operators. If you consider this tedious, then you must unmask the invalid operation exception and take care to handle this exception. If invalid operations generate exceptions, then the floating-point numbers become a totally-ordered set and NaN cannot exist (if a NaN comes from an external source, it will also generate an exception, while internally no NaN will ever be generated).
1-more 7mo ago
This got me curious, and yeah it turns out Elm's dictionary implementation uses values, not pointers when retrieving values. elm repl ---- Elm 0.19.1 ---------------------------------------------------------------- Say :help for help and :exit to exit! More at <https://elm-lang.org/0.19.1/repl> -------------------------------------------------------------------------------- > import Dict exposing (Dict) > nan = 0/0 NaN : Float > nan NaN : Float > nan == nan False : Bool > naan = 0/0 NaN : Float > d = Dict.fromList [(nan, "a"), (naan, "b")] Dict.fromList [(NaN,"a"),(NaN,"b")] : Dict Float String > Dict.toList d [(NaN,"a"),(NaN,"b")] : List ( Float, String ) > Dict.keys d [NaN,NaN] : List Float > Dict.get nan d Nothing : Maybe String > Dict.get naan d Nothing : Maybe String

35 points by freediver 7mo ago

NaN == NaN is truly a perversion of equality. It makes little sense that 1/0 is SIGFPE, but log(-5) is NaN in C. And the same is true for higher level languages, and their error facilities. What a mess.

Aardwolf 7mo ago
Python is extra annoying though with refusing to support division through zero the way other programming languages with IEEE floats do (i.e. output inf or nan instead of throwing an exception), even though it has no problem doing things like float('inf') / float('inf') --> nan. It specifically does it for division through zero as if it wants to be a junior grade calculator just for this one thing. They could at least have fixed this when breaking backwards incompatibility from python2 to 3...
- pletnes 7mo ago
  Just use numpy if you want to do math. Seriously.
- dgrunwald 7mo ago
  In most languages, `x: float = 0` involves an implicit conversion from int to float. In Python, type annotations have no impact on runtime behavior, so even though the type checker accepts this code, `type(x)` will be `int` -- python acts as if `int` was a subtype of `float`. It would be weird if the behavior of `1 / x` was different depending on whether `0` or `0.0` was passed to a `x: float` parameter -- if `int` is a subtype of `float`, then any operation allowed on `float` (e.g. division) should have the same behavior on both types. This means Python had to choose at least one: 1. division violates the liskov substitution principle 2. division by zero involving only integer inputs returns NaN 3. division by zero involving only float inputs throws exception 4. It's a type error to pass an int where a float is expected. They went with option 3, and I think I agree that this is the least harmful/surprising choice. Proper statically typed languages don't have to make this unfortunate tradeoff.
  - Aardwolf 7mo ago
    C does different things for 0.0 / 0.0 and 0 / 0 and it's not that weird to deal with (well it has other issues like it being platform dependent what happens with this). JS has no problem with it either (0.0 / 0.0 gives nan, 0n / 0n gives exception since it are integers). Python is the only language doing this (of the ones I use at least). I don't think the notation `x: float = 0` existed when it was new by the way so that can't be the design reason? since python seems to handle integer through integer divisions as float (e.g. 5 / 2 outputs 2.5), 0 / 0 giving nan would seem to be expected there > liskov substitution principle that would imply one is a subtype of another, is that really the case here? there are floats that can't be represented as an integer (e.g. 0.5) and integers that can't be represented as a double precision float (e.g. 18446744073709551615)
    - kbolino 7mo ago
      Python chose, quite some time ago, not to follow C's lead on division: PEP 238 – Changing the Division Operator (2001) [1] The rationale is basically that newcomers to Python should see the results that they would expect from grade school mathematics, not the results that an experienced programmer would expect from knowing C. While the PEP above doesn't touch on division by zero, it does point toward the objective being a cohesive, layman-friendly numeric system. [1]: https://peps.python.org/pep-0238/ https://peps.python.org/pep-0238/
    - xigoi 7mo ago
      C and JavaScript both treat integers and floats as separate types. In Python, ints and floats with the same numeric value are considered identical for almost all purposes.
    - dgrunwald 7mo ago
      Yes `int` acts as if it was a subtype of `float`: https://typing.python.org/en/latest/spec/special-types.html#special-cases-for-float-and-complex https://typing.python.org/en/latest/spec/special-types.html#...
  - paulddraper 7mo ago
    > In Python, type annotations have no impact on runtime behavior That's true of any "scripting" language types. E.g. TypeScript const x: number = 0n; // still is a bigint Granted, in both the Python and the TypeScript example, the type checker will complain at your incorrect type annotation.
- caditinpiscinam 7mo ago
  Nah. Python gets it right; all high level languages should operate this way. Division by zero is a bug 90% of the time. Errors should never pass silently. Special cases aren't special enough to break the rules. IEEE floats should be a base on which more reasonable math semantics are built. Saying that Python should return NaN or inf instead of throwing an error is like saying that Python should return a random value from memory or segfault when reading an out-of-bounds list index.
  - paulddraper 7mo ago
    Exactly. Programming language design should be coherent and consistent, not just for floats, but arrays, classes, pointers, or anything else that it offers. And the sensible thing will depend on that language.
- paulddraper 7mo ago
  > It specifically does it for division through zero as if it wants to be a junior grade calculator just for this one thing. Not just division by zero. math.sqrt(-1) math.log(-2) are errors as well. I agree that using both errors and float('nan') is odd. Tbf, C mixes errors (for integers) and NaN (for floats).
adamzochowski 7mo ago
Makes perfect sense . NaN is a special type indicating one can't reason about it normal way. It is an unknown or value that can't be represented. When comparing, think of it like comparing two bags of unknown amount of apples. One bag has NaN count of apples Other bag has NaN count of apples Do the two bags have equal number of apples? I wish all languages used nulls the way SQL does.
- caditinpiscinam 7mo ago
  Respectfully, I disagree. If NaNs were meant to represent unknown quantities, then they would return false for all comparisons. But NaN != NaN is true. Assuming that two unknowns are always different is just as incorrect as assuming that they're always the same. I'd also push back on the idea that this behavior makes sense. In my experience it's a consistent source of confusion for anyone learning to program. It's one of the clearest violations of the principle of least astonishment in programming language design. As others have noted, it makes conscientious languages like Rust do all sorts of gymnastics to accommodate. It's a weird edge case, and imo a design mistake. "Special cases aren't special enough to break the rules." Also, I think high level languages should avoid exposing programmers to NaN whenever possible. Python gets this right: 0/0 should be an error, not a NaN.
  - glkindlmann 7mo ago
    As a standard for floating point representation and computation, IEEE 754 solved multiple long-standing serious problems better than anything that came before it. I don't think its sensible to judge it with a PL design lens like "principle of least astonishment"; certainly not as if IEEE754 is a peer to Rust or Python. Or, you could learn about the surprise, frustration, and expense involved in trying to write portable numeric code in the 1970s, prior to IEEE 754: https://people.eecs.berkeley.edu/~wkahan/ieee754status/754story.html https://people.eecs.berkeley.edu/~wkahan/ieee754status/754st...
  - happytoexplain 7mo ago
    `NaN == NaN` vs `NaN != NaN` forces us to pick which idea to violate: 1. A value of a given type, either a literal or a value in memory, should represent a single concrete value, semantically (`5` represents the number five). 2. Classical logic (if A = B is true, then A != B is false). I vote for violating #1, because we control those terms better. Is it weird that we invented a value that represents "unrepresentable" (as opposed to representing "no value")? Yes, but there are practical reasons for why we did it. It would be much more weird/surprising to violate classical logic.
nitwit005 7mo ago
There's no non-confusing option for comparisons. You have two invalid values, but they aren't necessarily the same invalid value. There are multiple operations that can produce NaN. It's a sentinel value for an error. Once you have an error, doing math with the error code isn't sensible.
- caditinpiscinam 7mo ago
  There are no non-confusing options, but some of those are still clearly worse than others. What should sorted([3, nan, 2, 4, 1]) give you in Python? A) [1, 2, 3, 4, nan] is an good option B) [nan, 1, 2, 3, 4] is an good option C) An error is an good option D) [3, nan, 1, 2, 4] is a silly, bad option. It's definitely not what you want, and it's quiet enough to slip by unnoticed. This is what you get when Nan != NaN NaN == NaN is wrong. NaN != NaN is wrong, unintuitive, and breaks the rest of your code. If you want to signal that an operation is invalid, then throw an error. The silently nonsensical semantics of NaN are the worst possible response
  - paulddraper 7mo ago
    Yes. And to boot…it’s only floating point numbers In invalid math op with integers produces an error (or undefined behavior). Definitely not some self unequal abomination.

agwa 7mo ago

Fun fact - in C++ std::sort has undefined behavior, and can crash[1], if you try to sort a container with NaNs in it. [1] https://stackoverflow.com/questions/18291620/why-will-stdsort-crash-if-the-comparison-function-is-not-as-operator https://stackoverflow.com/questions/18291620/why-will-stdsor...

tialaramex 7mo ago
This particular defect is a Quality of Implementation problem that's cheerfully allowed by the standard. Rust's standard sorts not only lack the API footgun which causes so many C++ programmers to blow their feet off this way, but even if you go out of your way to provide a nonsensical comparison both sorts are safe anyway because it's a safe language and yet they're also faster than any popular implementation of std::sort / std::stable_sort as appropriate. It's kinda silly, as late as the Biden administration one of the popular C++ stdlib implementations wasn't even O(n log n) because apparently twenty years was not enough for them to have introduced an introsort yet... That one isn't the standard's fault, the C++ 11 standard does say you should provide an O(n log n) sort at least.

adampunk 7mo ago

NaN is weird? No, NaN is normal*, NaN PAYLOADS are weird: https://anniecherkaev.com/the-secret-life-of-nan https://anniecherkaev.com/the-secret-life-of-nan *This is false, NaN is weird, though maybe it needs to be. It is nowhere written arithmetic on computers must be straightforward.

tialaramex 7mo ago
Actually none of the floating point value are normal :D Although Almost All† Real Numbers are Normal, none of the floats are normal, and nor are most numbers any regular person would think of (possible exceptions Pi and Euler's Number which are conjectured to be Normal although it is unproven) † Almost All is a term of art in mathematics, remember there are uncountably many real numbers, so the fact the Normals are also uncountable puts them at least in the right ballpark, unlike the rationals.

kmeisthax 7mo ago

This is also why Rust has separate PartialEq and Eq traits - the latter is only available for types that don't have weird not-self-equal values like floating point NaNs or SQL NULLs. If you lie to Rust and create a wrapper type over f32 or f64 that has Eq, then you'd get unindexable NaN keys that just sit in your hashmap forever. The real surprise to me is that Python can index NaN keys sometimes, at least by reference to the original NaN. I knew CPython does some Weird Shit with primitive values, so I assume it's because the hashmap is comparing by reference first and then by value.

tialaramex 7mo ago
> unindexable NaN keys that just sit in your hashmap forever. Although you shouldn't put nonsense in a HashMap, HashMap::drain will give you (an iterator to get) back everything you put into the HashMap even if it's nonsense and HashMap::retain will let you provide a predicate to throw away the nonsense if you want to keep everything else, while HashMap::extract_if even allows you to get the nonsense back out again to put that somewhere more sensible. These work because they don't need to find your nonsense in particular using hashing, which may now be impossible, they're just looking at everything.

kubb 7mo ago

It would be more satisfying to learn why hash of nan is not guaranteed to be the same. It feels like a bug.

Jaxan 7mo ago
The hash is the same. But a hash set has to use == in case of equal hashes (to avoid collisions).
- kubb 7mo ago
  It's not always the same: >>> hash(float('nan')) 271103401 >>> hash(float('nan')) 271103657
  - zahlman 7mo ago
    Yes. The CPython hash algorithm for floats (https://github.com/python/cpython/blob/main/Python/pyhash.c#L87 https://github.com/python/cpython/blob/main/Python/pyhash.c#...) special-cases the non-finite values: floating-point infinities hash to special values modeled on the digits of pi (seriously! See https://github.com/python/cpython/blob/main/Include/cpython/pyhash.h#L19 https://github.com/python/cpython/blob/main/Include/cpython/...), and NaNs fall through ultimately to https://github.com/python/cpython/blob/main/Include/internal/pycore_pyhash.h#L10 https://github.com/python/cpython/blob/main/Include/internal... which is based on object identity (the pointer to the object is used rather than its data).
  - 1-more 7mo ago
    maybe it's that multiple bit patterns can be NaN and these are two different ones? In IEEE-754, a number with all the exponent bits set to 1 is +/-infinity if the fraction bits are all zero, otherwise it's NaN. So these could be values where the fractions differ. Can you see what the actual bits it's setting are?
AlotOfReading 7mo ago
At the standards level, NaN payload propagation isn't guaranteed, regardless of any other issues.

AndriyKunitsyn 7mo ago

NaN that is not equal to itself _even if it's the same variable_ is not a Python oddity, it's an IEEE 754 oddity.

riskassessment 7mo ago
Nor is that inequality an oddity at all. If you were to think NaN should equal NaN, that thought would probably stem from the belief that NaN is a singular entity which is a misunderstanding of its purpose. NaN rather signifies a specific number that is not representable as a floating point. Two specific numbers that cannot be represented are not necessarily equal because they may have resulted from different calculations! I'll add that, if I recall correctly, in R, the statement NaN == NaN evaluates to NA which basicall means "it is not known whether these numbers equal each other" which is a more reasonable result than False.
- AndriyKunitsyn 7mo ago
  It's the only "primitive type" that does that. If I deserialize data from wire, I'll be very surprised when the same bits deserialize as unequal variables. If it cannot be represented, then throwing makes more sense than trying to represent it.
  - adrian_b 7mo ago
    Other primitive types also do this, but this is not clearly visible from high-level programming languages, because most HLLs have only incomplete support for the CPU hardware. If you do a (signed) integer operation, the hardware does not fit the result in a register of the size expected in a HLL, but the result has some bits elsewhere, typically in a "flags" register. So the result of an integer arithmetic operation has an extra bit, usually named as the "overflow" bit. That bit is used to encode a not-a-number value, i.e. if the overflow bit is set, the result of the operation is an integer NaN. For correct results, one should check whether the result is a NaN, which is called checking for integer overflow (unlike for FP, the integer execution units do not distinguish between true overflow and undefined operations, i.e. there are no distinct encodings for infinity and for NaN). After checking that the result is not a NaN, the extra bit can be stripped from the result. If you serialize an integer number for sending it elsewhere, that implicitly assumes that wherever your number was produced, someone has tested for overflow, i.e. that the value is not a NaN, so the extra bit was correctly stripped from the value. If nobody has tested, your serialized value can be bogus, the same as when serializing a FP NaN and not checking later that it is a NaN, before using one of the 6 relational operators intended for total orders, which may be wrong for partial orders.

cmovq 7mo ago

> we had an unusual discussion about a Python oddity There are so many discussions about "X language is so weird about it handles numbers!" and it's just IEEE 754 floats.

swiftcoder 7mo ago
The oddity here is not the float itself, it's that Python provided a default hash implementation for floats
- JuniperMesos 7mo ago
  Yeah IEEE 754 floating point numbers should probably not be hashable, and the weird (but standard-defined) behaviour with respect to NaN equality is one good reason for this.
  - deleted 7mo ago
    [deleted]
- _cogg 7mo ago
  Python supports arithmetic on mixed numeric types, so it makes sense that floats and ints should have a hash function that behaves somewhat consistently. I don't write a lot of python, but having used other scripting languages it wouldn't surprise me if numeric types get mixed up by accident often enough. You probably want int(2) and float(2) to be considered the same key in a dictionary to avoid surprises. See: https://docs.python.org/3/library/stdtypes.html#hashing-of-numeric-types https://docs.python.org/3/library/stdtypes.html#hashing-of-n...
  - swiftcoder 7mo ago
    > You probably want int(2) and float(2) to be considered the same key in a dictionary to avoid surprises There are a variety of problems here: - floats can't represent whole numbers exactly as they get further from zero. - error accumulates in floating point arithmetic, so calculated keys may not match constant keys - Do you really want 2.0 and 2.00001 to refer to different objects?

munchler 7mo ago

>>> my_dict[nan] = 3 >>> my_dict[nan] 3 Wait, how does that work if nan isn't equal to itself?

gravel7623 7mo ago
I guess because the hash of an instance stays consistent (which is used to retrieve the value from the dict). The `__eq__` method must disregard the hash and return False for all nans.
- munchler 7mo ago
  But the hash alone shouldn't be enough to match the key. Isn't an equality check also needed to avoid a false positive? That's the idea behind a hash table, as I understand it. (I'm not a Python programmer.)
  - zahlman 7mo ago
    That equality check also considers object identity first: >>> class Evil: ... def __init__(self, hash_value): self.hash_value = hash_value ... def __eq__(self, other): return False ... def __hash__(self): return self.hash_value ... >>> e1, e2 = Evil(1), Evil(2) >>> {e1:1, e2:2} {<__main__.Evil object at ...>: 1, <__main__.Evil object at ...>: 2} >>> {e1:1, e2:2}[Evil(1)] Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: <__main__.Evil object at ...> >>> {e1:1, e2:2}[e1] 1 I'm pretty sure that this is meant as an optimization. (But it does have to find the instance via the hash lookup first. This won't work if you e.g. return a random number from `__hash__`.)

zahlman 7mo ago

> Last week in the Python Discord we had an unusual discussion about a Python oddity. Oh, I missed it. But yes, this is more to do with NaN than Python. > But, of course, you can't actually get to those values by their keys: ... That is, unless you stored the specific instance of nan as a variable: Worth noting that sets work the same way here, although this was glossed over: you can store multiple NaNs in a set, but not the same NaN instance multiple times. Even though it isn't equal to itself, the insertion process (for both dicts and sets) will consider object identity before object equality: >>> x = float('nan') >>> {x for _ in range(10)} {nan} And, yes, the same is of course true of `collections.Counter`.

cogman10 7mo ago
If I could change one thing in computing, it'd be how SQL handles NULL. But if I got a second thing, it'd be how IEEE handles NaN. I probably wouldn't even allow NaN as a representation. If some mathematical operation results in what would be NaN, I'd rather force the programming language to throw some sort of interrupt or exception. Much like what happens when you divide an integer by 0. Heck, I'd probably even stop infinity from being represented with floats. If someone did 1/0 or 0/0, I'd interrupt rather than generating an INF or NaN. In my experience, INF and NaN are almost always an indicator of programming error. If someone want's to programmatically represent those concepts, they could do it on top of and to the side of the floating point specification, not inside it.
- zahlman 7mo ago
  Floating-point infinity is actually really useful in Python because you can easily and efficiently compare it to Python's arbitrary-size integers.
  - cogman10 7mo ago
    For what purpose? When does this come up? I can also compare infinity to Java's BigDecimal values but I fail to see what I'd want to or need to.

wackych 7mo ago

Reminds me of this classic 4chan thread which started with an absurd-sounding comparison operator and ended with NaN semantic revelations. https://archive.tinychan.net/read/prog/1176222557 https://archive.tinychan.net/read/prog/1176222557

jeleh 7mo ago

...reminds me that object + object is NaN: > {} + {} NaN see https://www.destroyallsoftware.com/talks/wat https://www.destroyallsoftware.com/talks/wat PS: Wait for it ... Watman! =8-)

superjan 7mo ago

IEEE 754 prescripes, for better or worse, that any mathematical comparison operator (==, <, > ….) involving at least one NaN must always return false, including comparison against itself. This is annoying for something like dictionaries or hashtables. C# has a solution: if you call a.Equals(b) on two floats a and b, it will return true also if both are NaN. I think this is a cool solution: it keeps the meaning of math operators the same identical with other languages, but you still have sensible behavior for containers. I believe this behavior is copied from Java.

adrian_b 7mo ago
I consider this as a very bad solution, because it can lead to very subtle bugs. The correct solution for any programming language is to define all the 14 relational operators that are required by any partially-ordered set, instead of defining only the 6 of them that are sufficient for a totally-ordered set. If the programming language fails to define all 14 operators, then you must always test the operands for NaNs, before using any of the 6 ALGOL relational operators. If you consider this tedious, then you must unmask the invalid operation exception and take care to handle this exception. If invalid operations generate exceptions, then the floating-point numbers become a totally-ordered set and NaN cannot exist (if a NaN comes from an external source, it will also generate an exception, while internally no NaN will ever be generated).

1-more 7mo ago

This got me curious, and yeah it turns out Elm's dictionary implementation uses values, not pointers when retrieving values. elm repl ---- Elm 0.19.1 ---------------------------------------------------------------- Say :help for help and :exit to exit! More at <https://elm-lang.org/0.19.1/repl> -------------------------------------------------------------------------------- > import Dict exposing (Dict) > nan = 0/0 NaN : Float > nan NaN : Float > nan == nan False : Bool > naan = 0/0 NaN : Float > d = Dict.fromList [(nan, "a"), (naan, "b")] Dict.fromList [(NaN,"a"),(NaN,"b")] : Dict Float String > Dict.toList d [(NaN,"a"),(NaN,"b")] : List ( Float, String ) > Dict.keys d [NaN,NaN] : List Float > Dict.get nan d Nothing : Maybe String > Dict.get naan d Nothing : Maybe String