5 ms·

More proof that floats are evil. Moral of the story is to never use equality to compare floats.

trendia 10y ago
Pretty much. If you want to compare two floats a and b within some precision eps, use: abs(a - b) <= eps*0.5*(a + b) or even (abs(a - b) <= eps*abs(a)) && (abs(a - b) <= eps*abs(b)) which is implemented in Boost [0]. [0] http://www.boost.org/doc/libs/1_34_0/libs/test/doc/components/test_tools/floating_point_comparison.html http://www.boost.org/doc/libs/1_34_0/libs/test/doc/component...
- ratboy666 10y ago
  I thought it was (a / b) != (a / b)
geezerjay 10y ago
> More proof that floats are evil. Just because someone doesn't know the basics of floating point numbers it doesn't mean that the floating point format is somehow evil. Testing floats for equality is one of those no-nos which is taught right in the first class of introduction to FP numbers. Another very basic issue covered right in the intro clsses are problems caused by using decimal values to define numbers encoded in binary format, which is often explained through the famous "0.1 doesn't exist in binary" trivia bit.
- thechao 10y ago
  Part of my day job is to test floating point hardware and write compilers and assemblers. Testing equality---with some care---is completely fine. I can assure you this is a bug in GCC. The GCC bug is compounded by the nonstandard behavior of x87. The x87 unit is not "fixable" but GCC's behavior is.
  - nurettin 10y ago
    That's interesting, you make it sound like the production of new FPUs and new compilers is common. Which industry do you work in?
nom 10y ago
Floats are not evil. They are just not as intuitive as integers because it is impossible to represent every possible rational number in the range with a fixed number of bits. Once you get that, everything makes sense.
- monochromatic 10y ago
  That does not make sense of getting a failure from assert x/y == x/y
  - mikeash 10y ago
    Properly implemented floats will never fail that (barring NaN). That gcc sometimes does is a compiler bug, not a float problem.
  - cellularmitosis 10y ago
    I agree. Explaining that assertion requires explaining that the x87 FPU is providing a leaky abstraction. If you ask for double math, it gives you a result which is a long double while it resides in a register, and then becomes a double when moved to RAM. There are some cases where direct comparison of floating point values is what you want, and this issue can still bite you even then.
    - int_19h 10y ago
      For x87, it's not a leaky abstraction - it fully supports 80-bit floats as a type, and documents that its registers are 80-bit, and so are operations on them. There's nothing abstract about it. The bug is that GCC is utilizing the (documented, by-design) x87 behavior in a way that causes it to produce the wrong result, for the sake of performance.
      - abstractbeliefs 10y ago
        >compiler developers do unintuitive but technically legal thing in the name of performance I still think the relationship between compiler developers and developer-users is toxic as fuck, but as long as we keep comparing compilers by increasingly anal microbenchmarks instead of increasingly anal "correct in practice" tests, I don't see why we continue to find this a surprise. When we demand better performance, compiler developers are going to exploit undefined behavior as well as other language "loopholes" that don't make intuitive sense. Perhaps a new focus on correctness in practice would change this.
pkaye 10y ago
What would you suggest as an alternative to float?
- CalChris 10y ago
  UNUMs. https://en.wikipedia.org/wiki/Unum_(number_format) https://en.wikipedia.org/wiki/Unum_(number_format)
- dgudkov 10y ago
  128-bit decimal.
yongjik 10y ago
There are numbers that are safe to compare against. Storing 1.0 (or 0.0 or 0.25 or any small integer), and then later comparing for equality to check "Did I assign this value before?" is completely fine. Of course, it's not fine to ask "Does the result of this computation equal this value?", but everyone knows that.
- jobigoud 10y ago
  > Of course, it's not fine to ask "Does the result of this computation equal this value?", but everyone knows that. How about "Does the result of this computation equal the result of this exact same computation?"? That's the issue here.
fdej 10y ago
That's not the issue here at all. The IEEE 754 standard does in principle make floating-point operations completely deterministic, i.e. if you do a specific sequence of floating-point operations, then the result may indeed be slightly off from the result you would expect with infinite precision arithmetic, but it should be off in just the same way every time. For example (x + 0.1) + 0.1 and x + 0.2 can give different results due to rounding, but the respective result should be constant for a given x regardless of the machine, compiler, or phase of the moon. Knowing this, there are situations where exact comparisons are appropriate. The "bug" here is that a single operation x + 0.1 and x + 0.1 can give two different results depending on the mood of the compiler. It's not actually a bug, because the IEEE 754 standard permits this kind of sloppiness by default. You can get stricter semantics, but you have to ask for it explicitly by setting the right compiler flags. This is why I say that the IEEE 754 standard makes floating-point arithmetic deterministic "in principle" - in practice, you have to work for it. (Changes to the rounding mode are another big issue here.) An extra error of 1 ulp somewhere might not seem like much for your average numerical code, but it's a big deal in some situations: * Assumptions about the precise behavior of floating-point cancellations are often used in numerical libraries, and deviations can cause disastrous loss of accuracy. For example, Kahan summation is a widely used algorithm for computing sums with increased accuracy. If compilers optimize the code too aggressively, the algorithm doesn't work and you get worse accuracy. And ironically, x87 long doubles can make general numerical calculations less accurate (not more accurate as intended) due to the double rounding issue. * If you use floats in game logic, you might want it to behave exactly the same way on different platforms, e.g. for synchronization or replays. * In scientific computing, you might want results to be precisely reproducible by others. * Debugging!
- tbirdz 10y ago
  In practice, I don't think IEEE 754 floating point is deterministic, especially not across multiple ISAs, multiple vendors of the same ISA (Intel, AMD), different generations of the same vendor's cpus, different compilers and different math libraries, even just C's libm. So just a warning to everyone out there: unless you are very, very careful your floating point code will not be deterministic. If you thought you can just use floats and expect it to "just work" the same, then you're out of luck. Here's a couple of articles which go into more detail about problems with floating point determinism: http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/ http://gafferongames.com/networking-for-game-programmers/flo... https://randomascii.wordpress.com/2013/07/16/floating-point-determinism/ https://randomascii.wordpress.com/2013/07/16/floating-point-...
andrepd 10y ago
Floats are evil? o.O What do you propose instead if you want to deal with non-integer values?
- rnhmjoj 10y ago
  Continued fractions are perfect if you have infinite memory.
  - kccqzy 10y ago
    A real number can contain an infinite amount of information. If you have infinite memory, you would probably start by asking whether you have countably infinite or uncountably infinite memory.
- tbirdz 10y ago
  There's always fixed point integers[0]. They're less flexible than floats, but at least the amount of precision available is much more clear and uniform, and all the rounding is pretty explicit. https://en.wikipedia.org/wiki/Fixed-point_arithmetic https://en.wikipedia.org/wiki/Fixed-point_arithmetic https://en.wikipedia.org/wiki/Q_%28number_format%29 https://en.wikipedia.org/wiki/Q_%28number_format%29
  - rwallace 10y ago
    Back in the late eighties, early nineties, I was of the opinion that fixed point is really the way to go, the only thing holding it back was lack of adequate compiler support. Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. The headaches of floating point, annoying though they can be at times, are much less onerous.
    - MaulingMonkey 10y ago
      > Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. Floating point doesn't actually fix this issue. A fun bug I dealt with was a black screen caused by a few pixels having NaN values - which were then picked up by a blur pass which would poison basically the entire back buffer. The original NaNs were caused by divides by near-zero values causing out-of-range results. You now add precision issues: Multiple coordinate systems are a common hack-around to maintain local precision when using single-precision based physics engines. Stuttering effects when time is represented as "floating point seconds since uptime" when integer wraparound would avoid any problem in the first place. And this is before even adding e.g. half precision or worse floats into the mix. There's a reason floating point isn't used for a lot of serious banking calculations.

Pretty much. If you want to compare two floats a and b within some precision eps, use: abs(a - b) <= eps*0.5*(a + b) or even (abs(a - b) <= eps*abs(a)) && (abs(a - b) <= eps*abs(b)) which is implemented in Boost [0]. [0] http://www.boost.org/doc/libs/1_34_0/libs/test/doc/components/test_tools/floating_point_comparison.html http://www.boost.org/doc/libs/1_34_0/libs/test/doc/component...

ratboy666 10y ago
I thought it was (a / b) != (a / b)

geezerjay 10y ago

> More proof that floats are evil. Just because someone doesn't know the basics of floating point numbers it doesn't mean that the floating point format is somehow evil. Testing floats for equality is one of those no-nos which is taught right in the first class of introduction to FP numbers. Another very basic issue covered right in the intro clsses are problems caused by using decimal values to define numbers encoded in binary format, which is often explained through the famous "0.1 doesn't exist in binary" trivia bit.

thechao 10y ago
Part of my day job is to test floating point hardware and write compilers and assemblers. Testing equality---with some care---is completely fine. I can assure you this is a bug in GCC. The GCC bug is compounded by the nonstandard behavior of x87. The x87 unit is not "fixable" but GCC's behavior is.
- nurettin 10y ago
  That's interesting, you make it sound like the production of new FPUs and new compilers is common. Which industry do you work in?

nom 10y ago

Floats are not evil. They are just not as intuitive as integers because it is impossible to represent every possible rational number in the range with a fixed number of bits. Once you get that, everything makes sense.

monochromatic 10y ago
That does not make sense of getting a failure from assert x/y == x/y
- mikeash 10y ago
  Properly implemented floats will never fail that (barring NaN). That gcc sometimes does is a compiler bug, not a float problem.
- cellularmitosis 10y ago
  I agree. Explaining that assertion requires explaining that the x87 FPU is providing a leaky abstraction. If you ask for double math, it gives you a result which is a long double while it resides in a register, and then becomes a double when moved to RAM. There are some cases where direct comparison of floating point values is what you want, and this issue can still bite you even then.
  - int_19h 10y ago
    For x87, it's not a leaky abstraction - it fully supports 80-bit floats as a type, and documents that its registers are 80-bit, and so are operations on them. There's nothing abstract about it. The bug is that GCC is utilizing the (documented, by-design) x87 behavior in a way that causes it to produce the wrong result, for the sake of performance.
    - abstractbeliefs 10y ago
      >compiler developers do unintuitive but technically legal thing in the name of performance I still think the relationship between compiler developers and developer-users is toxic as fuck, but as long as we keep comparing compilers by increasingly anal microbenchmarks instead of increasingly anal "correct in practice" tests, I don't see why we continue to find this a surprise. When we demand better performance, compiler developers are going to exploit undefined behavior as well as other language "loopholes" that don't make intuitive sense. Perhaps a new focus on correctness in practice would change this.

pkaye 10y ago

What would you suggest as an alternative to float?

CalChris 10y ago
UNUMs. https://en.wikipedia.org/wiki/Unum_(number_format) https://en.wikipedia.org/wiki/Unum_(number_format)
dgudkov 10y ago
128-bit decimal.

yongjik 10y ago

There are numbers that are safe to compare against. Storing 1.0 (or 0.0 or 0.25 or any small integer), and then later comparing for equality to check "Did I assign this value before?" is completely fine. Of course, it's not fine to ask "Does the result of this computation equal this value?", but everyone knows that.

jobigoud 10y ago
> Of course, it's not fine to ask "Does the result of this computation equal this value?", but everyone knows that. How about "Does the result of this computation equal the result of this exact same computation?"? That's the issue here.

fdej 10y ago

That's not the issue here at all. The IEEE 754 standard does in principle make floating-point operations completely deterministic, i.e. if you do a specific sequence of floating-point operations, then the result may indeed be slightly off from the result you would expect with infinite precision arithmetic, but it should be off in just the same way every time. For example (x + 0.1) + 0.1 and x + 0.2 can give different results due to rounding, but the respective result should be constant for a given x regardless of the machine, compiler, or phase of the moon. Knowing this, there are situations where exact comparisons are appropriate. The "bug" here is that a single operation x + 0.1 and x + 0.1 can give two different results depending on the mood of the compiler. It's not actually a bug, because the IEEE 754 standard permits this kind of sloppiness by default. You can get stricter semantics, but you have to ask for it explicitly by setting the right compiler flags. This is why I say that the IEEE 754 standard makes floating-point arithmetic deterministic "in principle" - in practice, you have to work for it. (Changes to the rounding mode are another big issue here.) An extra error of 1 ulp somewhere might not seem like much for your average numerical code, but it's a big deal in some situations: * Assumptions about the precise behavior of floating-point cancellations are often used in numerical libraries, and deviations can cause disastrous loss of accuracy. For example, Kahan summation is a widely used algorithm for computing sums with increased accuracy. If compilers optimize the code too aggressively, the algorithm doesn't work and you get worse accuracy. And ironically, x87 long doubles can make general numerical calculations less accurate (not more accurate as intended) due to the double rounding issue. * If you use floats in game logic, you might want it to behave exactly the same way on different platforms, e.g. for synchronization or replays. * In scientific computing, you might want results to be precisely reproducible by others. * Debugging!

tbirdz 10y ago
In practice, I don't think IEEE 754 floating point is deterministic, especially not across multiple ISAs, multiple vendors of the same ISA (Intel, AMD), different generations of the same vendor's cpus, different compilers and different math libraries, even just C's libm. So just a warning to everyone out there: unless you are very, very careful your floating point code will not be deterministic. If you thought you can just use floats and expect it to "just work" the same, then you're out of luck. Here's a couple of articles which go into more detail about problems with floating point determinism: http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/ http://gafferongames.com/networking-for-game-programmers/flo... https://randomascii.wordpress.com/2013/07/16/floating-point-determinism/ https://randomascii.wordpress.com/2013/07/16/floating-point-...

andrepd 10y ago

Floats are evil? o.O What do you propose instead if you want to deal with non-integer values?

rnhmjoj 10y ago
Continued fractions are perfect if you have infinite memory.
- kccqzy 10y ago
  A real number can contain an infinite amount of information. If you have infinite memory, you would probably start by asking whether you have countably infinite or uncountably infinite memory.
tbirdz 10y ago
There's always fixed point integers[0]. They're less flexible than floats, but at least the amount of precision available is much more clear and uniform, and all the rounding is pretty explicit. https://en.wikipedia.org/wiki/Fixed-point_arithmetic https://en.wikipedia.org/wiki/Fixed-point_arithmetic https://en.wikipedia.org/wiki/Q_%28number_format%29 https://en.wikipedia.org/wiki/Q_%28number_format%29
- rwallace 10y ago
  Back in the late eighties, early nineties, I was of the opinion that fixed point is really the way to go, the only thing holding it back was lack of adequate compiler support. Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. The headaches of floating point, annoying though they can be at times, are much less onerous.
  - MaulingMonkey 10y ago
    > Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. Floating point doesn't actually fix this issue. A fun bug I dealt with was a black screen caused by a few pixels having NaN values - which were then picked up by a blur pass which would poison basically the entire back buffer. The original NaNs were caused by divides by near-zero values causing out-of-range results. You now add precision issues: Multiple coordinate systems are a common hack-around to maintain local precision when using single-precision based physics engines. Stuttering effects when time is represented as "floating point seconds since uptime" when integer wraparound would avoid any problem in the first place. And this is before even adding e.g. half precision or worse floats into the mix. There's a reason floating point isn't used for a lot of serious banking calculations.