5 ms·

Floating Point Demystified, Part 2: Why Doesn't 0.1 and 0.2 == 0.3?

25 points by vog 11y ago

maxxxxx 11y ago
I remember when I ran into this the first time with my C++ code. It took me quite a while to understand what's going on but it was a good learning experience. I don't do graphics anymore so not much floating point needed but I am still not sure how to do floating point comparisons in a stable and efficient way.
- deleted 11y ago
  [deleted]
- noobermin 11y ago
  This[0] is an example of how it's usually done. It might not be "efficient," as a == comparison perhaps, but it's more correct regardless. With FPs in computational stuff, there is usually an associated scale with a calculation, so you use that scale (rtol=0, atol=scale). Other times, it makes sense for something like abs(a-b)/(avg(abs(a),abs(b)). [0] http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.isclose.html http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/n...
  - maxxxxx 11y ago
    Are there any built-in functions that do floating point comparisons with rounding? Calling abs many times seems incredibly inefficient.
    - nitrogen 11y ago
      It seems that abs should be as simple as ANDing with a bitmask that has the sign bit set to 0 for IEEE floating point.
      - blt 11y ago
        It is, floating point abs is practically free
- dbcurtis 11y ago
  if (abs(a-b) < FUZZ) { }
natosaichek 11y ago
I look forward to the end of the float / double. Having variable-sized mantissa and exponent fields, along with error tracking, we'll get rid of that baloney for real. http://sites.ieee.org/scv-cs/files/2013/03/Right-SizingPrecision1.pdf http://sites.ieee.org/scv-cs/files/2013/03/Right-SizingPreci... http://www.amazon.com/The-End-Error-Computing-Computational/dp/1482239868 http://www.amazon.com/The-End-Error-Computing-Computational/...
- marcosdumay 11y ago
  Ok, so tell me, what is the "right size" that will make 0.1 + 0.2 == 0.3? Because, in binary, I don't think you'll be able to answer.
  - xaduha 11y ago
    Computers are here to help us. Creating abstractions is the name of the game. "Technically correct" isn't the kind of correct we, as humans, want here. Plenty of languages get it right the way you would naively expect already, as shown here http://0.30000000000000004.com http://0.30000000000000004.com
    - marcosdumay 11y ago
      That's a completely different problem, and really, I have no idea why it cycling on the comments. Yes, some languages print floating point numbers in a friendly way by default (really if you are going with the default format, anything will do). In what way does it change equality tests?
      - xaduha 11y ago
        No only print, but store too. If the result is mathematically accurate, then what does it matter how it works inside? That's why we use abstractions, for convenience. Yes, there are trade-offs, but that's the price we should be willing to pay. https://en.wikipedia.org/wiki/DWIM https://en.wikipedia.org/wiki/DWIM
xaduha 11y ago
~> perl6 -e "say(0.1 + 0.2)" 0.3 ~> nqp -e "say(0.1 + 0.2)" 0.3 ~> perl6 -e "say (0.1 + 0.2).WHAT" (Rat) http://doc.perl6.org/type/Rat http://doc.perl6.org/type/Rat
- raiph 11y ago
  From the article: $ lua Lua 5.3.2 Copyright (C) 1994-2015 Lua.org, PUC-Rio > print(0.1 + 0.2) 0.3 (For completely different reasons. I think it's worth being explicit that NQP and Perl 6 get the right result because they are doing this calculation with exactly correct numbers and operations rather than "using printf() with a %.14g format string".)
_mikz 11y ago
http://0.30000000000000004.com/ http://0.30000000000000004.com/
pdpi 11y ago
Binary can't represent 1/5 finitely, so binary floating point will always feel weird around things that we can represent fine in decimal, but can't be represented finitely in binary. For comparison, you can't do anything about representing 1/3 + 1/7 accurately in finite decimal representations
- dbcurtis 11y ago
  somebody gets it
- lokedhs 11y ago
  Common Lisp is one of the few languages that get this right: (+ 1/3 1/7) → 10/21
TazeTSchnitzel 11y ago
Lua's precision-discarding behaviour is unfortunately copied by PHP here. $ php -r 'var_dump(1.00000000000002, 1.00000000000001, 1.00000000000001 === 1.00000000000002);' float(1) float(1) bool(false) I'm trying to change that, though: http://news.php.net/php.internals/91363 http://news.php.net/php.internals/91363
TazeTSchnitzel 11y ago
> How to calculate this “shortest unambiguous string” efficiently is more tricky than you might expect. In fact the best known algorithm for calculating it was published only in 2010, in the paper Printing Floating-Point Numbers Quickly and Accurately with Integers That's no longer the state-of-the-art. The latest approach - and possibly the end of the road for research her - is Andrysco et al.: http://cseweb.ucsd.edu/~lerner/papers/fp-printing-popl16.pdf http://cseweb.ucsd.edu/~lerner/papers/fp-printing-popl16.pdf It's been on the front page previously: https://news.ycombinator.com/item?id=10915182 https://news.ycombinator.com/item?id=10915182
- haberman 11y ago
  Thanks for letting me know -- I'll update the article!
- StefanKarpinski 11y ago
  The Andrysco et al. paper and algorithm is excellent. The only obvious improvement would be, in cases where there is more than one shortest unambiguous string, printing the one which is closest to the true floating-point value, which I don't believe the Errol algorithm does. But that's getting into serious hair-splitting territory.

25 points by vog 11y ago

I remember when I ran into this the first time with my C++ code. It took me quite a while to understand what's going on but it was a good learning experience. I don't do graphics anymore so not much floating point needed but I am still not sure how to do floating point comparisons in a stable and efficient way.

deleted 11y ago
[deleted]
noobermin 11y ago
This[0] is an example of how it's usually done. It might not be "efficient," as a == comparison perhaps, but it's more correct regardless. With FPs in computational stuff, there is usually an associated scale with a calculation, so you use that scale (rtol=0, atol=scale). Other times, it makes sense for something like abs(a-b)/(avg(abs(a),abs(b)). [0] http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.isclose.html http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/n...
- maxxxxx 11y ago
  Are there any built-in functions that do floating point comparisons with rounding? Calling abs many times seems incredibly inefficient.
  - nitrogen 11y ago
    It seems that abs should be as simple as ANDing with a bitmask that has the sign bit set to 0 for IEEE floating point.
    - blt 11y ago
      It is, floating point abs is practically free
dbcurtis 11y ago
if (abs(a-b) < FUZZ) { }

natosaichek 11y ago

I look forward to the end of the float / double. Having variable-sized mantissa and exponent fields, along with error tracking, we'll get rid of that baloney for real. http://sites.ieee.org/scv-cs/files/2013/03/Right-SizingPrecision1.pdf http://sites.ieee.org/scv-cs/files/2013/03/Right-SizingPreci... http://www.amazon.com/The-End-Error-Computing-Computational/dp/1482239868 http://www.amazon.com/The-End-Error-Computing-Computational/...

marcosdumay 11y ago
Ok, so tell me, what is the "right size" that will make 0.1 + 0.2 == 0.3? Because, in binary, I don't think you'll be able to answer.
- xaduha 11y ago
  Computers are here to help us. Creating abstractions is the name of the game. "Technically correct" isn't the kind of correct we, as humans, want here. Plenty of languages get it right the way you would naively expect already, as shown here http://0.30000000000000004.com http://0.30000000000000004.com
  - marcosdumay 11y ago
    That's a completely different problem, and really, I have no idea why it cycling on the comments. Yes, some languages print floating point numbers in a friendly way by default (really if you are going with the default format, anything will do). In what way does it change equality tests?
    - xaduha 11y ago
      No only print, but store too. If the result is mathematically accurate, then what does it matter how it works inside? That's why we use abstractions, for convenience. Yes, there are trade-offs, but that's the price we should be willing to pay. https://en.wikipedia.org/wiki/DWIM https://en.wikipedia.org/wiki/DWIM

xaduha 11y ago

~> perl6 -e "say(0.1 + 0.2)" 0.3 ~> nqp -e "say(0.1 + 0.2)" 0.3 ~> perl6 -e "say (0.1 + 0.2).WHAT" (Rat) http://doc.perl6.org/type/Rat http://doc.perl6.org/type/Rat

raiph 11y ago
From the article: $ lua Lua 5.3.2 Copyright (C) 1994-2015 Lua.org, PUC-Rio > print(0.1 + 0.2) 0.3 (For completely different reasons. I think it's worth being explicit that NQP and Perl 6 get the right result because they are doing this calculation with exactly correct numbers and operations rather than "using printf() with a %.14g format string".)

_mikz 11y ago

http://0.30000000000000004.com/ http://0.30000000000000004.com/

pdpi 11y ago

Binary can't represent 1/5 finitely, so binary floating point will always feel weird around things that we can represent fine in decimal, but can't be represented finitely in binary. For comparison, you can't do anything about representing 1/3 + 1/7 accurately in finite decimal representations

dbcurtis 11y ago
somebody gets it
lokedhs 11y ago
Common Lisp is one of the few languages that get this right: (+ 1/3 1/7) → 10/21

TazeTSchnitzel 11y ago

Lua's precision-discarding behaviour is unfortunately copied by PHP here. $ php -r 'var_dump(1.00000000000002, 1.00000000000001, 1.00000000000001 === 1.00000000000002);' float(1) float(1) bool(false) I'm trying to change that, though: http://news.php.net/php.internals/91363 http://news.php.net/php.internals/91363

TazeTSchnitzel 11y ago

> How to calculate this “shortest unambiguous string” efficiently is more tricky than you might expect. In fact the best known algorithm for calculating it was published only in 2010, in the paper Printing Floating-Point Numbers Quickly and Accurately with Integers That's no longer the state-of-the-art. The latest approach - and possibly the end of the road for research her - is Andrysco et al.: http://cseweb.ucsd.edu/~lerner/papers/fp-printing-popl16.pdf http://cseweb.ucsd.edu/~lerner/papers/fp-printing-popl16.pdf It's been on the front page previously: https://news.ycombinator.com/item?id=10915182 https://news.ycombinator.com/item?id=10915182

haberman 11y ago
Thanks for letting me know -- I'll update the article!
StefanKarpinski 11y ago
The Andrysco et al. paper and algorithm is excellent. The only obvious improvement would be, in cases where there is more than one shortest unambiguous string, printing the one which is closest to the true floating-point value, which I don't believe the Errol algorithm does. But that's getting into serious hair-splitting territory.