2 ms·
This is a strange tweet. It assumes people are familiar with classical cancellation error, but not familiar with error analysis. Which in my experience... peopl
by dragontamer 6y ago
This is a strange tweet. It assumes people are familiar with classical cancellation error, but not familiar with error analysis. Which in my experience... people either understand both, or are ignorant of both concepts.
The general point is that "cancellation error" happens more than just in floating-point operations, but also in "classic scientific sig-fig error analysis".
---------
The tweet should either be dumbed down to discuss cancellation error in floating-point arithmetic, or elevated up and assume people know about sig-fig analysis. It sits at a weird point in the "assumed knowledge" curve.
---------
For people unfamiliar with cancellation error, try the two following statements in Python3 (which defaults to double-precision... aka 53-bits of mantissa).
poor_ordering = 9007199254740992.0 + 1.0 + 1.0 + 1.0 + 1.0 - 9007199254740992.0
good_ordering = 9007199254740992.0 - 9007199254740992.0 + 1.0 + 1.0 + 1.0 + 1.0
What are the values of "poor_ordering" vs "good_ordering" ?? What does this tell us about double-precision?
9007199254740992.0 == 2^53. So it is impossible for a double-precision number to accurately represent +/- 1.0 at 2^53. (Note that +/- 2.0 will work out just fine).
Play around with 9007199254740992.0 +/- 1.0, or 2.0, and other values for about 15 minutes, and you'll probably learn everything you need to know about cancellation error from that playtime alone.
Double-precision numbers are composed of 52-explicit bits + 1 implicit bit + 1 sign bit + 11-bit exponent bits (yes, 65-bits total. The implicit bit "doesn't count", but makes 0.0 and subnormal numbers harder to deal with)