2 ms·
It's called catastrophic cancellation. Consider the following example: 999999 + 0.0000001 ~= 999999
by xeonmc 3y ago
It's called catastrophic cancellation. Consider the following example:
999999 + 0.0000001 ~= 999999
- justinl33 3y agoWhy doesn't it wrap around?
- nzentzis 3y agoThe problem is that there aren’t enough digits of precision to represent the answer. The exact answer would be 999999.0000001, but let’s say you only have space to store six significant digits. The exact answer is way more precise, so you need to round it to the nearest representable value, 999999. Floating-point rounding issues are the same thing but with binary digits instead of decimal ones.