3 ms·
9,999,999,999,999,978 and 1 – 2 incorrect on iOS Calculator
A few observations (landscape mode only):
* The above results in 9,999,999,999,999,978.
* Add 1 a bunch of times and subtract 2 once to reach the same number.
* Add 1 a bunch of times and subtract 1 a bunch of times and the answer is still displayed in scientific notation, until 2 is subtracted, at which point the above number is displayed.
- pwg 5y agoYou should read this: What Every Computer Scientist Should Know About Floating-Point Arithmetic https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...
- yorwba 5y agoThen learn how to do better: https://cacm.acm.org/magazines/2017/8/219594-small-data-computing/fulltext https://cacm.acm.org/magazines/2017/8/219594-small-data-comp... (The Android calculator app uses exact arithmetic on arbitrary-precision real numbers.)
- fsflyer 5y agoMicrosoft open sourced the Windows calculator in 2019. Ratpak[0] is their infinite precision math library. [0] https://github.com/microsoft/calculator/tree/master/src/CalcManager/Ratpack https://github.com/microsoft/calculator/tree/master/src/Calc...
- deleted 5y ago[deleted]
- ksaj 5y agoThis reminds me of the Intel FDIV bug. https://en.wikipedia.org/wiki/Pentium_FDIV_bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug As well as the Windows 3.1 Calc.exe bug that would give bizarre results if you subtracted close decimals like 1.111 - 1.011.