3 ms·
Trying out Raymond's suggestion[½] to test arithmetic precision (1 / 3 * 10000000000 - 3333333333 =. The result is one third exactly. Type 1/x - 3 = and you get
by ygra 10y ago
Trying out Raymond's suggestion[½] to test arithmetic precision (1 / 3 * 10000000000 - 3333333333 =. The result is one third exactly. Type 1/x - 3 = and you get zero back) yields the exact result with calc on Windows 10. It's definitely not standard floating-point.
[½] from here: https://blogs.msdn.microsoft.com/oldnewthing/20040525-00/?p=39193 https://blogs.msdn.microsoft.com/oldnewthing/20040525-00/?p=...
- exDM69 10y agoMust have been something else but I recall a big fuss over incorrect results from calc.exe when Win10 came out. It's still inferior to the old calculator even if it has been fixed to give correct results.