4 ms·
MS Excel tries to be clever and disguise the most common places this is noticed. Give it =0.1+0.2-0.3 and it will see what you are trying to do and return 0.
by dspillett 7y ago
MS Excel tries to be clever and disguise the most common places this is noticed.
Give it =0.1+0.2-0.3 and it will see what you are trying to do and return 0.
Give it anything slightly more complicated such as =(0.1+0.2-0.3) and this won't trip, in this example displaying 5.55112E-17 or similar.
- piadodjanho 7y agoAre you sure it is not showing the exact answer because the the the cell precision set to a single decimal digit?
- deleted 7y ago[deleted]
- zingmars 7y agoYup: https://i.imgur.com/VuawaE1.png https://i.imgur.com/VuawaE1.png, on Excel v1911 (Build 12228.20332).
- FabHK 7y agoKahan (architect of IEEE 754) has a nice rant on it: https://people.eecs.berkeley.edu/~wkahan/Mind1ess.pdf https://people.eecs.berkeley.edu/~wkahan/Mind1ess.pdf (and plenty of other rants...: https://people.eecs.berkeley.edu/~wkahan/ https://people.eecs.berkeley.edu/~wkahan/ )