3 ms·
I don't think that's quite true. For 0.1,the algorithm for printing back the number will still result in 0.1, but the same is not true for 9999999999999999.0, w
by acjohnson55 8y ago
I don't think that's quite true. For 0.1,the algorithm for printing back the number will still result in 0.1, but the same is not true for 9999999999999999.0, which comes back 1e16. So compilers could easily warn for this specific situation, where what I'd call the round trip value of the literal is broken.
But I don't think such a warning would be all that helpful. How often do we use literals with 16 significant digits, expecting exact representation?
The bigger gotcha here is catastrophic cancelation. This is the issue of an insignificant rounding error becoming much more significant due to subtracting of very nearly equal numbers. You can't generally detect this at compile time if you don't know all your numbers in advance (e.g. you're not working with only literals).
- jmiserez 8y agoYou can do abstract interpretion and calculate rounding precision at each line. The problem is that as soon as you have loops, you'll pretty much get that warning everywhere. Being sure that there is no cancellation is even harder, but possible in some cases. I'm sure there are better approaches, there's tons of research papers in that area.