3 ms·
I think this is the first time I've seen a simple, approachable article that makes correct use of machine epsilon (specifically treating it as a _relative_ erro
by tomxor 4y ago
I think this is the first time I've seen a simple, approachable article that makes correct use of machine epsilon (specifically treating it as a _relative_ error)...
It's unfortunately very common for people to misguidedly treat it as an absolute error which makes no sense if you pause to think how it's derived and what it really means (the limit of the significand window, aka mantissa, i.e 2^-52 for a double)... that and the fact that if it's a constant it cannot possibly be an absolute error when the exponent is separate from the significand. There are libraries that have implemented conditions with macheps as an absolute, it's on my todo list to try convince the maintainers of such libraries that's it wrong, but it's a tricky subject to explain while also being massively underestimated by most, so I don't relish it - combined with the fact that checking against an arbitrarily very small absolute does happen to fix a lot of comparison errors - just not for the reason they think, and not reliably.