3 ms·
I'd love to see an in depth exploration of exactly how float imprecision happens. Why is the error at 4e-10 and not 3? What binary magic is causing these spurio
by rolenthedeep 4y ago
I'd love to see an in depth exploration of exactly how float imprecision happens. Why is the error at 4e-10 and not 3? What binary magic is causing these spurious results?
We've all been told about float imprecision in a very hand-wavy way that doesn't actually explain anything at all about the problem. On its face, one would assume that any two binary values being added would have a reliable and deterministic value, but that's not how floats work. There's some magic in the stack that causes errors to creep in, which isn't something we see with other types of numerical representation. It's very easy to show and understand that 01b + 01b = 10b, but that logic somehow doesn't apply to floats.
I think it's a very interesting subject and I wish there was more discussion of the real causes rather than just "oh yeah, floats do that sometimes, just ignore it"