5 ms·

> Just consider that with 64-bit floats 0.3 + 0.6 = 8.9999999999999991. > It really took no effort at all to get a precision problem! That sure looks like ser

by ucho 11y ago

> Just consider that with 64-bit floats 0.3 + 0.6 = 8.9999999999999991. > It really took no effort at all to get a precision problem! That sure looks like serious precision problem :)

mortoray 11y ago
I suppose in hindsight my example doesn't look as disturbing as it's meant to. :) It's hard to show concise examples that truly show how this little bit of error eventually compounds and destabilizes a calculation.
- jobigoud 11y ago
  I think he is referring to the fact that the result should be around 0.8999, not 8.9999.
- jacquesm 11y ago
  It looks plenty disturbing to me. I don't even think it should be qualified as a 'precision problem' it's more like a 'total failure'.