3 ms·
>Many people think floating point numbers are magically precise. Even worse, it kicks in earlier than one might naively think. People are often surprised to h
by RandomOpinion 10y ago
>Many people think floating point numbers are magically precise.
Even worse, it kicks in earlier than one might naively think. People are often surprised to hear a value as simple as 0.1 has no matching floating point representation.
(For those interested, http://www.exploringbinary.com/why-0-point-1-does-not-exist-in-floating-point/ http://www.exploringbinary.com/why-0-point-1-does-not-exist-... has a nicely illustrated explanation.)
- TD-Linux 10y agoBeing unable to represent 0.1 exactly is unrelated to any precision limitation. It's because the quantization of binary and decimal is different.
- greenleafjacob 10y agoThis isn't true with e.g. decimal64.
- astrodust 10y agoThose sorts of numbers aren't floating point, they're typically fixed point.
- SAI_Peregrinus 10y agoNo, that's explicitly a floating point format. It's one of the representations in IEE-754-2008. The three decimal floating point formats are decimal32, decimal64, and decimal128. That people often use them where fixed-point math would work doesn't mean they're not floating point formats.