2 ms·
>As a side note, Excel also uses floats for currency It's still problematic, but the defaults and handling there avoid some issues. So, for example: Excel: =
by tyingq 5y ago
>As a side note, Excel also uses floats for currency
It's still problematic, but the defaults and handling there avoid some issues. So, for example:
Excel: =1.03-.42 produces 0.61, by default, even if you expand out the digits very far.
Python: 1.03-.42 produces 0.6100000000000001, by default.
- slver 5y agoExcel rounds doubles to 15 digits for display and comparison. The exact precision of doubles is something like 15.6 digits, those remaining 0.6 digits causing some of those examples floating (heh) around.
- okl 5y agoThat depends https://randomascii.wordpress.com/2012/03/08/float-precisionfrom-zero-to-100-digits-2/ https://randomascii.wordpress.com/2012/03/08/float-precision...
- slver 5y agoA lot of these edge cases are about theoretical concerns like "how many digits we need in decimal to represent an exact IEEE binary float". In practice a double is 15.6 digits precise, which Excel rounds to 15 to eliminate some weirdness. In their documentation they do cite their number type as 15 digit precision type. Ergo that's the semantic they've settled on.