5 ms·

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitfalls of IEEE floats. Intel chips have had BCD support in machine code; I'm surprised nobody has made a decent fixed point lib that is widely used already.

IEEE754 7y ago
> Most think of it as base-10 [...] I'm surprised nobody has made a decent fixed point lib that is widely used already. Note that fixed radix point does not solve the common issues with representing rational base 10 fractions. A base10 fixed radix solution would, but so would IEEE754's decimal64 spec, which would eliminate representation error when working exclusively in the context of base10 e.g finance, but these are not found in common hardware and do not help reduce propagation of error due to compounding with limited precision in any base.
DannyB2 7y ago
Considering that back in the 1980's we figured out that you should use some kind of 'integer' or 'bcd' based type for financial calculations; it is astonishing that by almost 2020 people are making these same mistakes over and over again. A 64-bit integer is big enough to express the US National Debt in Argentine Pesos.
- toolslive 7y ago
  > it is astonishing that by almost 2020 people are making these same mistakes over and over again. It's actually logical: the number of developers doubles roughly every 5 years. It means that half of the developers have less than 5 years of experience. If they don't teach you this in school (university), you will have to learn from someone who knows, but chances are the other developers are as clueless as you.
  - carapace 7y ago
    OMG! The Eternal Eternal September.
- simonh 7y ago
  It's not enough to represent one US cent in original Zimbabwean dollars though as of 2009. Although those 'first' dollars hadn't been legal for quite some time, the currency having gone through three rounds of massive devaluation and collapse, totalling something like 1x10^30 by then.
rrss 7y ago
Replacing all the IEEE 754 hardware with posits won't fix this, though. If you don't care about performance, then the actual solution has no dependency on hardware: 1. Replace the default format for numbers with a decimal point in suitably high level languages with a infinite precision format. 2. Teach people using other languages about floating point and how they may want to use integers instead. The end. No multi-generation hardware transition required. IMO, IEEE 754 is an exceptionally good format. It has real problems, but they aren't widely known to people unfamiliar with floats (e.g. 1.0 + 2.0 != 3.0 isn't one of them).
- dmd 7y ago
  I'm pretty sure that 1.0 + 2.0 == 3.0 in IEEE 754. :) Now, 0.1 ...
  - ethbro 7y ago
    For reference to what they're talking about, the helpful https://0.30000000000000004.com/ https://0.30000000000000004.com/
  - rrss 7y ago
    Yep, sorry, meant to put 0.1 ... Thanks!
  - mark-r 7y ago
    One of the great qualities of IEEE 754 is its ability to represent many integers and operate on them without rounding errors.
  - piadodjanho 7y ago
    They are not the same thing, but they are close enough most of the time. The issue with floating point arrises when comparing very close numbers.
    - dmd 7y ago
      whooosh
PaulHoule 7y ago
The use of binary in the numerator is not the problem with using floats for financial math, it is the use of powers of 2 in the denominator. The numerator is just an integer and integers are just integers and the base doesn't matter. But if the exponent is base 2, then you can have 1/2, 1/4, 1/8 on the base but not 1/5 or 1/10.
piadodjanho 7y ago
In the paper "Do Developers Understand IEEE Floating Point" the authors surveys phd student and faculty members from computer science and found out that your observation is true: most people don't know how fp works. They have the survey online at [1] in case you want to see how much you know about fp behavior. [1] http://presciencelab.org/float http://presciencelab.org/float
- prirun 7y ago
  That was a waste of time: 6 pages of questions, and instead of "grading" it and letting me know where I stand on FP, it says "Thanks for the gift of your time".
  - antonyme 7y ago
    Likewise! FYI the actual paper is https://ieeexplore.ieee.org/document/8425212 https://ieeexplore.ieee.org/document/8425212 and you can get the PDF here: http://pdinda.org/Papers/ipdps18.pdf http://pdinda.org/Papers/ipdps18.pdf
    - ekimekim 7y ago
      Interestingly, they got one of their own answers wrong. The question is: if a and b are numbers, it is always the case that (a + b) == (b + a) and their notes on it: Is a simple statement involving the commutativity over addition true for floating point? Generally, floating point arithmetic follows the same commutativity laws as real number arithmetic. They make it clear in their notes that "are numbers" includes infinities but not NaNs. Now consider the case where a = inf and b = -inf. Then inf + (-inf) is NaN, and (-inf) + inf is NaN, and NaN != NaN. >>> a = float('inf') >>> b = float('-inf') >>> a + b == b + a False
      - piadodjanho 7y ago
        Nice catch. > They make it clear in their notes that "are numbers" includes infinities but not NaNs. This is definitely not very clear on the form, thought.
microcolonel 7y ago
Rational numbers are a good way to do many financial calculations (extra points for doing something useful with negative denominators!), since many financial calculations are specified with particular denominators (per cent, per mille, basis points; halves, sixths, twelfths, twenty-sixths, fifty-seconds of a basis point, etc.). However, as soon as you start doing anything interesting, you have limited precision as a matter of course.
mcv 7y ago
If there's one thing I've always really appreciated about Groovy, it's that it used BigDecimal as the default for fractions, because 9 times out of 10, you need accuracy more than you need high performance and large exponents (and if you do need high performance, you wouldn't be using Groovy anyway). Sadly most languages don't support something like that out of the box.
ScottFree 7y ago
> Very few developers truly understand floating point representation. Where would one go to better understand how floating points are represented?
- kps 7y ago
  “What every computer scientist should know about floating point” by David Goldberg. Readily available free online.
  - ScottFree 7y ago
    Thanks!
piadodjanho 7y ago
Sorry, but you actually sounds like one those people who dosen't really know how FP work. > I shudder to think how many e-commerce sites use `float` for financial transactions! The float IEEE-754 represent up to 9 decimal digits (or 23 binary digits) with precision. The double, represent 17 decimal digits. The error upper bound is (0.00000000000000001)/2 per operation. Likely irrelevant for most e-commerce. Also, the database stores in currency values using fixed point. > Intel chips have had BCD support in machine code BCD is floating point encoding not fixed point. AFAIK, only Intel supports it and very precariously. > I'm surprised nobody has made a decent fixed point lib that is widely used already. Nonsense. If you do any scientific computation you have likely have Boost, GMP, MPFR installed in your system. They support arbitrary precision arithmetic with integer (aka fixed point), rational and floating point.
- antonyme 7y ago
  > Sorry, but you actually sounds like one those people who dosen't really know how FP work. LOL, sure ok. Worked on banking systems for 2 years and been doing scientific computing for many more. Pretty comfortable with fixed and floating point. > [error bounds] Likely irrelevant for most e-commerce. Those bounds are theoretical, and there are plenty of occasions I have come across in the past when rounding errors were observed. It was forbidden in the bank to use floating point! We went to enormous lengths to ensure numerical accuracy and stability across systems. I think this article has a pretty good explanation: https://dzone.com/articles/never-use-float-and-double-for-monetary-calculatio https://dzone.com/articles/never-use-float-and-double-for-mo... > Nonsense. If you do any scientific computation you have likely have Boost, GMP, MPFR installed in your system. They support arbitrary precision arithmetic with integer (aka fixed point), rational and floating point. Yes, absolutely right; I have used several of those 3rd party libs myself, as well as hand-rolling fixed point code (esp for embedded systems). I didn't write what I intended. I meant to say that very few languages have first-class fixed point in their standard library. So long as the simple `float` is available as a POD, people will (mis-)use it. I think in a general purpose HLL, a fixed decimal type should be the default, and you should have to opt in to IEEE-754 floating point.
  - 7y ago

IEEE754 7y ago

> Most think of it as base-10 [...] I'm surprised nobody has made a decent fixed point lib that is widely used already. Note that fixed radix point does not solve the common issues with representing rational base 10 fractions. A base10 fixed radix solution would, but so would IEEE754's decimal64 spec, which would eliminate representation error when working exclusively in the context of base10 e.g finance, but these are not found in common hardware and do not help reduce propagation of error due to compounding with limited precision in any base.

DannyB2 7y ago

Considering that back in the 1980's we figured out that you should use some kind of 'integer' or 'bcd' based type for financial calculations; it is astonishing that by almost 2020 people are making these same mistakes over and over again. A 64-bit integer is big enough to express the US National Debt in Argentine Pesos.

toolslive 7y ago
> it is astonishing that by almost 2020 people are making these same mistakes over and over again. It's actually logical: the number of developers doubles roughly every 5 years. It means that half of the developers have less than 5 years of experience. If they don't teach you this in school (university), you will have to learn from someone who knows, but chances are the other developers are as clueless as you.
- carapace 7y ago
  OMG! The Eternal Eternal September.
simonh 7y ago
It's not enough to represent one US cent in original Zimbabwean dollars though as of 2009. Although those 'first' dollars hadn't been legal for quite some time, the currency having gone through three rounds of massive devaluation and collapse, totalling something like 1x10^30 by then.

rrss 7y ago

Replacing all the IEEE 754 hardware with posits won't fix this, though. If you don't care about performance, then the actual solution has no dependency on hardware: 1. Replace the default format for numbers with a decimal point in suitably high level languages with a infinite precision format. 2. Teach people using other languages about floating point and how they may want to use integers instead. The end. No multi-generation hardware transition required. IMO, IEEE 754 is an exceptionally good format. It has real problems, but they aren't widely known to people unfamiliar with floats (e.g. 1.0 + 2.0 != 3.0 isn't one of them).

dmd 7y ago
I'm pretty sure that 1.0 + 2.0 == 3.0 in IEEE 754. :) Now, 0.1 ...
- ethbro 7y ago
  For reference to what they're talking about, the helpful https://0.30000000000000004.com/ https://0.30000000000000004.com/
- rrss 7y ago
  Yep, sorry, meant to put 0.1 ... Thanks!
- mark-r 7y ago
  One of the great qualities of IEEE 754 is its ability to represent many integers and operate on them without rounding errors.
- piadodjanho 7y ago
  They are not the same thing, but they are close enough most of the time. The issue with floating point arrises when comparing very close numbers.
  - dmd 7y ago
    whooosh

PaulHoule 7y ago

The use of binary in the numerator is not the problem with using floats for financial math, it is the use of powers of 2 in the denominator. The numerator is just an integer and integers are just integers and the base doesn't matter. But if the exponent is base 2, then you can have 1/2, 1/4, 1/8 on the base but not 1/5 or 1/10.

piadodjanho 7y ago

In the paper "Do Developers Understand IEEE Floating Point" the authors surveys phd student and faculty members from computer science and found out that your observation is true: most people don't know how fp works. They have the survey online at [1] in case you want to see how much you know about fp behavior. [1] http://presciencelab.org/float http://presciencelab.org/float

prirun 7y ago
That was a waste of time: 6 pages of questions, and instead of "grading" it and letting me know where I stand on FP, it says "Thanks for the gift of your time".
- antonyme 7y ago
  Likewise! FYI the actual paper is https://ieeexplore.ieee.org/document/8425212 https://ieeexplore.ieee.org/document/8425212 and you can get the PDF here: http://pdinda.org/Papers/ipdps18.pdf http://pdinda.org/Papers/ipdps18.pdf
  - ekimekim 7y ago
    Interestingly, they got one of their own answers wrong. The question is: if a and b are numbers, it is always the case that (a + b) == (b + a) and their notes on it: Is a simple statement involving the commutativity over addition true for floating point? Generally, floating point arithmetic follows the same commutativity laws as real number arithmetic. They make it clear in their notes that "are numbers" includes infinities but not NaNs. Now consider the case where a = inf and b = -inf. Then inf + (-inf) is NaN, and (-inf) + inf is NaN, and NaN != NaN. >>> a = float('inf') >>> b = float('-inf') >>> a + b == b + a False
    - piadodjanho 7y ago
      Nice catch. > They make it clear in their notes that "are numbers" includes infinities but not NaNs. This is definitely not very clear on the form, thought.

microcolonel 7y ago

Rational numbers are a good way to do many financial calculations (extra points for doing something useful with negative denominators!), since many financial calculations are specified with particular denominators (per cent, per mille, basis points; halves, sixths, twelfths, twenty-sixths, fifty-seconds of a basis point, etc.). However, as soon as you start doing anything interesting, you have limited precision as a matter of course.

mcv 7y ago

If there's one thing I've always really appreciated about Groovy, it's that it used BigDecimal as the default for fractions, because 9 times out of 10, you need accuracy more than you need high performance and large exponents (and if you do need high performance, you wouldn't be using Groovy anyway). Sadly most languages don't support something like that out of the box.

ScottFree 7y ago

> Very few developers truly understand floating point representation. Where would one go to better understand how floating points are represented?

kps 7y ago
“What every computer scientist should know about floating point” by David Goldberg. Readily available free online.
- ScottFree 7y ago
  Thanks!

piadodjanho 7y ago

Sorry, but you actually sounds like one those people who dosen't really know how FP work. > I shudder to think how many e-commerce sites use `float` for financial transactions! The float IEEE-754 represent up to 9 decimal digits (or 23 binary digits) with precision. The double, represent 17 decimal digits. The error upper bound is (0.00000000000000001)/2 per operation. Likely irrelevant for most e-commerce. Also, the database stores in currency values using fixed point. > Intel chips have had BCD support in machine code BCD is floating point encoding not fixed point. AFAIK, only Intel supports it and very precariously. > I'm surprised nobody has made a decent fixed point lib that is widely used already. Nonsense. If you do any scientific computation you have likely have Boost, GMP, MPFR installed in your system. They support arbitrary precision arithmetic with integer (aka fixed point), rational and floating point.

antonyme 7y ago
> Sorry, but you actually sounds like one those people who dosen't really know how FP work. LOL, sure ok. Worked on banking systems for 2 years and been doing scientific computing for many more. Pretty comfortable with fixed and floating point. > [error bounds] Likely irrelevant for most e-commerce. Those bounds are theoretical, and there are plenty of occasions I have come across in the past when rounding errors were observed. It was forbidden in the bank to use floating point! We went to enormous lengths to ensure numerical accuracy and stability across systems. I think this article has a pretty good explanation: https://dzone.com/articles/never-use-float-and-double-for-monetary-calculatio https://dzone.com/articles/never-use-float-and-double-for-mo... > Nonsense. If you do any scientific computation you have likely have Boost, GMP, MPFR installed in your system. They support arbitrary precision arithmetic with integer (aka fixed point), rational and floating point. Yes, absolutely right; I have used several of those 3rd party libs myself, as well as hand-rolling fixed point code (esp for embedded systems). I didn't write what I intended. I meant to say that very few languages have first-class fixed point in their standard library. So long as the simple `float` is available as a POD, people will (mis-)use it. I think in a general purpose HLL, a fixed decimal type should be the default, and you should have to opt in to IEEE-754 floating point.
- 7y ago