3 ms·

The problem is that we're still stuck with only binary floating point types in our CPUs and compilers and runtime environments, over a decade after ieee754 rele

The problem is that we're still stuck with only binary floating point types in our CPUs and compilers and runtime environments, over a decade after ieee754 released their decimal float spec [1]. Once we finally move away from binary floats, these nasty binary-exponent-decimal-exponent lossy conversions will end (as will the horribly complicated nasty scanf and printf algorithms). This is a solved problem. Our actual problem now is momentum of adoption. I've anticipated that this will eventually happen (hopefully sooner rather than later), and proactively marked binary floats as "legacy formats" [2]. [1] https://en.wikipedia.org/wiki/Decimal64_floating-point_format https://en.wikipedia.org/wiki/Decimal64_floating-point_forma... [2] https://github.com/kstenerud/concise-encoding/blob/master/ce-structure.md#binary-floating-point https://github.com/kstenerud/concise-encoding/blob/master/ce...

apaprocki 5y ago
Many compilers and runtimes support them just fine. CPUs don’t really support them, with the exception of POWER. The dirty little secret (use case measurement), however, is that Intel’s software library on x86-64 is faster than POWER’s instructions. I don’t believe there is any roadmap (e.g., 5 years out) to add hardware support mainly because customers don’t need them or want them (globally speaking) and software is fast enough for majority of use cases.
vintermann 5y ago
I agree that's cool. It's usually sold as for finance applications, for compliance with rounding rules from the pre-binary era etc. But really, any number that lived as a decimal digit string somewhere in its lifetime should probably have been decimal throughout. But I don't think it would have mattered here anyway. With finite precision floating point addition is not going to be associative, decimal or not.
- kstenerud 5y ago
  In practice they don't need to be perfectly associative across all possible values, because most real-world calculations tend to stay within a few orders of magnitude, and don't require more than 10-15 significant digits (even for Earth orbital calculations). Once you need more than that, you'd have to use a multi-word numeric format regardless (same as if your operations started overflowing your largest int type).
jwmerrill 5y ago
Decimal floating point still doesn’t have associative addition or multiplication.
- kstenerud 5y ago
  But it can be shaped by rules to be associative enough to be useful for the common case. All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits (much like you'd guard against overflow on integer types). Once you need more range, you switch to a multi-word numeric format (or impose rounding rules).
  - jwmerrill 5y ago
    > All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits If you have a lot of control of the scale of the numbers you’re computing with, you could use fixed point instead of floating point. The situations that make it useful to have a floating point (rather than fixed point) are also the situations where addition isn’t associative.
    - kstenerud 5y ago
      > If you have a lot of control of the scale of the numbers you’re computing with, you could use fixed point instead of floating point. You could, but why would you go to the extra complication (and ensuing bugs) of adding an implied fixed point to integers when a decimal float type gives it to you for free? As I said, it's not a silver bullet; it's just for the majority case (much like 32 and 64-bit integers are for the majority case when dealing with whole numbers).
jcranmer 5y ago
I predict IPv4 will be widely considered legacy before decimal floating point numbers are common place, let alone binary floats being considered legacy. From what I've seen, there's just simply no demand for decimal floating point; there's basically one or two people who ask for it, and this carries up all the way through to C committee standardization. Far more people care about things like non-default rounding mode or exception support than care about decimal floating point, and that's an area of floating point that itself is pretty damn low on people's priority lists. The advantage of decimal floating points is that it fixes the rounding error when converting to/from decimal strings... and that's it. It doesn't fix non-associativity, it doesn't fix gradual precision loss, it doesn't fix portability issues because different libraries have different precision targets. And you get this for the slight cost of making every single floating point operation slower. And here's the other truism about floating-point: most people who care about it value speed over correctness. There's strong demand for operations like approximate division that don't do the refinement steps, and consequently throw away half the bits. Fast-math flags are similarly pretty common, because users already aren't expecting bit-precise results, so the resulting changes in precision are quite often acceptable for them.
- kstenerud 5y ago
  Except it does fix gradual precision loss, because that loss is incurred by the decimal-to-binary loss that exists in pretty much every human-input value (in text conversion AND in source code). Need to do a bunch of multiplications by 0.2 and 0.3? Well you've lost precision on every operation because those multiplicands weren't actually 0.2 and 0.3 since they got converted to close binary float values. If they where decimal floats, you'd have control over the process because it would be exactly decimal and you could apply standard significant digit rounding with predictable results like we do with calculators. TBH I think the main problem is that people just aren't aware of how binary floats actually work, and instead latch upon the "floats are imprecise" mantra and call it a day. So nothing ever gets fixed because nobody knows to demand it.
  - jcranmer 5y ago
    It fixes gradual precision loss only if you assume every value was perfectly precise to begin with, and if you never ever try to divide (or I guess use any operation other then +, -, and ×). 1 / 3 is still an imprecise number in decimal floating point. The only numbers I'd be reasonably certain of such a precision input would be financial values, and the precision loss still kicks in if you do operations as mundane as averaging values, let alone trying to do some linear algebra (as in doing operations analysis). By the time you get to a lot of scientific or engineering applications, your input numbers are already imprecise to begin with (and a lot of times, they're already coming from something like a 12-bit counter or the like).

apaprocki 5y ago

Many compilers and runtimes support them just fine. CPUs don’t really support them, with the exception of POWER. The dirty little secret (use case measurement), however, is that Intel’s software library on x86-64 is faster than POWER’s instructions. I don’t believe there is any roadmap (e.g., 5 years out) to add hardware support mainly because customers don’t need them or want them (globally speaking) and software is fast enough for majority of use cases.

vintermann 5y ago

I agree that's cool. It's usually sold as for finance applications, for compliance with rounding rules from the pre-binary era etc. But really, any number that lived as a decimal digit string somewhere in its lifetime should probably have been decimal throughout. But I don't think it would have mattered here anyway. With finite precision floating point addition is not going to be associative, decimal or not.

kstenerud 5y ago
In practice they don't need to be perfectly associative across all possible values, because most real-world calculations tend to stay within a few orders of magnitude, and don't require more than 10-15 significant digits (even for Earth orbital calculations). Once you need more than that, you'd have to use a multi-word numeric format regardless (same as if your operations started overflowing your largest int type).

jwmerrill 5y ago

Decimal floating point still doesn’t have associative addition or multiplication.

kstenerud 5y ago
But it can be shaped by rules to be associative enough to be useful for the common case. All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits (much like you'd guard against overflow on integer types). Once you need more range, you switch to a multi-word numeric format (or impose rounding rules).
- jwmerrill 5y ago
  > All you need to do is ensure that your calculations are in no danger of blowing past the max significant digits If you have a lot of control of the scale of the numbers you’re computing with, you could use fixed point instead of floating point. The situations that make it useful to have a floating point (rather than fixed point) are also the situations where addition isn’t associative.
  - kstenerud 5y ago
    > If you have a lot of control of the scale of the numbers you’re computing with, you could use fixed point instead of floating point. You could, but why would you go to the extra complication (and ensuing bugs) of adding an implied fixed point to integers when a decimal float type gives it to you for free? As I said, it's not a silver bullet; it's just for the majority case (much like 32 and 64-bit integers are for the majority case when dealing with whole numbers).

jcranmer 5y ago

I predict IPv4 will be widely considered legacy before decimal floating point numbers are common place, let alone binary floats being considered legacy. From what I've seen, there's just simply no demand for decimal floating point; there's basically one or two people who ask for it, and this carries up all the way through to C committee standardization. Far more people care about things like non-default rounding mode or exception support than care about decimal floating point, and that's an area of floating point that itself is pretty damn low on people's priority lists. The advantage of decimal floating points is that it fixes the rounding error when converting to/from decimal strings... and that's it. It doesn't fix non-associativity, it doesn't fix gradual precision loss, it doesn't fix portability issues because different libraries have different precision targets. And you get this for the slight cost of making every single floating point operation slower. And here's the other truism about floating-point: most people who care about it value speed over correctness. There's strong demand for operations like approximate division that don't do the refinement steps, and consequently throw away half the bits. Fast-math flags are similarly pretty common, because users already aren't expecting bit-precise results, so the resulting changes in precision are quite often acceptable for them.

kstenerud 5y ago
Except it does fix gradual precision loss, because that loss is incurred by the decimal-to-binary loss that exists in pretty much every human-input value (in text conversion AND in source code). Need to do a bunch of multiplications by 0.2 and 0.3? Well you've lost precision on every operation because those multiplicands weren't actually 0.2 and 0.3 since they got converted to close binary float values. If they where decimal floats, you'd have control over the process because it would be exactly decimal and you could apply standard significant digit rounding with predictable results like we do with calculators. TBH I think the main problem is that people just aren't aware of how binary floats actually work, and instead latch upon the "floats are imprecise" mantra and call it a day. So nothing ever gets fixed because nobody knows to demand it.
- jcranmer 5y ago
  It fixes gradual precision loss only if you assume every value was perfectly precise to begin with, and if you never ever try to divide (or I guess use any operation other then +, -, and ×). 1 / 3 is still an imprecise number in decimal floating point. The only numbers I'd be reasonably certain of such a precision input would be financial values, and the precision loss still kicks in if you do operations as mundane as averaging values, let alone trying to do some linear algebra (as in doing operations analysis). By the time you get to a lot of scientific or engineering applications, your input numbers are already imprecise to begin with (and a lot of times, they're already coming from something like a 12-bit counter or the like).