5 ms·

It's not so easy because -ffast-math can actually be more correct than running without.

adrian_b 2y ago
In most cases it is unpredictable when such an improvement happens and what is unpredictable about correctness is useless. It is much more frequent for -ffast-math to generate bigger errors than to benefit from an accidental cancellation of the errors. The only case when the result of underflows is predictable is when the values whose computation may result in an underflow are immediately added to other values that are known to be big enough, and they are not used for anything else. It is not frequent to have so much information about which expressions will underflow and which will not, and about the range of possible values of the values that would be added. When there really exists so much information about what must be computed, then there are chances that the computation can be rearranged in a way that would avoid the underflows, in which case there would be no need to change MXCSR to ignore the underflows, because they would not happen.
- bjourne 2y ago
  > It is much more frequent for -ffast-math to generate bigger errors than to benefit from an accidental cancellation of the errors. In my experience underflows are very uncommon and indicate that you're doing something wrong, whereas fp contractions are extremely common. So I disagree with you.
  - mbauman 2y ago
    fp contract isn't a uniform "decrease in error" either, though. As a simple example, it introduces error in the straightforward: a*b - a*b
  - adrian_b 2y ago
    I agree that using fused multiply-add in almost all cases improves the accuracy of the results. Nevertheless, I consider that a compiler option that gives you good FMA, but only together with bad behavior at underflows, is something exceedingly stupid. FMA is a standard operation, while ignoring underflows is rightly prohibited by the floating-point standard. There should never have been a compiler option that mixes standard operations with non-standard operations. In C or C++, you can use the "fma" function to get FMA where desired. This is more awkward, but it is safe. Any better solution requires a change in the programming language. For instance, it could be required that adjacent * and + must always be contracted into an FMA, unless separated by parentheses. This would allow the programmer to select between FMA and FMUL+FADD, as desired. FMA has appeared in 1990 (in IBM POWER) and it has been the greatest advance in floating-point computation after the introduction of Intel 8087 in 1980 (which has resulted in the IEEE standard a few years later). It is weird that 34 years later, most, if not all, programming languages do not handle well the generation of FMA operations.

In most cases it is unpredictable when such an improvement happens and what is unpredictable about correctness is useless. It is much more frequent for -ffast-math to generate bigger errors than to benefit from an accidental cancellation of the errors. The only case when the result of underflows is predictable is when the values whose computation may result in an underflow are immediately added to other values that are known to be big enough, and they are not used for anything else. It is not frequent to have so much information about which expressions will underflow and which will not, and about the range of possible values of the values that would be added. When there really exists so much information about what must be computed, then there are chances that the computation can be rearranged in a way that would avoid the underflows, in which case there would be no need to change MXCSR to ignore the underflows, because they would not happen.

bjourne 2y ago
> It is much more frequent for -ffast-math to generate bigger errors than to benefit from an accidental cancellation of the errors. In my experience underflows are very uncommon and indicate that you're doing something wrong, whereas fp contractions are extremely common. So I disagree with you.
- mbauman 2y ago
  fp contract isn't a uniform "decrease in error" either, though. As a simple example, it introduces error in the straightforward: a*b - a*b
- adrian_b 2y ago
  I agree that using fused multiply-add in almost all cases improves the accuracy of the results. Nevertheless, I consider that a compiler option that gives you good FMA, but only together with bad behavior at underflows, is something exceedingly stupid. FMA is a standard operation, while ignoring underflows is rightly prohibited by the floating-point standard. There should never have been a compiler option that mixes standard operations with non-standard operations. In C or C++, you can use the "fma" function to get FMA where desired. This is more awkward, but it is safe. Any better solution requires a change in the programming language. For instance, it could be required that adjacent * and + must always be contracted into an FMA, unless separated by parentheses. This would allow the programmer to select between FMA and FMUL+FADD, as desired. FMA has appeared in 1990 (in IBM POWER) and it has been the greatest advance in floating-point computation after the introduction of Intel 8087 in 1980 (which has resulted in the IEEE standard a few years later). It is weird that 34 years later, most, if not all, programming languages do not handle well the generation of FMA operations.