4 ms·

> The behavior of "-Ofast" where it affects globally the content of MXCSR is unacceptable. > The right behavior would have been for any function compiled with

> The behavior of "-Ofast" where it affects globally the content of MXCSR is unacceptable. > The right behavior would have been for any function compiled with "-Ofast" to save MXCSR, put in it any desired value for the duration of the function execution and restore it at function exit. Moreover, when invoking any other function it should restore the original MXCSR before the call and put again in it the desired local value after the function returns back. I completely agree here. The problem is that introducing floating-point environment into the programming model requires disabling some of the transformations you want to do with fast-math in the first place, so doing it this way kind of doesn't work to enable what you wanted to do. Furthermore, one of the SPEC benchmarks gets like a 30% speedup if you turn on denormal flushing, and that's the kind of change that gives compiler engineer management heartburn if you want to tell them to forgo that speedup because the optimization is dumb. I am grateful for blog posts like this one because this does help build a case for getting the compiler to stop doing stupid stuff like globally enabling DAZ/FTZ bits.

adrian_b 2y ago
Not all CPUs behave badly when handling underflows, denormals or other exceptional cases. On some CPUs the penalties for exceptional cases are negligible (e.g. on AMD Zen), so it is impossible for any SPEC benchmark to get a 30% speedup when ignoring underflows. There are also bad CPU models, which have a microprogrammed handling of the exceptional cases, which can add e.g. a penalty of somewhat more than one hundred CPU cycles for each underflow, which is very similar to the penalty for a load from the main memory (which misses the caches). If one of the SPEC benchmarks has so many underflows that on a bad CPU model it can be sped up by 30% when ignoring the underflows, then it is pretty much guaranteed that when the underflows are ignored the results computed by that benchmark are erroneous. Cheating at the SPEC benchmark by ignoring the underflows could be easily avoided by changing the benchmarking rules, by providing a file with the valid results and by not accepting any benchmark claims where the results do not match those known to be good. You can cheat at any SPEC benchmark with bogus compiler options, if there is no constraint on the compiled program to be correct. For instance you could make a compilation option that deletes 90% of the generated machine code, resulting in a 10 times faster benchmark program. For some reason, the compiler writers that handle the generation of code for floating-point operations feel that they are licensed to generate incorrect code, even when similarly incorrect code would be rejected in other contexts. The fact that the floating-point operations are inexact, so they are accompanied by some inherent errors, does not give the right to a compiler writer or library writer to introduce arbitrary errors in the computations. The reason why a standard exists is precisely for guaranteeing that the errors are bounded and you can predict their effects.

adrian_b 2y ago

Not all CPUs behave badly when handling underflows, denormals or other exceptional cases. On some CPUs the penalties for exceptional cases are negligible (e.g. on AMD Zen), so it is impossible for any SPEC benchmark to get a 30% speedup when ignoring underflows. There are also bad CPU models, which have a microprogrammed handling of the exceptional cases, which can add e.g. a penalty of somewhat more than one hundred CPU cycles for each underflow, which is very similar to the penalty for a load from the main memory (which misses the caches). If one of the SPEC benchmarks has so many underflows that on a bad CPU model it can be sped up by 30% when ignoring the underflows, then it is pretty much guaranteed that when the underflows are ignored the results computed by that benchmark are erroneous. Cheating at the SPEC benchmark by ignoring the underflows could be easily avoided by changing the benchmarking rules, by providing a file with the valid results and by not accepting any benchmark claims where the results do not match those known to be good. You can cheat at any SPEC benchmark with bogus compiler options, if there is no constraint on the compiled program to be correct. For instance you could make a compilation option that deletes 90% of the generated machine code, resulting in a 10 times faster benchmark program. For some reason, the compiler writers that handle the generation of code for floating-point operations feel that they are licensed to generate incorrect code, even when similarly incorrect code would be rejected in other contexts. The fact that the floating-point operations are inexact, so they are accompanied by some inherent errors, does not give the right to a compiler writer or library writer to introduce arbitrary errors in the computations. The reason why a standard exists is precisely for guaranteeing that the errors are bounded and you can predict their effects.