3 ms·

> No more getting slightly different result when changing platform, OS, or even just updating libm Optimizing compilers are usually used with flags that cause

• thxg 5y ago
  Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread. By default at -O3, unfortunately, gcc and icc both take responsibility for deciding whether or not to use FMA instructions when they see (a x b + c) expressions. As a result we can even see slightly different result across compiles!! However, as far as I understand, specifying a proper language standard (for example -std=c99, -std=c++14, etc.) restores sane behavior (see -ffast-math, and more specifically -fexcess-precision). Are there other cases where -O3 breaks IEEE754 on modern setups?
  • jcranmer 5y ago
    Fast math flags are actually far more complicated in C/C++ compilers than just a simple -ffast-math flag, although I suppose most users may not be acquainted with the degree of control possible. In LLVM, for example, fast math flags are actually a set of 7 independently-toggled flags (no-signed-zero, no-infinity, no-NaN, allow FMAs, allow reassociation, allow approximate reciprocal, and allow approximate functions), that at the C/C++ level can be toggled either by command-line options or on a per-scope basis. ICC actually has two different levels of fast math [1]. The dark underside of IEEE 754 is that full support for it also requires features that most languages don't provide access for: rounding mode and exception handling (aka sticky bits). In C/C++, using these features correctly requires STDC FENV_ACCESS support which is only in ICC and recent clang (neither GCC nor MSVC support it). And there's usually two hardware bits for handling denormals (flush-to-zero and denormals-are-zero) that aren't in IEEE 754, which tend to be on by default when you're compiling for some targets (most notably GPUs). [1] https://software.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top/compiler-reference/compiler-options/compiler-option-details/floating-point-options/fp-model-fp.html https://software.intel.com/content/www/us/en/develop/documen... -- note fp-model=fast=[1|2]
    • BeeOnRope 5y ago
      What is a "per scope basis"?
      • jcranmer 5y ago
        You can change it between any pair of curly braces. E.g.: void foo(int N) { for (int i = 0; i < N; i++) { #pragma STDC FENV_ACCESS ON // This applies only to the for loop... } // But not here, for example! }
  • wumpus 5y ago
    > Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread This is not always true. Some FMA operations round after the multiply, and some do not.
• titzer 5y ago
  > Optimizing compilers are usually used with flags that cause them to break IEEE754. For languages that don't mandate IEEE-754. There are plenty of languages that do, like Java, JavaScript, C#, and WebAssembly. It's worth noting that usually such optimizations are things like enabling algebraic reassociation and commutation (of primitives like +, -, /, ==, etc) operations, fused-multiply add, and others. It's really not clear to me how much disabling all those optimization costs in the real world. Things get crazy when compilers want to do major reorganization of loops, optimizing stencils, polyhedral optimizations. I am not an expert here, but I think these rely on properties of float arithmetic that don't hold in all cases, i.e. they rely on UB.
  • wumpus 5y ago
    > It's really not clear to me how much disabling all those optimization costs in the real world. I assure you that SPECcpu scores are a lot lower, if that's what you care about. And yes, it's extremely helpful to do major reorganization of loops if the program has multi-dimensional loops in the wrong order.