4 ms·

I'd be concerned about whether the behavior of such a pattern is actually well defined. Like for instance what if the optimizer is super clever and evaluates yo

by im3w1l 4y ago

I'd be concerned about whether the behavior of such a pattern is actually well defined. Like for instance what if the optimizer is super clever and evaluates your sin(infinity) at compile time? Or what if it undoes your bit-set? Or what if it moves the computation before the bit set?

jcranmer 4y ago
There are essentially three different flavors of floating-point models: strict, precise, and fast. Which one you get is dependent primarily on compiler flags, although most compilers will default to "precise" (a few default to "fast", but they really shouldn't). The strict model requires that everything be evaluated at runtime and that the optimizer be aware of the potential for floating-point operations to have interactions with floating-point control and status words (i.e., rounding modes and fp exceptions)--essentially all optimizations are floating-point are forbidden. The precise model allows compilers to ignore the dependency on rounding mode and fp exceptions, and treat floating-point optimizations as side-effect-free pure expressions, but still requires them to generally follow IEEE 754 arithmetic. As such, standard optimizations like code motion or constant propagation are legal, but arithmetic optimizations (like replacing x + 0.0 with x) are not generally legal. The fast model allows compilers to also apply the arithmetic optimizations. Most people who actually care about this level of detail are aware of the panoply of compiler options that are used to control floating-point optimization in the compiler.
- nextaccountic 4y ago
  How can replacing x + 0.0 with x not be legal??
  - jcranmer 4y ago
    x + 0.0 is not the same as x when x is -0.0: -0.0 + 0.0 is +0.0, which is not the same as -0.0, though they do compare equal.
    - eesmith 4y ago
      I decided to test every non-NaN 32-bit float, and confirmed that's the only such case. Edit: Hee-hee! And if I put my source into Godbolt's Compiler Explorer, compile with gcc 12.2 -O3 -ffast-math then my program turns into: main: xor eax, eax ret
      - jcranmer 4y ago
        The other case it doesn't work is that x + 0.0 is defined to quiet an sNaN (because almost every fp operation turns sNaN to qNaN)... except even generally strict floating-point models generally say "fuck you" to sNaN and treat sNaN and qNaN as interchangeable. (Speaking of fast math, my planned blog post title if I ever write all this up into a blog post is "Floating Point, or How I Learned to Start Worrying and Hate Fast Math.")
        eesmith 4y ago
        I deliberately added a !isnan() to my tests. There be dragons there. ;) More precisely, my test compared the bit-patterns directly: i = 0 .. 2^32-1 memcpy(&x, &i, 4); y = x + 0.0f; memcpy(&j, &y, 4) if (i != j) {printf("%f + 0.0 -> %f\n", x, y); and I'm pretty sure that NaN addition, while resulting in a NaN, does not need to preserve the original bit pattern. Regarding fast-not-so-math - Praise Kahan!
        jcranmer 4y ago
        > and I'm pretty sure that NaN addition, while resulting in a NaN, does not need to preserve the original bit pattern. "it be complicated" Except for certain operations (fneg, fcopysign, and fabs are all defined to only affect the sign bit, to the point that they're an exception to the sNaN-always-signals rule; and there are a few operations defined to operate on the NaN payload information), IEEE 754 generally says no more than a qNaN is generated. There's a recommendation that NaN payloads propagate, and most hardware does do that, but compilers actually typically don't even guarantee that sNaNs are not produced by operations.

by im3w1l 4y ago

jcranmer 4y ago

There are essentially three different flavors of floating-point models: strict, precise, and fast. Which one you get is dependent primarily on compiler flags, although most compilers will default to "precise" (a few default to "fast", but they really shouldn't). The strict model requires that everything be evaluated at runtime and that the optimizer be aware of the potential for floating-point operations to have interactions with floating-point control and status words (i.e., rounding modes and fp exceptions)--essentially all optimizations are floating-point are forbidden. The precise model allows compilers to ignore the dependency on rounding mode and fp exceptions, and treat floating-point optimizations as side-effect-free pure expressions, but still requires them to generally follow IEEE 754 arithmetic. As such, standard optimizations like code motion or constant propagation are legal, but arithmetic optimizations (like replacing x + 0.0 with x) are not generally legal. The fast model allows compilers to also apply the arithmetic optimizations. Most people who actually care about this level of detail are aware of the panoply of compiler options that are used to control floating-point optimization in the compiler.

nextaccountic 4y ago
How can replacing x + 0.0 with x not be legal??
- jcranmer 4y ago
  x + 0.0 is not the same as x when x is -0.0: -0.0 + 0.0 is +0.0, which is not the same as -0.0, though they do compare equal.
  - eesmith 4y ago
    I decided to test every non-NaN 32-bit float, and confirmed that's the only such case. Edit: Hee-hee! And if I put my source into Godbolt's Compiler Explorer, compile with gcc 12.2 -O3 -ffast-math then my program turns into: main: xor eax, eax ret
    - jcranmer 4y ago
      The other case it doesn't work is that x + 0.0 is defined to quiet an sNaN (because almost every fp operation turns sNaN to qNaN)... except even generally strict floating-point models generally say "fuck you" to sNaN and treat sNaN and qNaN as interchangeable. (Speaking of fast math, my planned blog post title if I ever write all this up into a blog post is "Floating Point, or How I Learned to Start Worrying and Hate Fast Math.")
      - eesmith 4y ago
        I deliberately added a !isnan() to my tests. There be dragons there. ;) More precisely, my test compared the bit-patterns directly: i = 0 .. 2^32-1 memcpy(&x, &i, 4); y = x + 0.0f; memcpy(&j, &y, 4) if (i != j) {printf("%f + 0.0 -> %f\n", x, y); and I'm pretty sure that NaN addition, while resulting in a NaN, does not need to preserve the original bit pattern. Regarding fast-not-so-math - Praise Kahan!
        jcranmer 4y ago
        > and I'm pretty sure that NaN addition, while resulting in a NaN, does not need to preserve the original bit pattern. "it be complicated" Except for certain operations (fneg, fcopysign, and fabs are all defined to only affect the sign bit, to the point that they're an exception to the sNaN-always-signals rule; and there are a few operations defined to operate on the NaN payload information), IEEE 754 generally says no more than a qNaN is generated. There's a recommendation that NaN payloads propagate, and most hardware does do that, but compilers actually typically don't even guarantee that sNaNs are not produced by operations.