4 ms·

TIL rounding denormals to zero is what -ffast-math actually does.

kibwen 2y ago
Not sure if I should be relieved or concerned that even Ted Unangst doesn't know what -ffast-math actually does. (For the record, it does a whole lot of terrible things: https://stackoverflow.com/questions/7420665/what-does-gccs-ffast-math-actually-do https://stackoverflow.com/questions/7420665/what-does-gccs-f... )
pclmulqdq 2y ago
There's more. It also enables some limited (unsafe) rearrangement of your floating point expressions, as well as a few other settings inside the floating point system. Flushing denorms to 0 is only part of it. It also will do things like using an approximate reciprocal square root instruction plus a refinement iteration instead of fsqrt then fdiv.
CJefferson 2y ago
-ffast-mast does a whole bunch of things, which I wish people's didn't so commonly combine. For example, I think the things it does which are sensible for most people are: * Rounding subnormals to zero * Disabling signed zeroes * Disables support for 'trapping' (throwing SIGFPE) Then there are the 'middle' things, which annoy some people: * Allow associative operations, and things like sqrt(xy)=sqrt(x)sqrt(y), exp(x)*exp(y)=exp(x+y) However, it also (which I often find break code) assumes no operation will make a NaN or an Infinity -- these last two don't really help, and also break code in confusing ways. This being gcc, they don't just change things like std::isnan or std::isinf into an 'abort' (which would make sense, in -ffast-math they don't make sense), they just return nonsense instead.
- clausecker 2y ago
  For me, the most important part is -fno-math-errno which allows the compiler to ignore that libm functions are allowed to set errno. This is perfectly safe (unless you rely on that rarely known side effect) and is usually the one flag I explicitly set.
  - gary_0 2y ago
    And the primary benefit of doing that is so the compiler can inline math functions like sqrt() as a tiny number of instructions (on modern CPUs) instead of having to call the standard C function, which is much slower.
    - adgjlsfhk1 2y ago
      > as a tiny number of instructions specifically 1
      - gary_0 2y ago
        For sqrt() on x86_64 and gcc/clang, yes. But functions like fmod() are generally more instructions. And as for trig functions like sin(), AFAIK most compilers will always use a function call, because the x86 trig instructions don't have good speed/accuracy compared to a modern stdlib. And YMMV when it comes to other arch's and compilers (and -fmath settings).
  - CJefferson 2y ago
    You are absolutely right, I forgot one of the most important things piled into -ffast-math!
sgerenser 2y ago
I also learned from experience that -ffast-math only enables the FTZ/DAZ optimization on the main thread, at least on Linux/X86. I don’t know if its universal or this has changed since I debugged it ~5-6 years ago, but that proved to be a bit hard to get to the bottom of since I immediately suspected the big CPU spike when the volume was set very low was caused by denormals, yet we were using the —ffast-math gcc flag.
- svantana 2y ago
  With clang on my intel mac it doesn't work at all, the only solution is to set those flags using _mm_setcsr().
gpderetta 2y ago
I think the linking with the flush-to-zero code nonsense has been moved to a separate flag in recent GCCs.
omoikane 2y ago
See also: https://news.ycombinator.com/item?id=29482350 https://news.ycombinator.com/item?id=29482350 - Understanding Fast-Math (2021) https://pspdfkit.com/blog/2021/understanding-fast-math/ https://pspdfkit.com/blog/2021/understanding-fast-math/

Not sure if I should be relieved or concerned that even Ted Unangst doesn't know what -ffast-math actually does. (For the record, it does a whole lot of terrible things: https://stackoverflow.com/questions/7420665/what-does-gccs-ffast-math-actually-do https://stackoverflow.com/questions/7420665/what-does-gccs-f... )

pclmulqdq 2y ago

There's more. It also enables some limited (unsafe) rearrangement of your floating point expressions, as well as a few other settings inside the floating point system. Flushing denorms to 0 is only part of it. It also will do things like using an approximate reciprocal square root instruction plus a refinement iteration instead of fsqrt then fdiv.

CJefferson 2y ago

-ffast-mast does a whole bunch of things, which I wish people's didn't so commonly combine. For example, I think the things it does which are sensible for most people are: * Rounding subnormals to zero * Disabling signed zeroes * Disables support for 'trapping' (throwing SIGFPE) Then there are the 'middle' things, which annoy some people: * Allow associative operations, and things like sqrt(xy)=sqrt(x)sqrt(y), exp(x)*exp(y)=exp(x+y) However, it also (which I often find break code) assumes no operation will make a NaN or an Infinity -- these last two don't really help, and also break code in confusing ways. This being gcc, they don't just change things like std::isnan or std::isinf into an 'abort' (which would make sense, in -ffast-math they don't make sense), they just return nonsense instead.

clausecker 2y ago
For me, the most important part is -fno-math-errno which allows the compiler to ignore that libm functions are allowed to set errno. This is perfectly safe (unless you rely on that rarely known side effect) and is usually the one flag I explicitly set.
- gary_0 2y ago
  And the primary benefit of doing that is so the compiler can inline math functions like sqrt() as a tiny number of instructions (on modern CPUs) instead of having to call the standard C function, which is much slower.
  - adgjlsfhk1 2y ago
    > as a tiny number of instructions specifically 1
    - gary_0 2y ago
      For sqrt() on x86_64 and gcc/clang, yes. But functions like fmod() are generally more instructions. And as for trig functions like sin(), AFAIK most compilers will always use a function call, because the x86 trig instructions don't have good speed/accuracy compared to a modern stdlib. And YMMV when it comes to other arch's and compilers (and -fmath settings).
- CJefferson 2y ago
  You are absolutely right, I forgot one of the most important things piled into -ffast-math!

sgerenser 2y ago

I also learned from experience that -ffast-math only enables the FTZ/DAZ optimization on the main thread, at least on Linux/X86. I don’t know if its universal or this has changed since I debugged it ~5-6 years ago, but that proved to be a bit hard to get to the bottom of since I immediately suspected the big CPU spike when the volume was set very low was caused by denormals, yet we were using the —ffast-math gcc flag.

svantana 2y ago
With clang on my intel mac it doesn't work at all, the only solution is to set those flags using _mm_setcsr().

gpderetta 2y ago

I think the linking with the flush-to-zero code nonsense has been moved to a separate flag in recent GCCs.

omoikane 2y ago

See also: https://news.ycombinator.com/item?id=29482350 https://news.ycombinator.com/item?id=29482350 - Understanding Fast-Math (2021) https://pspdfkit.com/blog/2021/understanding-fast-math/ https://pspdfkit.com/blog/2021/understanding-fast-math/