3 ms·
Also, the article concludes that FMA hurts accuracy, and this is at best misleading (FMA is better in the worst case). But the example given does highlight some
by thxg 5y ago
Also, the article concludes that FMA hurts accuracy, and this is at best misleading (FMA is better in the worst case). But the example given does highlight something important:
Depending on the case, the accuracy gains can be tiny (so much so that in this example the FMA version does indeed return larger errors in most runs), and the performance gains can be small too.
On the other hand, the price to pay for letting the compiler freely decide whether/when to fuse * and + is huge: reproducibility. Since x87 was deprecated, having floating-point results be bit-for-bit consistent across platforms, compilers and optimization flags has been amazing for debugging. Losing that is a step back and a recipe for headaches.
I was surprised that gcc enables -fexcess-precision=fast by default (in GNU mode), hence enabling transparent FMA and potential headaches. Specifying the standard (e.g. -stdc=c17) fixes that.
- dnautics 5y agoIirc x87 was even worse on some processors depending on the processor state, the processor could cache the CPU state on context switch and revert the x87 register to a f64, leading to irreproducibility at the runtime level (randomly fails if you get an unlucky thread thrash)