3 ms·

On the one hand, the ambiguity of FMA is kind of annoying, but on the other, FMA is a godsend for writing accurate arithmatic. One reason for this, is that the

On the one hand, the ambiguity of FMA is kind of annoying, but on the other, FMA is a godsend for writing accurate arithmatic. One reason for this, is that the error of `ab` is `fma(a, b, -ab)` which makes compensated algorithms much faster to write. Secondly, polynomial evaluations using horner's method converge to .5 ULP (units in last place) with FMA, but to 1.5 ULP without. This is incredibly useful since polynomial approximations are at the heart of pretty much everything else. I think Julia takes an interesting approach here of never reordering floating point computations unless you explecitly give permission using @fastmath (which acts locally instead of globally like a --fast-math in C/C++). This makes it much easier to use ieee specific tricks and be confident that the compiler won't mess with you, while still making it easy to tell the compiler to do what it wants.

thxg 5y ago
> FMA is a godsend for writing accurate arithmatic. True, and the small performance boost (on some platforms) is nice too > never reordering floating point computations unless you explecitly give permission Yes, requiring explicit code for FMA and arithmetic grouping/reordering in general seems like the sane approach.
titzer 5y ago
> I think Julia takes an interesting approach here of never reordering floating point computations WebAssembly as well. IEEE-754 is mandated and there are no unsafe compiler optimizations allowed. This is the only sane choice. It's just too hard to reason about programs otherwise. Nothing about a super-intelligent machine completely reorganizing your code in a way that subtly breaks it is good, IMHO.