2 ms·
Your arguments are correct, but the conclusion does not result from them. If we assume that underflows happen in your program and this, as you say, is a sign t
by adrian_b 5y ago
Your arguments are correct, but the conclusion does not result from them.
If we assume that underflows happen in your program and this, as you say, is a sign that greater problems will be caused by that, then you must not enable flush-to-zero, but you must enable trap-on-underflow, to see where underflows happen and to investigate the reason and maybe rearrange your formulas to avoid the too small results.
Flush-to-zero may sometimes lead to crashes, when it becomes obvious that something is wrong, but more often you just get some results with large errors that are difficult to distinguish from good results.
Opinions obviously vary, but I have never seen any good use case for flush-to-zero.
Underflows are either so rare that their performance does not matter, or if they are so frequent that flush-to-zero will increase the speed, then your code has a problem and the formulas should be restructured, which will both increase the speed and eliminate the loss of precision.
- marcan_42 5y agoIt is normal to get values close to zero in a correctly implemented floating point algorithm - what matters is that, at that point, you don't rely on their dynamic range or precision. Trap-on-underflow would produce many false positives for algorithms which would be perfectly happy having those numbers be treated as zero. This is not uncommon - it is completely expected with, say, anything implementing an exponential decay! A slow exponential decay could run into subnormals for large spans of time, yet be fine having those treated as zero.
- stephencanon 5y ago> Opinions obviously vary, but I have never seen any good use case for flush-to-zero. The classical use case is real-time audio, where an IIR filter may have quite slow decay, such that the signal stays in the subnormal regime for many samples. If this happens and your hardware has significant stalls for subnormal data, you may miss your real-time deadline resulting in clicking or other audio corruption.
- adrian_b 5y agoI agree that this is a good example and there are probably similar cases in video processing and in graphics rendering. However not all CPUs stall for subnormal data and if the CPU designers would have ensured that none of them stall, no such discussions would have been ever needed. Outside such special cases like real-time DSP and graphics, using flush-to-zero or options like -Ofast or -ffast-math are seldom justified and they can be dangerous, especially when they are used without a prior analysis and tests to determine whether exception conditions really appear when the program is run, and why.
- stephencanon 5y ago> However not all CPUs stall for subnormal data and if the CPU designers would have ensured that none of them stall, no such discussions would have been ever needed. I don't really disagree (I've pushed hard for no subnormal stalls for years), but there are some timing and area tradeoffs that come with this that make it a somewhat hard sell for FPUs that aren't built entirely around FMA (there are tradeoffs even for designs that are purely FMA based, but they're a lot smaller). Mainstream x86 designs _still_ have some subnormal stalls!
- nyanpasu64 5y ago> Outside such special cases like real-time DSP and graphics Personally I see audio and graphics as the common case, and scientific computing as the special case ;) Do game physics engines expect denormals to be preserved?