4 ms·
> Is that likely in practice? I thought nearly everything will conform to IEEE 754 and give identical results for floating-point arithmetic. What compiler sett
by dragontamer 5y ago
> Is that likely in practice? I thought nearly everything will conform to IEEE 754 and give identical results for floating-point arithmetic.
What compiler settings? With -O3, GCC will flush-to-zero your denormals, because Intel CPUs have _SEVERE_ penalties on subnormal floats.
You don't even get the same results with -O vs -O3 on the same programming language on the same system... let alone different CPUs.
https://carlh.net/plugins/denormals.php https://carlh.net/plugins/denormals.php
Denormals are part of IEEE 754, but in practice are more complicated for hardware to figure out. So as a common efficiency trick, you disable the processing of denormals entirely (IE: all denormals are seen as "0" to the hardware, speeding up processing severely).
On Intel processors, a configurable register can set up how your floating-point units act when they come across a denormal. So an inline-assembly statement can change your results, even if it was no where "close" to your code.
- kimixa 5y ago-O3 shouldn't enable flushing denormals to zero on gcc, the standard optimisation flags shouldn't affect the results - if they do it's likely a bug. To enable that, you need to use -Ofast or -ffast-math, there's a reason they're not default enabled after all.
- SavantIdiot 5y agoDo you know what are typically circumstances under which this becomes problematic? Clearly these issues don't impact FPS games, or music quality, but do they start becoming problems in weather forecasting or particle simulations? What kind of applications are impacted?
- dragontamer 5y ago> Clearly these issues don't impact FPS games On the contrary. One of the most common multiplayer bugs is the multiplayer-desync due to different settings of floating-point arithmetic. https://stackoverflow.com/questions/1948929/keeping-sync-in-multiplayer-rts-game-that-uses-floating-point-arithmetic https://stackoverflow.com/questions/1948929/keeping-sync-in-... This stuff happens in pretty much every video game. You "solve" it by... 1. In FPS games, most _clients_ serve as the source of truth. This means that in most FPS games, you allow the client to "hack" the location data for clipping, speed-hacks, and the like. (Server calculated you at point X. Client claims to be at point X+.05. Well, client wins the argument). 2. In games like Factorio, an extraordinary amount of effort is placed upon finding all sources of desync (https://wiki.factorio.com/Desynchronization https://wiki.factorio.com/Desynchronization) and squashing them. Protocols communicate with integers as much as possible, avoiding any floating point issues. > but do they start becoming problems in weather forecasting or particle simulations? What kind of applications are impacted? I'm no supercomputer expert. But for reliability, a lot of supercomputer simulations step through twice to "verify" the results. That means both simulations must be setup with the same settings, so that your floats all line up. In my undergraduate studies, my professor for numeric computing stressed the importance of sorting numbers: not only to avoid floating-point cancellation error, but also to have a precisely defined ordering for supercomputer applications. You want to do things like add / subtract numbers from smallest to biggest, and all the positive numbers add together, then all the negative numbers add together and only do one subtraction. Cancellation error is serious business!!
- SavantIdiot 5y agoWow, this is such a great answer. Thanks for the information!
- kergonath 5y ago> I'm no supercomputer expert. But for reliability, a lot of supercomputer simulations step through twice to "verify" the results. That means both simulations must be setup with the same settings, so that your floats all line up. In such cases, the result of one simulation is not enough, and we do several ones with slightly different initial conditions, just to make sure that we don’t spend time analysing an outlier. Exact reproducibility is important to validate the codes running test suites, but for the purpose of using simulations to make predictions, it does not really matter as long as each simulation is physically accurate enough.
- db48x 5y agoThe Factorio Friday Facts dev diaries are a great read, and they’ve talked about the effects of floating point numbers multiple times. Sometimes in reference to multiplayer, but even bugs in single player have been tracked back to floating point numbers: https://www.factorio.com/blog/post/fff-297 https://www.factorio.com/blog/post/fff-297