12 ms·
Generally floating point works fine for most things. Every now and then it doesn't, and you have to be aware enough of its weaknesses to (a) detect this, (b) un
by taneq 7y ago
Generally floating point works fine for most things. Every now and then it doesn't, and you have to be aware enough of its weaknesses to (a) detect this, (b) understand what the hell is going on, and (c) find a workaround for it.
- magicalhippo 7y agoIndeed. Had a fun issue once where a colleague had prematurely optimized an expression to (a * c * e) / (b * d * f), this was in a somewhat hot path hence he wanted to eliminate divisions. Turned out that in certain cases, the factors were all tiny and so both the numerator and denominator became denormalized and the division returned a NaN, ruining all further computations. After a lot of debugging trying to find the source of the NaNs and understanding why exactly this happened the solution was quite clear, simply expand it back out: (a/b) * (c/d) * (e/f). This was because due to the nature of the math being implemented, those ratios would always be roughly around order unity, even though each individual variable could potentially contain a very small number. This costs us like 0.1% performance, but made the code handle all inputs without issue.
- hrgiger 7y agoI had similar problem too, that multiplication results had to be in range of 0 < x < 1 because after that, some log operations was resulting NAN, first I was using if/else which was slowed down noticeably and after I end up applying sigmoid which speed up things a bit, still I dont know exact cause and I am not brave to dive into that mess again
- srean 7y agoThis comes up a lot in calculating odds, or ratio of probabilities. How one implements these in code is a good indicator of how much experience a person has with real world scenarios. One of those shibboleths. Another telltale giveaway is accumulating the dot product of float32s in float32s.
- eutectic 7y agoSerious probability calculations are usually best made in log-space, turning multiplication into addition.
- srean 7y agoIndeed -- https://en.wikipedia.org/wiki/Log_semiring https://en.wikipedia.org/wiki/Log_semiring
- taneq 7y agoSee this is why we keep you maths people around. Sometimes all that abstract wizardry is really useful! :)
- srean 7y agoI just hope that wasn't bitter sarcasm. I think this would have been a more relevant link https://en.wikipedia.org/wiki/LogSumExp#log-sum-exp_trick_for_log-domain_calculations https://en.wikipedia.org/wiki/LogSumExp#log-sum-exp_trick_fo...