5 ms·

It looks like -fassociative-math is "safe" in the sense that it can not be used to get UB in working code? That's a good property to make it easier to use in th

It looks like -fassociative-math is "safe" in the sense that it can not be used to get UB in working code? That's a good property to make it easier to use in the right context.

simonbyrne 5y ago
Not necessarily: if your cospi(x) function is always returning 1.0 (https://github.com/JuliaLang/julia/issues/30073#issuecomment-439707503 https://github.com/JuliaLang/julia/issues/30073#issuecomment...), but you wrote your code assuming the result was in a different interval, then you could quite easily invoke undefined behavior.
adgjlsfhk1 5y ago
Yeah. It's safe in that you won't get UB, but it's bad in that you can get arbitrarily wrong answers.
- wiml 5y ago
  To be fair, if you're using floating point at all you can get arbitrarily wrong answers. The nice thing about ieee754 conformance is that you can, with a lot of expertise, somewhat reason about the kinds of error you're getting. But for code that wasn't written by someone skilled in numerical techniques, and that's the vast majority of fp code, is fast-math worse than the status quo?
  - simonbyrne 5y ago
    From personal experience, yes: I've seen multiple cases of scientists finding the ultimate cause of their bugs was some fast-math-related optimization. The problem isn't necessarily the code they wrote themselves: it is often that they've compiled someone else's code or an open source library with fast-math, which broke some internal piece.
mbauman 5y ago
See the one footnote: you can re-associate a list of 2046 numbers such that they sum to _any_ floating point number between 0 and 2^970. https://discourse.julialang.org/t/array-ordering-and-naive-summation/1929 https://discourse.julialang.org/t/array-ordering-and-naive-s...
- StefanKarpinski 5y ago
  To be fair though, as noted further down in that thread, naive left-to-right summation is the worst case here since the tree of operations is as skewed as possible. I think that any other tree shape is better and compilers will tend to make the tree more balanced, not less, when they use associativity to enable SIMD optimizations. So while reassociation is theoretically arbitrarily bad, in practice it's probably mostly ok. Probably.
  - stephencanon 5y ago
    This is exactly right. Also, _all_ of the summation orderings satisfy the usual backwards-stability bounds, and so all of them are perfectly stable in normal computations.
- evilotto 5y ago
  Floating point math is fun: def re_add(a,b,c,d,e): return (a+b+c+d+e) == (2 * (c+a+b+d+e)) print(re_add(1e17, -1e17, 3, 2, 1))

It looks like -fassociative-math is "safe" in the sense that it can not be used to get UB in working code? That's a good property to make it easier to use in the right context.

simonbyrne 5y ago

Not necessarily: if your cospi(x) function is always returning 1.0 (https://github.com/JuliaLang/julia/issues/30073#issuecomment-439707503 https://github.com/JuliaLang/julia/issues/30073#issuecomment...), but you wrote your code assuming the result was in a different interval, then you could quite easily invoke undefined behavior.

adgjlsfhk1 5y ago

Yeah. It's safe in that you won't get UB, but it's bad in that you can get arbitrarily wrong answers.

wiml 5y ago
To be fair, if you're using floating point at all you can get arbitrarily wrong answers. The nice thing about ieee754 conformance is that you can, with a lot of expertise, somewhat reason about the kinds of error you're getting. But for code that wasn't written by someone skilled in numerical techniques, and that's the vast majority of fp code, is fast-math worse than the status quo?
- simonbyrne 5y ago
  From personal experience, yes: I've seen multiple cases of scientists finding the ultimate cause of their bugs was some fast-math-related optimization. The problem isn't necessarily the code they wrote themselves: it is often that they've compiled someone else's code or an open source library with fast-math, which broke some internal piece.

mbauman 5y ago

See the one footnote: you can re-associate a list of 2046 numbers such that they sum to _any_ floating point number between 0 and 2^970. https://discourse.julialang.org/t/array-ordering-and-naive-summation/1929 https://discourse.julialang.org/t/array-ordering-and-naive-s...

StefanKarpinski 5y ago
To be fair though, as noted further down in that thread, naive left-to-right summation is the worst case here since the tree of operations is as skewed as possible. I think that any other tree shape is better and compilers will tend to make the tree more balanced, not less, when they use associativity to enable SIMD optimizations. So while reassociation is theoretically arbitrarily bad, in practice it's probably mostly ok. Probably.
- stephencanon 5y ago
  This is exactly right. Also, _all_ of the summation orderings satisfy the usual backwards-stability bounds, and so all of them are perfectly stable in normal computations.
evilotto 5y ago
Floating point math is fun: def re_add(a,b,c,d,e): return (a+b+c+d+e) == (2 * (c+a+b+d+e)) print(re_add(1e17, -1e17, 3, 2, 1))