4 ms·

I think this is the key. My understanding is the FMADD on arm64 does what the spec says it should do, so there's nothing broken. Next up though is whether fuse

I think this is the key. My understanding is the FMADD on arm64 does what the spec says it should do, so there's nothing broken. Next up though is whether fused multiply add is part of the IEEE floating point standards, and whether the arm implementation meets these. Chances are it does (I see references to it from 2008). If the x64 FMADD produces difference results, it's likely that either the standard is wooly in some respects and allows wiggle room (so they both meet the standard), or, the Intel/AMD implementation pre-dates the standard and isn't compliant. I'm sure someone who knows these standards and processors better than me will be along in a minute to comment.

ascar 5y ago
I wouldn't be surprised if the difference in rounding is an innate property of how the math behind an FMA instruction works and that guaranteeing the same rounding (which is essentially rounding twice, once after mul and once after add) heavily impacts the performance benefits. Thus implementing FMA with an inconsistent rounding to separate mul and add instruction is in the interest of performance. I'm just guessing here tho.
brandmeyer 5y ago
The standard mandates that the result of fma is rounded as-if an infinite precision result is computed internally followed by a single correct rounding. There isn't any wiggle room.
cwzwarich 5y ago
For finite inputs, the results are unambiguously specified by the IEEE spec. For singularities involving zero/infinite inputs, the IEEE spec allows two sets of behavior, which basically correspond to x86 and everyone else.
jcranmer 5y ago
From IEEE 754-2019: > The operation fusedMultiplyAdd(x, y, z) computes (x × y) + z as if with unbounded range and precision, rounding only once to the destination format. [...] fusedMultiplyAdd differs from a multiplication operation followed by an addition operation. [elided exception discussion] I expect that the ARM processor precisely implements this standard, as does x86-64. Any "bugs" here are the user misunderstanding how the precision is supposed to work.

ascar 5y ago

I wouldn't be surprised if the difference in rounding is an innate property of how the math behind an FMA instruction works and that guaranteeing the same rounding (which is essentially rounding twice, once after mul and once after add) heavily impacts the performance benefits. Thus implementing FMA with an inconsistent rounding to separate mul and add instruction is in the interest of performance. I'm just guessing here tho.

brandmeyer 5y ago

The standard mandates that the result of fma is rounded as-if an infinite precision result is computed internally followed by a single correct rounding. There isn't any wiggle room.

cwzwarich 5y ago

For finite inputs, the results are unambiguously specified by the IEEE spec. For singularities involving zero/infinite inputs, the IEEE spec allows two sets of behavior, which basically correspond to x86 and everyone else.

jcranmer 5y ago

From IEEE 754-2019: > The operation fusedMultiplyAdd(x, y, z) computes (x × y) + z as if with unbounded range and precision, rounding only once to the destination format. [...] fusedMultiplyAdd differs from a multiplication operation followed by an addition operation. [elided exception discussion] I expect that the ARM processor precisely implements this standard, as does x86-64. Any "bugs" here are the user misunderstanding how the precision is supposed to work.