4 ms·

You can exploit the exactness of (specific) floating point operations in test data by using sums of powers of 2. Polynomials with such coefficients produce exac

You can exploit the exactness of (specific) floating point operations in test data by using sums of powers of 2. Polynomials with such coefficients produce exact results so long as the overall powers are within ~53 powers of 2 (don't quote me exactly on that, I generally don't push the range very high!). You can find exact polynomial solutions to linear PDEs with such powers using high enough order finite difference methods for example. However, the story about non-determinism is no myth. The intel processors have a separate math coprocessor that supports 80bit floats (https://en.wikipedia.org/wiki/Extended_precision#x86_extended-precision_format https://en.wikipedia.org/wiki/Extended_precision#x86_extende...). Moving a float from a register in this coprocessor to memory truncates the float. Repeated math can be done inside this coprocessor to achieve higher precision so hot loops generally don't move floats outside of these registers. Non-determinism occurs in programs running on intel with floats when threads are interrupted and the math coprocessor flushed. The non-determinism isn't intrinsic to the floating point arithmetic but to the non-determinism of when this truncation may occur. This is more relevant for fields where chaotic dynamics occur. So the same program with the same inputs can produce different results. NaN is an error. If you take the square root of a negative number you get a NaN. This is just a type error, use complex numbers to overcome this one. But then you get 0. / 0. and that's a NaN or Inf - Inf and a whole slew of other things that produce out of bounds results. Whether it is expected or not is another story, but it does mean that you are unable to represent the value with a float and that is a type error.

AshamedCaptain 1y ago
> Non-determinism occurs in programs running on intel with floats when threads are interrupted and the math coprocessor flushed That's ridiculous. No OS in his right mind would flush FPU regs to 64 bits only, because that would break many things, most obviously "real" 80 bit FP which is still a thing and the only reason x87 instructions still work. It would even break plain equality comparisons making all FP useless. For 64 bit FP most compilers prefer SSE rather than x87 instructions these days.
- dur-randir 1y ago
  https://bugs.php.net/bug.php?id=53632 https://bugs.php.net/bug.php?id=53632 Never, for sure.
bobmcnamara 1y ago
> Non-determinism occurs in programs running on intel FTFY. They even changed some of the more obscure handling between 8087,80287,80387. So much hoop jumping if you cared about binary reproducibility. Seems to be largely fixed with targeting SSE even for scalar code now.
jcranmer 1y ago
Wow, you're crossing a few wires in your zeal to provide information to the point that you're repeating myths. > The intel processors have a separate math coprocessor that supports 80bit floats x86 processors have two FPU units, the x87 unit (that you're describing) and the SSE unit. Anyone compiling for x86-64 uses the SSE unit for default, and most x86-32 compilers still default to SSE anyways. > Moving a float from a register in this coprocessor to memory truncates the float. No it doesn't. The x87 unit has load and store instructions for 32-bit, 64-bit, and 80-bit floats. If you want to spill 80-bit values as 80-bit values, you can do so. > Repeated math can be done inside this coprocessor to achieve higher precision so hot loops generally don't move floats outside of these registers. Hot loops these days use the SSE stuff because they're so much faster than x87. Friends don't let friends use long double without good reason! > Non-determinism occurs in programs running on intel with floats when threads are interrupted and the math coprocessor flushed. Lol, nope. You'll spill the x87 register stack on thread context switch with FSAVE or FXSAVE or XSAVE, all of which will store the registers as 80-bit values without loss of precision. That said, there was a problem with programs that use the x87 unit, but it has absolutely nothing to do with what you're describing. The x87 unit doesn't have arithmetic for 32-bit and 64-bit values, only 80-bit values. Many compilers, though, just pretended that the x87 unit supported arithmetic on 32-bit and 64-bit values, so that FADD would simultaneously be a 32-bit addition, a 64-bit addition, and a 80-bit addition. If the compiler needed to spill a floating-point register, they would spill the value as a 32-bit value (if float) or 64-bit value (if double), and register spills are pretty unpredictable for user code. That's the nondeterminism you're referring to, and it's considered a bug in every compiler I'm aware of. (See https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3715r0.html#FLT_EVAL_METHOD https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p37... for a more thorough description of the problem).
- BlackFly 1y ago
  It isn't zeal, it's 15 years past hazy memory of getting different results on different executions in the same supercomputer. The story that went around was the one I relayed, but certainly your link does a better job explaining things that happen in the user perspective section: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3715r0.html#reproducible https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p37... Summarized as, > Most users cannot be expected to know all of the ways that their floating-point code is not reproducible. Glad to know that the situation is defaulting to SSE2 nowadays though.
nyeah 1y ago
NaN is not necessarily an error. It might be fine. It depends on what you're doing with it. If NaN is invalid input for the next step, then sure why not treat it as an error? But that's a design decision not an imperative that everybody must follow. (I picture Mel Brooks' 15 commandments, #11-15, that fell and broke. This is not like that.)
- kccqzy 1y ago
  Yes it is totally fine. I've seen code basically treating NaN as missing data as opposed to std::optional<double> or equivalent in your language. NaN propagates so this works like using such types as a monad.
- BlackFly 1y ago
  Sure, not every runtime type error needs to panic your application, nor even panic a request handler, nor even result in a failed handling. That doesn't mean you didn't encounter an error though. Error doesn't mean fatal. The design decision isn't, "Was this a type error?" but, "What do I need to do about this type error?"
  - deleted 1y ago
    [deleted]
  - nyeah 1y ago
    “BRITANNUS (shocked). Caesar, this is a type error. THEODOTUS (outraged). How! CAESAR (recovering his self-possession). Pardon him. Theodotus: he is a barbarian, and thinks that the customs of his tribe and island are the definition of the float type.”

AshamedCaptain 1y ago

> Non-determinism occurs in programs running on intel with floats when threads are interrupted and the math coprocessor flushed That's ridiculous. No OS in his right mind would flush FPU regs to 64 bits only, because that would break many things, most obviously "real" 80 bit FP which is still a thing and the only reason x87 instructions still work. It would even break plain equality comparisons making all FP useless. For 64 bit FP most compilers prefer SSE rather than x87 instructions these days.

dur-randir 1y ago
https://bugs.php.net/bug.php?id=53632 https://bugs.php.net/bug.php?id=53632 Never, for sure.

bobmcnamara 1y ago

> Non-determinism occurs in programs running on intel FTFY. They even changed some of the more obscure handling between 8087,80287,80387. So much hoop jumping if you cared about binary reproducibility. Seems to be largely fixed with targeting SSE even for scalar code now.

jcranmer 1y ago

Wow, you're crossing a few wires in your zeal to provide information to the point that you're repeating myths. > The intel processors have a separate math coprocessor that supports 80bit floats x86 processors have two FPU units, the x87 unit (that you're describing) and the SSE unit. Anyone compiling for x86-64 uses the SSE unit for default, and most x86-32 compilers still default to SSE anyways. > Moving a float from a register in this coprocessor to memory truncates the float. No it doesn't. The x87 unit has load and store instructions for 32-bit, 64-bit, and 80-bit floats. If you want to spill 80-bit values as 80-bit values, you can do so. > Repeated math can be done inside this coprocessor to achieve higher precision so hot loops generally don't move floats outside of these registers. Hot loops these days use the SSE stuff because they're so much faster than x87. Friends don't let friends use long double without good reason! > Non-determinism occurs in programs running on intel with floats when threads are interrupted and the math coprocessor flushed. Lol, nope. You'll spill the x87 register stack on thread context switch with FSAVE or FXSAVE or XSAVE, all of which will store the registers as 80-bit values without loss of precision. That said, there was a problem with programs that use the x87 unit, but it has absolutely nothing to do with what you're describing. The x87 unit doesn't have arithmetic for 32-bit and 64-bit values, only 80-bit values. Many compilers, though, just pretended that the x87 unit supported arithmetic on 32-bit and 64-bit values, so that FADD would simultaneously be a 32-bit addition, a 64-bit addition, and a 80-bit addition. If the compiler needed to spill a floating-point register, they would spill the value as a 32-bit value (if float) or 64-bit value (if double), and register spills are pretty unpredictable for user code. That's the nondeterminism you're referring to, and it's considered a bug in every compiler I'm aware of. (See https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3715r0.html#FLT_EVAL_METHOD https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p37... for a more thorough description of the problem).

BlackFly 1y ago
It isn't zeal, it's 15 years past hazy memory of getting different results on different executions in the same supercomputer. The story that went around was the one I relayed, but certainly your link does a better job explaining things that happen in the user perspective section: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p3715r0.html#reproducible https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p37... Summarized as, > Most users cannot be expected to know all of the ways that their floating-point code is not reproducible. Glad to know that the situation is defaulting to SSE2 nowadays though.

nyeah 1y ago

NaN is not necessarily an error. It might be fine. It depends on what you're doing with it. If NaN is invalid input for the next step, then sure why not treat it as an error? But that's a design decision not an imperative that everybody must follow. (I picture Mel Brooks' 15 commandments, #11-15, that fell and broke. This is not like that.)

kccqzy 1y ago
Yes it is totally fine. I've seen code basically treating NaN as missing data as opposed to std::optional<double> or equivalent in your language. NaN propagates so this works like using such types as a monad.
BlackFly 1y ago
Sure, not every runtime type error needs to panic your application, nor even panic a request handler, nor even result in a failed handling. That doesn't mean you didn't encounter an error though. Error doesn't mean fatal. The design decision isn't, "Was this a type error?" but, "What do I need to do about this type error?"
- deleted 1y ago
  [deleted]
- nyeah 1y ago
  “BRITANNUS (shocked). Caesar, this is a type error. THEODOTUS (outraged). How! CAESAR (recovering his self-possession). Pardon him. Theodotus: he is a barbarian, and thinks that the customs of his tribe and island are the definition of the float type.”