3 ms·

FMA can't be broken on Windows because FMA is implemented in hardware by Intel. What's broken is the compiler that Julia uses on Windows.

ChrisRackauckas 4y ago
When FMA isn't in the hardware (due to using some chip where it doesn't exist) it has a fallback to a software-based emulation. That is incorrectly implemented in Windows. Julia ends up calling that in this case because that's what LLVM ends up calling, and so any LLVM-based language will see this issue.
- celrod 4y ago
  Even when FMA is implemented in hardware, LLVM will generally use the software version when the arguments are known at compile time.
adgjlsfhk1 4y ago
the problem is that LLVM will happily miscompile fma instructions by turning them into incorrect constants due to windows having a broken libm. This is a bug in C/C++, and I'm currently unaware of a language that has fma and a good compiler which gives correct fma results on Windows.
- Const-me 4y ago
  CPUs support these instructions for 9 years now. When ignoring these old CPUs, most languages and compilers are usually doing a good job. Example in C which does not depend on any library functions: double fma( double a, double b, double c ) { __m128d av = _mm_set_sd( a ); __m128d bv = _mm_set_sd( b ); __m128d cv = _mm_set_sd( c ); return _mm_cvtsd_f64( _mm_fmadd_sd( av, bv, cv ) ); }
  - adgjlsfhk1 4y ago
    2 problems: Julia supports cpus without FMA, and on windows, llvm will use libc to constant fold the value of fma even on computers that have fma in hardware.
    - Const-me 4y ago
      Hardware requirements are up to the product management. For instance, many modern videogames (they generally want compatibility because directly translates to sales) no longer support 32-bit or pre-AVX1 processors. Technically, Julia can drop the support of pre-FMA3 processors if it helps moving forward. It’s inevitable anyway due to the changing hardware requirements of the OS, the only question is “when”. I don’t think Windows 10 21H2 supports any CPU which doesn’t have SSE 4.1, it’s only a matter of time when Windows will require support of newer instruction sets. About LLVM, can’t they compile that thing with an option like -mfma to use hardware FMA3 for constant folding?
stephencanon 4y ago
FMA is only implemented in hardware on Haswell and later uArches. If you’re running on (or compiling for) IVB or earlier, you’ll get a libcall instead, and MSVC’s has been broken since forever.
- Diggsey 4y ago
  Is this actually broken in MSVC, or is it broken because Julia is using mingw and linking to an ancient version of libc on windows (which is intentionally left as-is for back-compat)? (I genuinely don't know, but the linked issue mentioned mingw specifically)
  - adgjlsfhk1 4y ago
    It's broken in MSVC and mingw (in different ways). See https://github.com/MicrosoftDocs/cpp-docs/pull/3526 https://github.com/MicrosoftDocs/cpp-docs/pull/3526.
    - Diggsey 4y ago
      Thanks for the link!

ChrisRackauckas 4y ago

When FMA isn't in the hardware (due to using some chip where it doesn't exist) it has a fallback to a software-based emulation. That is incorrectly implemented in Windows. Julia ends up calling that in this case because that's what LLVM ends up calling, and so any LLVM-based language will see this issue.

celrod 4y ago
Even when FMA is implemented in hardware, LLVM will generally use the software version when the arguments are known at compile time.

adgjlsfhk1 4y ago

the problem is that LLVM will happily miscompile fma instructions by turning them into incorrect constants due to windows having a broken libm. This is a bug in C/C++, and I'm currently unaware of a language that has fma and a good compiler which gives correct fma results on Windows.

Const-me 4y ago
CPUs support these instructions for 9 years now. When ignoring these old CPUs, most languages and compilers are usually doing a good job. Example in C which does not depend on any library functions: double fma( double a, double b, double c ) { __m128d av = _mm_set_sd( a ); __m128d bv = _mm_set_sd( b ); __m128d cv = _mm_set_sd( c ); return _mm_cvtsd_f64( _mm_fmadd_sd( av, bv, cv ) ); }
- adgjlsfhk1 4y ago
  2 problems: Julia supports cpus without FMA, and on windows, llvm will use libc to constant fold the value of fma even on computers that have fma in hardware.
  - Const-me 4y ago
    Hardware requirements are up to the product management. For instance, many modern videogames (they generally want compatibility because directly translates to sales) no longer support 32-bit or pre-AVX1 processors. Technically, Julia can drop the support of pre-FMA3 processors if it helps moving forward. It’s inevitable anyway due to the changing hardware requirements of the OS, the only question is “when”. I don’t think Windows 10 21H2 supports any CPU which doesn’t have SSE 4.1, it’s only a matter of time when Windows will require support of newer instruction sets. About LLVM, can’t they compile that thing with an option like -mfma to use hardware FMA3 for constant folding?

stephencanon 4y ago

FMA is only implemented in hardware on Haswell and later uArches. If you’re running on (or compiling for) IVB or earlier, you’ll get a libcall instead, and MSVC’s has been broken since forever.

Diggsey 4y ago
Is this actually broken in MSVC, or is it broken because Julia is using mingw and linking to an ancient version of libc on windows (which is intentionally left as-is for back-compat)? (I genuinely don't know, but the linked issue mentioned mingw specifically)
- adgjlsfhk1 4y ago
  It's broken in MSVC and mingw (in different ways). See https://github.com/MicrosoftDocs/cpp-docs/pull/3526 https://github.com/MicrosoftDocs/cpp-docs/pull/3526.
  - Diggsey 4y ago
    Thanks for the link!