3 ms·

> -Ofast to allow lax numerics Please don't do this if you want your results to be reproducible.

• stephencanon 5y ago
  -Ofast doesn't license any transforms that can break GEMM. Everything that it licenses is already used universally by GEMM kernels, because those transforms have no effect on the standard backwards stability bounds for matrix multiplication.
  • gnufx 5y ago
    Indeed, the BLIS test suite with generic C kernels passes, not just level 3, though only the -funsafe-math-optimizations of -ffast-math is used by default. (I don't know the specifics for OpenBLAS, but it must be similar.) At least some of BLAS was defined before there even was an IEEE floating point standard.
• mkbosmans 5y ago
  Your life will be a lot easier if you decide not to chase bit-level reproducibility when doing floating point calculations. You can try setting CFLAGS to get the same result on all hardware, but that is a lot of effort, especially if you don't want to leave any performance on the table. And for what gain, just to make it simpler for your tests to compare against a fixed reference? In practice you don't gain anything else by having reproducible calculations because your inputs will have a lot more uncertainty in them than just beyond the 16th digit. Better to embrace the difference in results between hardware architecture, parallel execution, compiler version, optimization level, etc. This means spending time making your test suite more robust to small variations, but it saves you time investigating failed test runs that are insignificant. And as a bonus, it forces you to understand the numerical properties of your code better.