4 ms·

Most Python math and ML libraries are compiled using GCC or LLVM and are linked against against cBLAS or OpenBLAS[1]. The latter is highly performant on both In

Most Python math and ML libraries are compiled using GCC or LLVM and are linked against against cBLAS or OpenBLAS[1]. The latter is highly performant on both Intel and AMD (and other platforms). Some libraries are optionally compiled against the MKL, in particular those that are distributed with the Anaconda Python distribution. [1] https://numpy.org/doc/stable/reference/routines.linalg.html?highlight=blas https://numpy.org/doc/stable/reference/routines.linalg.html?...

bitL 6y ago
I'd have to disagree, whenever I install new pip/conda packages, an MKL version is downloaded by default (I've never seen any *BLAS version by default in my life).
- ddavis 6y ago
  NumPy binary wheels on PyPI (i.e. from pip) are built with OpenBLAS. NumPy from the official Anaconda, Inc. conda channel defaults to MKL. The conda-forge channel defaults to OpenBLAS. Explained in the NumPy docs here: https://numpy.org/install/ https://numpy.org/install/
erwincoumans 6y ago
PyTorch uses MKL https://www.google.com/amp/s/amp.reddit.com/r/MachineLearning/comments/iap6yo/discussion_pytorch_favors_intel_against_amds/ https://www.google.com/amp/s/amp.reddit.com/r/MachineLearnin... and the workaround for AMD has been disabled by Intel
- celrod 6y ago
  I think MKL actually fixed Zen performance. That is, the workaround no longer makes any difference because it is no longer needed. Small matrix multiply benchmarks on a Zen2 (Ryzen 7 4700U), featuring MKL 2020.1.216+0, OpenBLAS, and Eigen: https://gist.github.com/stillyslalom/bd916e3d26b4531364676ac09d8469ad#gistcomment-3403272 https://gist.github.com/stillyslalom/bd916e3d26b4531364676ac... MKL's benchmark performance requires AVX + FMA. 3.5 GHz * (4 add + 4 multiply) * 2 fma/cycle = 56 peak GFLOPS. To exceed 50 GFLOPS without them would imply the CPU ran at 12.5 GHz. OpenBLAS, on the other hand, actually performed poorly because it was limited to SSE thanks to a bug preventing the CPU from being recognized as Zen2.
  - shaklee3 6y ago
    Do you know if this applies to epyc as well?
    - celrod 6y ago
      No, I don't. The 32-core AWS systems must be Epyc, so I'll try benchmarking there. When OpenBLAS identifies the arch, it is competitive with MKL in single threaded performance, at least for matrices with a couple hundred rows and columns or more. But MKL truly shines with multiple threads, so scaling on a 32 core system would be interesting to look at.
      - gnufx 6y ago
        You can see BLIS on Intel's home turf at https://github.com/flame/blis/blob/master/docs/Performance.md https://github.com/flame/blis/blob/master/docs/Performance.m... (52-core SKX) and compare with OpenBLAS on 32-core Zen1. (Multithreaded BLAS isn't typically used in HPC, where the parallelism is elsewhere.)
  - danieldk 6y ago
    I think MKL actually fixed Zen performance. That is, the workaround no longer makes any difference because it is no longer needed. Odd. I am trying on my 3700X and it is definitely not using AVX, FMA or AVX2 code paths. Intel MKL 2020 update 2: ldd ~/git/sticker2/target/release/sticker2 | grep mkl_intel libmkl_intel_lp64.so => /nix/store/jpjwkkv1dqk4nn8swjzr5qqzp0dpzk2f-mkl-2020.2.254/lib/libmkl_intel_lp64.so (0x00007fe786862000) I checked the instructions in with perf and it is using an SSE code path. Also, as reported elsewhere, MKL_DEBUG_CPU_TYPE=5 does not enable AVX2 support as it used to do.
    - celrod 6y ago
      Hmm. FWIW, on my [Skylake/Cascadelake]-X Intel systems, Intel's compilers performed well, almost always outperforming GCC and Clang. But on Zen, their performance was terrible. So I was happy to see that MKL, unlike the compilers, did not appear to gimp AMD. It's disappointing that MKL doesn't use optimized code paths on the 3700X. I messaged the person who actually ran the benchmarks and owns the laptop, asking them to chime in with more information. I'm just the person who wrote that benchmark suite.
MiroF 6y ago
> Most Python math and ML libraries are compiled using GCC or LLVM and are linked against against cBLAS or OpenBLAS[1] This is definitely understating MKLs market share in BLAS. It's an extremely common BLAS backing for Python libraries.

bitL 6y ago

I'd have to disagree, whenever I install new pip/conda packages, an MKL version is downloaded by default (I've never seen any *BLAS version by default in my life).

ddavis 6y ago
NumPy binary wheels on PyPI (i.e. from pip) are built with OpenBLAS. NumPy from the official Anaconda, Inc. conda channel defaults to MKL. The conda-forge channel defaults to OpenBLAS. Explained in the NumPy docs here: https://numpy.org/install/ https://numpy.org/install/

erwincoumans 6y ago

PyTorch uses MKL https://www.google.com/amp/s/amp.reddit.com/r/MachineLearning/comments/iap6yo/discussion_pytorch_favors_intel_against_amds/ https://www.google.com/amp/s/amp.reddit.com/r/MachineLearnin... and the workaround for AMD has been disabled by Intel

celrod 6y ago
I think MKL actually fixed Zen performance. That is, the workaround no longer makes any difference because it is no longer needed. Small matrix multiply benchmarks on a Zen2 (Ryzen 7 4700U), featuring MKL 2020.1.216+0, OpenBLAS, and Eigen: https://gist.github.com/stillyslalom/bd916e3d26b4531364676ac09d8469ad#gistcomment-3403272 https://gist.github.com/stillyslalom/bd916e3d26b4531364676ac... MKL's benchmark performance requires AVX + FMA. 3.5 GHz * (4 add + 4 multiply) * 2 fma/cycle = 56 peak GFLOPS. To exceed 50 GFLOPS without them would imply the CPU ran at 12.5 GHz. OpenBLAS, on the other hand, actually performed poorly because it was limited to SSE thanks to a bug preventing the CPU from being recognized as Zen2.
- shaklee3 6y ago
  Do you know if this applies to epyc as well?
  - celrod 6y ago
    No, I don't. The 32-core AWS systems must be Epyc, so I'll try benchmarking there. When OpenBLAS identifies the arch, it is competitive with MKL in single threaded performance, at least for matrices with a couple hundred rows and columns or more. But MKL truly shines with multiple threads, so scaling on a 32 core system would be interesting to look at.
    - gnufx 6y ago
      You can see BLIS on Intel's home turf at https://github.com/flame/blis/blob/master/docs/Performance.md https://github.com/flame/blis/blob/master/docs/Performance.m... (52-core SKX) and compare with OpenBLAS on 32-core Zen1. (Multithreaded BLAS isn't typically used in HPC, where the parallelism is elsewhere.)
- danieldk 6y ago
  I think MKL actually fixed Zen performance. That is, the workaround no longer makes any difference because it is no longer needed. Odd. I am trying on my 3700X and it is definitely not using AVX, FMA or AVX2 code paths. Intel MKL 2020 update 2: ldd ~/git/sticker2/target/release/sticker2 | grep mkl_intel libmkl_intel_lp64.so => /nix/store/jpjwkkv1dqk4nn8swjzr5qqzp0dpzk2f-mkl-2020.2.254/lib/libmkl_intel_lp64.so (0x00007fe786862000) I checked the instructions in with perf and it is using an SSE code path. Also, as reported elsewhere, MKL_DEBUG_CPU_TYPE=5 does not enable AVX2 support as it used to do.
  - celrod 6y ago
    Hmm. FWIW, on my [Skylake/Cascadelake]-X Intel systems, Intel's compilers performed well, almost always outperforming GCC and Clang. But on Zen, their performance was terrible. So I was happy to see that MKL, unlike the compilers, did not appear to gimp AMD. It's disappointing that MKL doesn't use optimized code paths on the 3700X. I messaged the person who actually ran the benchmarks and owns the laptop, asking them to chime in with more information. I'm just the person who wrote that benchmark suite.

MiroF 6y ago

> Most Python math and ML libraries are compiled using GCC or LLVM and are linked against against cBLAS or OpenBLAS[1] This is definitely understating MKLs market share in BLAS. It's an extremely common BLAS backing for Python libraries.