5 ms·

Intel did a great thing for people interested in ML and numeric research by making their MKL library and compiler free and cross platform compatible. Even today

Intel did a great thing for people interested in ML and numeric research by making their MKL library and compiler free and cross platform compatible. Even today on my AMD zen3 Ryzen machine intel's MKL linked numpy and pytorch are in some operations 10* (yes that is really ten times) faster in comparison with the next best alternative (openBlas etc). I was shocked to discover how much of a difference MKL makes for cpu workloads. This is mostly because it makes use of AVX2 cpu extensions which make certain matrix operations a lot faster.

ysleepy 4y ago
They crippled the performance on non-Intel CPUs on purpose until recently. Intel's anti competitive behaviour follows them throughout their history.
- Roark66 4y ago
  Yes, this is true, but even back then (during the time they crippled it) it was possible to pretend to run on Intel hardware. I haven't done this, but I read it was possible.
  - ysleepy 4y ago
    I just felt your first sentence, without a qualifier, gave too much credit looking at the context.
- pantalaimon 4y ago
  They crippled their own consumer CPUs by retroactively disabling AVX512
chrchang523 4y ago
That was my previous experience, but have you tried linking to AMD AOCL recently? I would not expect the performance gap between Intel MKL and AMD AOCL to still be as large as you describe on a Zen 3.
- Scene_Cast2 4y ago
  Would you know if AOCL is supported by numpy? And are there any benchmarks out there? (Especially interested in Zen4)
  - gnufx 4y ago
    Since BLAS (and LAPACK) have an essentially well-defined ABI, anything dynamically ELF-linked doesn't need specific support. You can just LD_PRELOAD, say, OpenBLAS or blis+libflame (as in AOCL) to use them. You can also make a trivial libblas.so.3 shim to put on LD_LIBRARY_PATH. (Debian has a heavyweight approach to linear algebra alternatives, with separate builds of the whole library.) See, e.g., https://fx.srht.site/posts/2019-10-20-library-aliases.html https://fx.srht.site/posts/2019-10-20-library-aliases.html
galangalalgol 4y ago
Compiling eigen uses avx512 just fine. I don't think it is quite the discriminator it once was. However, I amp happy they made it open and it would be great of oneAPI became a real alternative to cuda for more tasks. I tried getting it to work with flux.jl a while back and ran into some difficulty.
gnufx 4y ago
That's simply implausible for any proper measurement of the large-dimension level-3 operations that normally matter (and probably others I haven't measured). OpenBLAS certainly uses AVX2 assembler on Zen if it's built and run correctly, but you can get >60% of MKL's AVX2 performance with a plain C micro-kernel. The only plausible reason for an order of magnitude difference on large GEMM would be multi-threaded versus serial. I keep saying there's far too much mythology around MKL, disproved experimentally, and if you're using Zen I don't know why you wouldn't use AMD's version of BLIS. It doesn't even make sense to talk about MKL on Zen without the version, since the story keeps changing. OpenBLAS and BLIS actually are free software and cross-platform. MKL is still proprietary, and certainly doesn't run on the POWER platform I support. Also note that it only got the small matrix performance relevant to tensorflow after libxsmm showed the way.

ysleepy 4y ago

They crippled the performance on non-Intel CPUs on purpose until recently. Intel's anti competitive behaviour follows them throughout their history.

Roark66 4y ago
Yes, this is true, but even back then (during the time they crippled it) it was possible to pretend to run on Intel hardware. I haven't done this, but I read it was possible.
- ysleepy 4y ago
  I just felt your first sentence, without a qualifier, gave too much credit looking at the context.
pantalaimon 4y ago
They crippled their own consumer CPUs by retroactively disabling AVX512

chrchang523 4y ago

That was my previous experience, but have you tried linking to AMD AOCL recently? I would not expect the performance gap between Intel MKL and AMD AOCL to still be as large as you describe on a Zen 3.

Scene_Cast2 4y ago
Would you know if AOCL is supported by numpy? And are there any benchmarks out there? (Especially interested in Zen4)
- gnufx 4y ago
  Since BLAS (and LAPACK) have an essentially well-defined ABI, anything dynamically ELF-linked doesn't need specific support. You can just LD_PRELOAD, say, OpenBLAS or blis+libflame (as in AOCL) to use them. You can also make a trivial libblas.so.3 shim to put on LD_LIBRARY_PATH. (Debian has a heavyweight approach to linear algebra alternatives, with separate builds of the whole library.) See, e.g., https://fx.srht.site/posts/2019-10-20-library-aliases.html https://fx.srht.site/posts/2019-10-20-library-aliases.html

galangalalgol 4y ago

Compiling eigen uses avx512 just fine. I don't think it is quite the discriminator it once was. However, I amp happy they made it open and it would be great of oneAPI became a real alternative to cuda for more tasks. I tried getting it to work with flux.jl a while back and ran into some difficulty.

gnufx 4y ago

That's simply implausible for any proper measurement of the large-dimension level-3 operations that normally matter (and probably others I haven't measured). OpenBLAS certainly uses AVX2 assembler on Zen if it's built and run correctly, but you can get >60% of MKL's AVX2 performance with a plain C micro-kernel. The only plausible reason for an order of magnitude difference on large GEMM would be multi-threaded versus serial. I keep saying there's far too much mythology around MKL, disproved experimentally, and if you're using Zen I don't know why you wouldn't use AMD's version of BLIS. It doesn't even make sense to talk about MKL on Zen without the version, since the story keeps changing. OpenBLAS and BLIS actually are free software and cross-platform. MKL is still proprietary, and certainly doesn't run on the POWER platform I support. Also note that it only got the small matrix performance relevant to tensorflow after libxsmm showed the way.