4 ms·

The article asks "Which Micro-optimizations matter for BLAS3?", implying small dimensions, but doesn't actually tell me. The problem is well-studied, depending

by gnufx 5y ago

The article asks "Which Micro-optimizations matter for BLAS3?", implying small dimensions, but doesn't actually tell me. The problem is well-studied, depending on what you consider "small". The most important thing is to avoid the packing step below an appropriate threshold. Implementations include libxsmm, blasfeo, and the "sup" version in blis (with papers on libxsmm and blasfeo). Eigen might also be relevant. https://libxsmm.readthedocs.io/ https://libxsmm.readthedocs.io/ https://blasfeo.syscop.de/ https://blasfeo.syscop.de/ https://github.com/flame/blis https://github.com/flame/blis

celrod 5y ago
Blis is very bad at small sizes, last I checked. Their approach was based on having a single microkernel, and setting up calls to it. Last I checked, blasfeo did not support AVX512, and this performed poorly on CPUs supporting it. I'm not really familiar with xsmm, but someone showed me this: https://haampie.github.io/smm-bench/cascadelake/ https://haampie.github.io/smm-bench/cascadelake/ https://haampie.github.io/smm-bench/skylake-avx512/ https://haampie.github.io/smm-bench/skylake-avx512/ LoopVectorization.jl performs best with AVX512, but not badly without it: https://haampie.github.io/smm-bench/znver2/ https://haampie.github.io/smm-bench/znver2/ Versus Eigen, LoopVectorization.jl does better at small sizes (less than a couple hundred, up until packing matters) on my computer.
adgjlsfhk1 5y ago
One thing to note is that LoopVectorization is much more general than traditional BLAS interfaces since it can automatically generate complicated efficient kernels eg tanh(A*v+y). The traditional model of calling out to hand optimized BLAS is severely lacking because that means that your code has to make use of existing hand designed kernels which ends up leading to extra passes over memory.
- gnufx 5y ago
  What does that have to do with what I wrote? At least compare maths function apples with apples. Most of the point of optimized L3 BLAS is optimizing for memory so it's compute-bound asymptotically but, for instance, header-only libxsmm isn't normal BLAS. (I'm used to just using -Ofast to get vectorization, not having to include a special library.)
  - celrod 5y ago
    If you want the -Ofast equivalent in Julia, there is @fastmath. LoopVectorization.jl is substantially more powerful. It also has tons of limitations, but I am rewriting it to fix most of them. Disclosure: I'm the primary author of both SimpleChains.jl and LoopVectorization.jl.
  - krastanov 5y ago
    Could you elaborate on "just using -Ofast to get vectorization"? I can not really believe this works for anything but the simplest of kernels.
    - ChrisRackauckas 5y ago
      -Ofast can replace some functions with more vectorizable versions (in Julia terms, tanh to tanh_fast) which can then let the compiler optimization passes add SIMD where they see fit. The issue that we've seen though is that the compiler optimization passes really do not perform as well as you'd hope.

by gnufx 5y ago

celrod 5y ago

Blis is very bad at small sizes, last I checked. Their approach was based on having a single microkernel, and setting up calls to it. Last I checked, blasfeo did not support AVX512, and this performed poorly on CPUs supporting it. I'm not really familiar with xsmm, but someone showed me this: https://haampie.github.io/smm-bench/cascadelake/ https://haampie.github.io/smm-bench/cascadelake/ https://haampie.github.io/smm-bench/skylake-avx512/ https://haampie.github.io/smm-bench/skylake-avx512/ LoopVectorization.jl performs best with AVX512, but not badly without it: https://haampie.github.io/smm-bench/znver2/ https://haampie.github.io/smm-bench/znver2/ Versus Eigen, LoopVectorization.jl does better at small sizes (less than a couple hundred, up until packing matters) on my computer.

adgjlsfhk1 5y ago

One thing to note is that LoopVectorization is much more general than traditional BLAS interfaces since it can automatically generate complicated efficient kernels eg tanh(A*v+y). The traditional model of calling out to hand optimized BLAS is severely lacking because that means that your code has to make use of existing hand designed kernels which ends up leading to extra passes over memory.

gnufx 5y ago
What does that have to do with what I wrote? At least compare maths function apples with apples. Most of the point of optimized L3 BLAS is optimizing for memory so it's compute-bound asymptotically but, for instance, header-only libxsmm isn't normal BLAS. (I'm used to just using -Ofast to get vectorization, not having to include a special library.)
- celrod 5y ago
  If you want the -Ofast equivalent in Julia, there is @fastmath. LoopVectorization.jl is substantially more powerful. It also has tons of limitations, but I am rewriting it to fix most of them. Disclosure: I'm the primary author of both SimpleChains.jl and LoopVectorization.jl.
- krastanov 5y ago
  Could you elaborate on "just using -Ofast to get vectorization"? I can not really believe this works for anything but the simplest of kernels.
  - ChrisRackauckas 5y ago
    -Ofast can replace some functions with more vectorizable versions (in Julia terms, tanh to tanh_fast) which can then let the compiler optimization passes add SIMD where they see fit. The issue that we've seen though is that the compiler optimization passes really do not perform as well as you'd hope.