3 ms·

> the Intel CPU dispatcher does not only check which instruction set is supported by the CPU, it also checks the vendor ID string. If the vendor string says "Ge

by Rd6n6 5y ago

> the Intel CPU dispatcher does not only check which instruction set is supported by the CPU, it also checks the vendor ID string. If the vendor string says "GenuineIntel" then it uses the optimal code path. If the CPU is not from Intel then, in most cases, it will run the slowest possible version of the code, even if the CPU is fully compatible with a better version.[1] I’ve been a little shy about using intel software since reading about this years ago [1] https://www.agner.org/optimize/blog/read.php?i=49 https://www.agner.org/optimize/blog/read.php?i=49

SavantIdiot 5y ago
That's just plain sinister. I'm really surprised popular numerical computing Python packages don't already have optimized hardware back-ends for things like NumPy... similar to ORC (OIL) which has been around for quite some time: https://github.com/GStreamer/orc https://github.com/GStreamer/orc But I don't know that much about Python under the hood, and I'm willing to be since so many academics work on this there's already optimized FFIs. I've used TensorFlow and it can offload tensor math to GPUs, but only NVIDIA's AFAIK.
- danieldk 5y ago
  I'm really surprised popular numerical computing Python packages don't already have optimized hardware back-ends for things like NumPy I think it is hard to beat modern BLAS implementations for common operations. E.g. Apple Accelerate (which also implement the BLAS/LAPACK APIs) uses undocumented AMX instructions for large speedups compared to an ARM NEON implementation.
- deleted 5y ago
  [deleted]
danieldk 5y ago
This has gotten a bit better. Last time I checked, MKL now uses Zen-specific kernels for sgemm/dgemm. Unfortunately, these kernels are slower than the AVX2 kernels. But at least, it does not use the pre-modern SIMD kernels for AMD Zen anymore. Edit, comparison: $ perf record target/release/gemm-benchmark -d 1024 Threads: 1 Iterations per thread: 1000 Matrix shape: 1024 x 1024 GFLOPS/s: 96.36 $ perf report --stdio -q | head -n3 97.18% gemm-benchmark gemm-benchmark [.] mkl_blas_def_sgemm_kernel_0_zen 1.94% gemm-benchmark gemm-benchmark [.] mkl_blas_def_sgemm_scopy_down16_bdz 0.78% gemm-benchmark gemm-benchmark [.] mkl_blas_def_sgemm_scopy_right4_bdz After disabling Intel CPU detection: $ perf record target/release/gemm-benchmark -d 1024 Threads: 1 Iterations per thread: 1000 Matrix shape: 1024 x 1024 GFLOPS/s: 129.12 $ perf report --stdio -q | head -n3 97.02% gemm-benchmark libmkl_avx2.so.1 [.] mkl_blas_avx2_sgemm_kernel_0 1.77% gemm-benchmark libmkl_avx2.so.1 [.] mkl_blas_avx2_sgemm_scopy_down24_ea 1.02% gemm-benchmark libmkl_avx2.so.1 [.] mkl_blas_avx2_sgemm_scopy_right4_ea Benchmarked using https://github.com/danieldk/gemm-benchmark https://github.com/danieldk/gemm-benchmark and oneMKL 2021.3.0.
- Royi 5y ago
  How could one do your trick on Windows?

by Rd6n6 5y ago

SavantIdiot 5y ago

That's just plain sinister. I'm really surprised popular numerical computing Python packages don't already have optimized hardware back-ends for things like NumPy... similar to ORC (OIL) which has been around for quite some time: https://github.com/GStreamer/orc https://github.com/GStreamer/orc But I don't know that much about Python under the hood, and I'm willing to be since so many academics work on this there's already optimized FFIs. I've used TensorFlow and it can offload tensor math to GPUs, but only NVIDIA's AFAIK.

danieldk 5y ago
I'm really surprised popular numerical computing Python packages don't already have optimized hardware back-ends for things like NumPy I think it is hard to beat modern BLAS implementations for common operations. E.g. Apple Accelerate (which also implement the BLAS/LAPACK APIs) uses undocumented AMX instructions for large speedups compared to an ARM NEON implementation.
deleted 5y ago
[deleted]

danieldk 5y ago

This has gotten a bit better. Last time I checked, MKL now uses Zen-specific kernels for sgemm/dgemm. Unfortunately, these kernels are slower than the AVX2 kernels. But at least, it does not use the pre-modern SIMD kernels for AMD Zen anymore. Edit, comparison: $ perf record target/release/gemm-benchmark -d 1024 Threads: 1 Iterations per thread: 1000 Matrix shape: 1024 x 1024 GFLOPS/s: 96.36 $ perf report --stdio -q | head -n3 97.18% gemm-benchmark gemm-benchmark [.] mkl_blas_def_sgemm_kernel_0_zen 1.94% gemm-benchmark gemm-benchmark [.] mkl_blas_def_sgemm_scopy_down16_bdz 0.78% gemm-benchmark gemm-benchmark [.] mkl_blas_def_sgemm_scopy_right4_bdz After disabling Intel CPU detection: $ perf record target/release/gemm-benchmark -d 1024 Threads: 1 Iterations per thread: 1000 Matrix shape: 1024 x 1024 GFLOPS/s: 129.12 $ perf report --stdio -q | head -n3 97.02% gemm-benchmark libmkl_avx2.so.1 [.] mkl_blas_avx2_sgemm_kernel_0 1.77% gemm-benchmark libmkl_avx2.so.1 [.] mkl_blas_avx2_sgemm_scopy_down24_ea 1.02% gemm-benchmark libmkl_avx2.so.1 [.] mkl_blas_avx2_sgemm_scopy_right4_ea Benchmarked using https://github.com/danieldk/gemm-benchmark https://github.com/danieldk/gemm-benchmark and oneMKL 2021.3.0.

Royi 5y ago
How could one do your trick on Windows?