5 ms·
Everything on that benchmark page is listed as GFLOPS/core so at 30GFLOPS that Epyc 7551 should be pushing around 2 TFLOPS. You can take a look at reproducing t
by lhl 7y ago
Everything on that benchmark page is listed as GFLOPS/core so at 30GFLOPS that Epyc 7551 should be pushing around 2 TFLOPS. You can take a look at reproducing them if you want if you have a similar system: https://github.com/flame/blis/blob/master/docs/Performance.md#reproduction https://github.com/flame/blis/blob/master/docs/Performance.m...
I ran into some compile issues, but I care more about numpy so I gave this script a try: https://markus-beuckelmann.de/blog/boosting-numpy-blas.html https://markus-beuckelmann.de/blog/boosting-numpy-blas.html with some quick/dirty results on my Ryzen 3700X (8C16T) workstation (BLIS doesn't seem to perform very well on this small test). You can see performance basically double on MKL when MKL_DEBUG_CPU_TYPE=5 is used. I'll probably continue to stick w/ OpenBLAS for now:
## blis 0.6.0 h516909a_0
# conda activate numpy-blis
# conda run python bench.py
Dotted two 4096x4096 matrices in 2.30 s.
Dotted two vectors of length 524288 in 0.08 ms.
SVD of a 2048x1024 matrix in 0.94 s.
Cholesky decomposition of a 2048x2048 matrix in 0.24 s.
Eigendecomposition of a 2048x2048 matrix in 6.36 s.
## libopenblas 0.3.7 h5ec1e0e_4
# conda activate numpy-openblas
# conda run python bench.py
Dotted two 4096x4096 matrices in 0.41 s.
Dotted two vectors of length 524288 in 0.02 ms.
SVD of a 2048x1024 matrix in 0.55 s.
Cholesky decomposition of a 2048x2048 matrix in 0.14 s.
Eigendecomposition of a 2048x2048 matrix in 5.53 s.
## mkl 2019.4 243
# conda activate numpy-mkl
# conda run python bench.py
Dotted two 4096x4096 matrices in 1.53 s.
Dotted two vectors of length 524288 in 0.02 ms.
SVD of a 2048x1024 matrix in 0.51 s.
Cholesky decomposition of a 2048x2048 matrix in 0.29 s.
Eigendecomposition of a 2048x2048 matrix in 4.79 s.
# export MKL_DEBUG_CPU_TYPE=5
# conda run python bench.py
Dotted two 4096x4096 matrices in 0.33 s.
Dotted two vectors of length 524288 in 0.02 ms.
SVD of a 2048x1024 matrix in 0.29 s.
Cholesky decomposition of a 2048x2048 matrix in 0.15 s.
Eigendecomposition of a 2048x2048 matrix in 3.29 s.
- shaklee3 7y agoThanks. I completely missed that legend. Does it actually scale linearly by cores? I would think there's a plateau after a certain number of cores.
- lhl 7y agoThat benchmark page I referenced shows scaling on a SkylakeX at 1, 26, and 52 cores, on a Haswell at 1, 12, and 24 cores, and on Zen1 at 1, 32, and 64 cores so you can probably map out a ballpark scaling co-efficent from that. I'm sure there are theoretical and practical limits but it's probably completely dependent on implementation and each system architecture, but that's beyond my pay grade. Maybe start here: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra...
- gnufx 7y agoIf you want to understand implementation and performance, that page won't help you. Read the papers referenced on the BLIS web site, and/or run their benchmark code, which is now referenced somewhere in a README.
- lhl 7y agoOh, just for fun, I manually replaced the BLIS libs w/ the AMD BLIS 2.0 fork (Ubuntu MT binary on an Arch system: https://developer.amd.com/amd-aocl/blas-library/ https://developer.amd.com/amd-aocl/blas-library/) and it seemed to work ok. Slightly faster than BLIS 0.6.0 but not actually faster than OpenBLAS on this test... ## AMD BLIS 2.0 https://developer.amd.com/amd-aocl/blas-library/ # conda activate numpy-amdblis Dotted two 4096x4096 matrices in 2.33 s. Dotted two vectors of length 524288 in 0.08 ms. SVD of a 2048x1024 matrix in 0.78 s. Cholesky decomposition of a 2048x2048 matrix in 0.23 s. Eigendecomposition of a 2048x2048 matrix in 5.84 s.