7 ms·

Based on Zen1 Epyc performance it would seem that BLIS might be significantly more performant even without AMD-specific optimizations (I know AMD has their own

by lhl 7y ago

Based on Zen1 Epyc performance it would seem that BLIS might be significantly more performant even without AMD-specific optimizations (I know AMD has their own fork, but check out the 64 core results for "vanilla" BLIS: https://github.com/flame/blis/blob/master/docs/Performance.md#epyc https://github.com/flame/blis/blob/master/docs/Performance.m...)

shaklee3 7y ago
Something is still weird with the gemm benchmarks. That link you sent has BLIS winning all around, but the absolute value of the BLIS is poor. MKL gets around 2TFLOPS on xeons, and this blis benchmark is only hitting around 30-50GFLOPS. Are AMDs hitting over 1TFLOPS anywhere?
- shaklee3 7y ago
  To illustrate this, let's assume the GEMM used avx512 perfectly. Roughly that's 2.5e9 (clock) * 16 (ops per cycle) * 2 (FMA per cycle) * 64 (cores). That puts Intel's peak performance over 2TFLOPs, which they're showing. That's about 80x higher than AMD on BLIS...
- lhl 7y ago
  Everything on that benchmark page is listed as GFLOPS/core so at 30GFLOPS that Epyc 7551 should be pushing around 2 TFLOPS. You can take a look at reproducing them if you want if you have a similar system: https://github.com/flame/blis/blob/master/docs/Performance.md#reproduction https://github.com/flame/blis/blob/master/docs/Performance.m... I ran into some compile issues, but I care more about numpy so I gave this script a try: https://markus-beuckelmann.de/blog/boosting-numpy-blas.html https://markus-beuckelmann.de/blog/boosting-numpy-blas.html with some quick/dirty results on my Ryzen 3700X (8C16T) workstation (BLIS doesn't seem to perform very well on this small test). You can see performance basically double on MKL when MKL_DEBUG_CPU_TYPE=5 is used. I'll probably continue to stick w/ OpenBLAS for now: ## blis 0.6.0 h516909a_0 # conda activate numpy-blis # conda run python bench.py Dotted two 4096x4096 matrices in 2.30 s. Dotted two vectors of length 524288 in 0.08 ms. SVD of a 2048x1024 matrix in 0.94 s. Cholesky decomposition of a 2048x2048 matrix in 0.24 s. Eigendecomposition of a 2048x2048 matrix in 6.36 s. ## libopenblas 0.3.7 h5ec1e0e_4 # conda activate numpy-openblas # conda run python bench.py Dotted two 4096x4096 matrices in 0.41 s. Dotted two vectors of length 524288 in 0.02 ms. SVD of a 2048x1024 matrix in 0.55 s. Cholesky decomposition of a 2048x2048 matrix in 0.14 s. Eigendecomposition of a 2048x2048 matrix in 5.53 s. ## mkl 2019.4 243 # conda activate numpy-mkl # conda run python bench.py Dotted two 4096x4096 matrices in 1.53 s. Dotted two vectors of length 524288 in 0.02 ms. SVD of a 2048x1024 matrix in 0.51 s. Cholesky decomposition of a 2048x2048 matrix in 0.29 s. Eigendecomposition of a 2048x2048 matrix in 4.79 s. # export MKL_DEBUG_CPU_TYPE=5 # conda run python bench.py Dotted two 4096x4096 matrices in 0.33 s. Dotted two vectors of length 524288 in 0.02 ms. SVD of a 2048x1024 matrix in 0.29 s. Cholesky decomposition of a 2048x2048 matrix in 0.15 s. Eigendecomposition of a 2048x2048 matrix in 3.29 s.
  - shaklee3 7y ago
    Thanks. I completely missed that legend. Does it actually scale linearly by cores? I would think there's a plateau after a certain number of cores.
    - lhl 7y ago
      That benchmark page I referenced shows scaling on a SkylakeX at 1, 26, and 52 cores, on a Haswell at 1, 12, and 24 cores, and on Zen1 at 1, 32, and 64 cores so you can probably map out a ballpark scaling co-efficent from that. I'm sure there are theoretical and practical limits but it's probably completely dependent on implementation and each system architecture, but that's beyond my pay grade. Maybe start here: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra...
      - gnufx 7y ago
        If you want to understand implementation and performance, that page won't help you. Read the papers referenced on the BLIS web site, and/or run their benchmark code, which is now referenced somewhere in a README.
  - lhl 7y ago
    Oh, just for fun, I manually replaced the BLIS libs w/ the AMD BLIS 2.0 fork (Ubuntu MT binary on an Arch system: https://developer.amd.com/amd-aocl/blas-library/ https://developer.amd.com/amd-aocl/blas-library/) and it seemed to work ok. Slightly faster than BLIS 0.6.0 but not actually faster than OpenBLAS on this test... ## AMD BLIS 2.0 https://developer.amd.com/amd-aocl/blas-library/ # conda activate numpy-amdblis Dotted two 4096x4096 matrices in 2.33 s. Dotted two vectors of length 524288 in 0.08 ms. SVD of a 2048x1024 matrix in 0.78 s. Cholesky decomposition of a 2048x2048 matrix in 0.23 s. Eigendecomposition of a 2048x2048 matrix in 5.84 s.
gnufx 7y ago
AMD maintain a fork which is merged back as they make it available, so mainline BLIS typically has that work soon after. That was last done in October, since the current release (0.6.0). The FLAME people encourage you to use the current master branch, but you might bother about correctness, as it was failing BLAS tests recently on POWER and the generic target.

by lhl 7y ago

shaklee3 7y ago

Something is still weird with the gemm benchmarks. That link you sent has BLIS winning all around, but the absolute value of the BLIS is poor. MKL gets around 2TFLOPS on xeons, and this blis benchmark is only hitting around 30-50GFLOPS. Are AMDs hitting over 1TFLOPS anywhere?

shaklee3 7y ago
To illustrate this, let's assume the GEMM used avx512 perfectly. Roughly that's 2.5e9 (clock) * 16 (ops per cycle) * 2 (FMA per cycle) * 64 (cores). That puts Intel's peak performance over 2TFLOPs, which they're showing. That's about 80x higher than AMD on BLIS...
lhl 7y ago
Everything on that benchmark page is listed as GFLOPS/core so at 30GFLOPS that Epyc 7551 should be pushing around 2 TFLOPS. You can take a look at reproducing them if you want if you have a similar system: https://github.com/flame/blis/blob/master/docs/Performance.md#reproduction https://github.com/flame/blis/blob/master/docs/Performance.m... I ran into some compile issues, but I care more about numpy so I gave this script a try: https://markus-beuckelmann.de/blog/boosting-numpy-blas.html https://markus-beuckelmann.de/blog/boosting-numpy-blas.html with some quick/dirty results on my Ryzen 3700X (8C16T) workstation (BLIS doesn't seem to perform very well on this small test). You can see performance basically double on MKL when MKL_DEBUG_CPU_TYPE=5 is used. I'll probably continue to stick w/ OpenBLAS for now: ## blis 0.6.0 h516909a_0 # conda activate numpy-blis # conda run python bench.py Dotted two 4096x4096 matrices in 2.30 s. Dotted two vectors of length 524288 in 0.08 ms. SVD of a 2048x1024 matrix in 0.94 s. Cholesky decomposition of a 2048x2048 matrix in 0.24 s. Eigendecomposition of a 2048x2048 matrix in 6.36 s. ## libopenblas 0.3.7 h5ec1e0e_4 # conda activate numpy-openblas # conda run python bench.py Dotted two 4096x4096 matrices in 0.41 s. Dotted two vectors of length 524288 in 0.02 ms. SVD of a 2048x1024 matrix in 0.55 s. Cholesky decomposition of a 2048x2048 matrix in 0.14 s. Eigendecomposition of a 2048x2048 matrix in 5.53 s. ## mkl 2019.4 243 # conda activate numpy-mkl # conda run python bench.py Dotted two 4096x4096 matrices in 1.53 s. Dotted two vectors of length 524288 in 0.02 ms. SVD of a 2048x1024 matrix in 0.51 s. Cholesky decomposition of a 2048x2048 matrix in 0.29 s. Eigendecomposition of a 2048x2048 matrix in 4.79 s. # export MKL_DEBUG_CPU_TYPE=5 # conda run python bench.py Dotted two 4096x4096 matrices in 0.33 s. Dotted two vectors of length 524288 in 0.02 ms. SVD of a 2048x1024 matrix in 0.29 s. Cholesky decomposition of a 2048x2048 matrix in 0.15 s. Eigendecomposition of a 2048x2048 matrix in 3.29 s.
- shaklee3 7y ago
  Thanks. I completely missed that legend. Does it actually scale linearly by cores? I would think there's a plateau after a certain number of cores.
  - lhl 7y ago
    That benchmark page I referenced shows scaling on a SkylakeX at 1, 26, and 52 cores, on a Haswell at 1, 12, and 24 cores, and on Zen1 at 1, 32, and 64 cores so you can probably map out a ballpark scaling co-efficent from that. I'm sure there are theoretical and practical limits but it's probably completely dependent on implementation and each system architecture, but that's beyond my pay grade. Maybe start here: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprogra...
    - gnufx 7y ago
      If you want to understand implementation and performance, that page won't help you. Read the papers referenced on the BLIS web site, and/or run their benchmark code, which is now referenced somewhere in a README.
- lhl 7y ago
  Oh, just for fun, I manually replaced the BLIS libs w/ the AMD BLIS 2.0 fork (Ubuntu MT binary on an Arch system: https://developer.amd.com/amd-aocl/blas-library/ https://developer.amd.com/amd-aocl/blas-library/) and it seemed to work ok. Slightly faster than BLIS 0.6.0 but not actually faster than OpenBLAS on this test... ## AMD BLIS 2.0 https://developer.amd.com/amd-aocl/blas-library/ # conda activate numpy-amdblis Dotted two 4096x4096 matrices in 2.33 s. Dotted two vectors of length 524288 in 0.08 ms. SVD of a 2048x1024 matrix in 0.78 s. Cholesky decomposition of a 2048x2048 matrix in 0.23 s. Eigendecomposition of a 2048x2048 matrix in 5.84 s.

gnufx 7y ago

AMD maintain a fork which is merged back as they make it available, so mainline BLIS typically has that work soon after. That was last done in October, since the current release (0.6.0). The FLAME people encourage you to use the current master branch, but you might bother about correctness, as it was failing BLAS tests recently on POWER and the generic target.