4 ms·

Fotran can be a pain to write, but it can lead to very efficient programs, especially with the incorporation of advanced math kernels like ATLAS or MKL. Of cour

by bmh100 12y ago

Fotran can be a pain to write, but it can lead to very efficient programs, especially with the incorporation of advanced math kernels like ATLAS or MKL. Of course, CUDA will lead to 10x-40x boost in GEMM performance in certain algorithms as well. If you have a very matrix heavy hot loop, or could possibly represent an algorithm as a matrix operation [1], it might be worth your time to write some modern Fortran [2]. [1]: http://kukuruku.co/hub/algorithms/using-the-quick-raise-of-matrices-to-power-to-write-a-very-fast-interpreter-of-a-simple-programming-language?ModPagespeed=noscript http://kukuruku.co/hub/algorithms/using-the-quick-raise-of-m... [2]: http://fortranwiki.org/fortran/show/Fortran+2008 http://fortranwiki.org/fortran/show/Fortran+2008

jedbrown 12y ago
1. ATLAS hasn't had great performance in ages because it explores the wrong parameter space. OpenBLAS or MKL are much better choices and don't need autotuning. 2. You can call these libraries from any language, so Fortran offers no advantage there. 3. Fortran performance is similar to C and C++ (assuming use of restrict where needed). 4. CUDA will not lead to 10-40x boost unless your CPU implementation is garbage. The realistic standard is about 2x when normalized by Watts; see the Green500 for example. The 2x also holds for price when using server-grade parts (like Tesla), but GPUs come out further ahead if you buy consumer grade. Note that you still have to pay to buy and power a host, and that GPUs are a lot less versatile in terms of problem size/turn-around time and workload. They have their place, but evaluate your workload carefully before concluding that the cost proposition favors GPUs.
- timeu 12y ago
  1. Some time ago I did some benchmarking of different BLAS implementations on our HPC using different CPUs (http://stackoverflow.com/questions/5260068/multithreaded-blas-in-python-numpy/7645939#7645939 http://stackoverflow.com/questions/5260068/multithreaded-bla...). MKL and OpenBLAS performed best but ATLAS is not that bad. But of course I would use rather OpenBLAS/MKL rather than ATLAS because it performs better and is also much easier to install.
- draven 12y ago
  It never occurred to me to consider the performance per watt of CUDA vs C/C++/Fortan on the CPU (instead of only raw performance.) So one should consider CUDA only if they want to favor raw performance over cost? (and only if their algorithms yield a significant improvement when implemented to run on a GPU.)
  - jedbrown 12y ago
    CUDA on consumer GPUs for workloads that execute efficiently on those architectures can provide about 2x benefit in energy efficiency and somewhat bigger in acquisition cost, provided there is enough work at each stage of your algorithm to amortize kernel launch latency and PCIe transfer latency and bandwidth. If you buy server-grade GPUs, the cost proposition is similar to power -- at best ~2x benefit.
- bmh100 12y ago
  1. I certainly did not mean to imply that ATLAS or MKL were the best CPU linear algebra libraries out there. Thanks for pointing out OpenBLAS and offering that comparison. 2. If I were to run a highly iterative algorithm, say with 100,000 or more iterations, would the memory marshaling not be significant? Or would it be unnoticeable since I would be using these libraries anyway? I also personally find the array notation and slicing to be quite nice. 3. The use of "restrict" is nice. I was not aware of that feature. Looking at this reference [1], it seems that Fortran basically does the equivalent of automatically using "restrict". 4. According to NVIDIA [2], cuBLAS outperforms MKL by 6x-17x (it seems MKL is catching up since I last checked). Is this misleading by NVIDIA? What is your expectation for a reasonable execution speed increase? The fixed-cost efficiency and variable-cost efficiency are good points to bring into the discussion. Your point about carefully evaluating workload is also a good one. I don't want people to think that throwing an algorithm into CUDA will magically speed up the overall program. Network I/O, disk I/O, and device (GPU) I/O are all important bottlenecks to consider. [1]: http://www-cs-students.stanford.edu/~blynn/c/ch05.html http://www-cs-students.stanford.edu/~blynn/c/ch05.html [2]: https://developer.nvidia.com/cuBLAS https://developer.nvidia.com/cuBLAS
  - jedbrown 12y ago
    2. Most languages have a way to store raw arrays that can be passed directly to the numerical routines. If you have to marshal, then it depends on the problem size and amount of work done in the numerical routine. 3. Yes. 4. NVIDIA has a track record of misleading comparisons. They cleaned up their act in the these CUDA-7 benchmarks: http://devblogs.nvidia.com/parallelforall/cuda-7-release-candidate-feature-overview/ http://devblogs.nvidia.com/parallelforall/cuda-7-release-can... in response to this G+ discussion https://plus.google.com/+JeffHammondScience/posts/G1MzHqZaxyz https://plus.google.com/+JeffHammondScience/posts/G1MzHqZaxy... . Thanks to Szilárd Páll for calling attention to this discussion and to Mark Harris for updating the plots. Note that this should not be interpreted as "MKL/Xeon is catching up" but rather that performance comparisons are sensitive to the details of the experiment and it can be hard to recognize the consequences of biased configurations. Better normalization and standards for comparison can help.

by bmh100 12y ago

jedbrown 12y ago

1. ATLAS hasn't had great performance in ages because it explores the wrong parameter space. OpenBLAS or MKL are much better choices and don't need autotuning. 2. You can call these libraries from any language, so Fortran offers no advantage there. 3. Fortran performance is similar to C and C++ (assuming use of restrict where needed). 4. CUDA will not lead to 10-40x boost unless your CPU implementation is garbage. The realistic standard is about 2x when normalized by Watts; see the Green500 for example. The 2x also holds for price when using server-grade parts (like Tesla), but GPUs come out further ahead if you buy consumer grade. Note that you still have to pay to buy and power a host, and that GPUs are a lot less versatile in terms of problem size/turn-around time and workload. They have their place, but evaluate your workload carefully before concluding that the cost proposition favors GPUs.

timeu 12y ago
1. Some time ago I did some benchmarking of different BLAS implementations on our HPC using different CPUs (http://stackoverflow.com/questions/5260068/multithreaded-blas-in-python-numpy/7645939#7645939 http://stackoverflow.com/questions/5260068/multithreaded-bla...). MKL and OpenBLAS performed best but ATLAS is not that bad. But of course I would use rather OpenBLAS/MKL rather than ATLAS because it performs better and is also much easier to install.
draven 12y ago
It never occurred to me to consider the performance per watt of CUDA vs C/C++/Fortan on the CPU (instead of only raw performance.) So one should consider CUDA only if they want to favor raw performance over cost? (and only if their algorithms yield a significant improvement when implemented to run on a GPU.)
- jedbrown 12y ago
  CUDA on consumer GPUs for workloads that execute efficiently on those architectures can provide about 2x benefit in energy efficiency and somewhat bigger in acquisition cost, provided there is enough work at each stage of your algorithm to amortize kernel launch latency and PCIe transfer latency and bandwidth. If you buy server-grade GPUs, the cost proposition is similar to power -- at best ~2x benefit.
bmh100 12y ago
1. I certainly did not mean to imply that ATLAS or MKL were the best CPU linear algebra libraries out there. Thanks for pointing out OpenBLAS and offering that comparison. 2. If I were to run a highly iterative algorithm, say with 100,000 or more iterations, would the memory marshaling not be significant? Or would it be unnoticeable since I would be using these libraries anyway? I also personally find the array notation and slicing to be quite nice. 3. The use of "restrict" is nice. I was not aware of that feature. Looking at this reference [1], it seems that Fortran basically does the equivalent of automatically using "restrict". 4. According to NVIDIA [2], cuBLAS outperforms MKL by 6x-17x (it seems MKL is catching up since I last checked). Is this misleading by NVIDIA? What is your expectation for a reasonable execution speed increase? The fixed-cost efficiency and variable-cost efficiency are good points to bring into the discussion. Your point about carefully evaluating workload is also a good one. I don't want people to think that throwing an algorithm into CUDA will magically speed up the overall program. Network I/O, disk I/O, and device (GPU) I/O are all important bottlenecks to consider. [1]: http://www-cs-students.stanford.edu/~blynn/c/ch05.html http://www-cs-students.stanford.edu/~blynn/c/ch05.html [2]: https://developer.nvidia.com/cuBLAS https://developer.nvidia.com/cuBLAS
- jedbrown 12y ago
  2. Most languages have a way to store raw arrays that can be passed directly to the numerical routines. If you have to marshal, then it depends on the problem size and amount of work done in the numerical routine. 3. Yes. 4. NVIDIA has a track record of misleading comparisons. They cleaned up their act in the these CUDA-7 benchmarks: http://devblogs.nvidia.com/parallelforall/cuda-7-release-candidate-feature-overview/ http://devblogs.nvidia.com/parallelforall/cuda-7-release-can... in response to this G+ discussion https://plus.google.com/+JeffHammondScience/posts/G1MzHqZaxyz https://plus.google.com/+JeffHammondScience/posts/G1MzHqZaxy... . Thanks to Szilárd Páll for calling attention to this discussion and to Mark Harris for updating the plots. Note that this should not be interpreted as "MKL/Xeon is catching up" but rather that performance comparisons are sensitive to the details of the experiment and it can be hard to recognize the consequences of biased configurations. Better normalization and standards for comparison can help.