3 ms·

My experience is that you need the additional caveat of a streaming fairly homogeneous and highly parallelizable approach or the memory transfer, branching and

My experience is that you need the additional caveat of a streaming fairly homogeneous and highly parallelizable approach or the memory transfer, branching and communication overheads will eat away nearly all the GPU gains. I've also noticed that a lot of the time, people are comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations. For a large swathe of problem types, using the fastest math library will reduce the CPU/GPU gap to less than an order of magnitude or less.

twtw 8y ago
> comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations In your experience, are these comparisons using naive GPU implementations as well? If so, that still seems like a valuable comparison.
shaklee3 8y ago
No, they're not. Straight front the horse's mouth: https://software.intel.com/en-us/mkl/features/benchmarks https://software.intel.com/en-us/mkl/features/benchmarks In no case will you see it get close to 10TFLOPS. GPUs easily do this, and can approach 100TFLOPS with tensor cores.
- Cybiote 8y ago
  I don't dispute this. If you can remove memory transfer overhead and meet the requirements I mentioned, then GPUs will be much better. But there are many problems that do not fit that (SIMD per warp) regime. In short, GPUs are no panacea and have their own trade-offs and bottleneck sensitivities, just like everything else.
  - shaklee3 8y ago
    I agree that they are no panacea, but we also have an example of the Intel Xeon phi, which did have high speed hbm memory. That too did not compete with GPUs. I think the majority of it has to do with the GPU chip itself being extremely simple, with no Branch prediction, no branch difference, very simple caching behavior, etc. They are able to allocate a much larger portion of the die to just throw fma units at it.

twtw 8y ago

> comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations In your experience, are these comparisons using naive GPU implementations as well? If so, that still seems like a valuable comparison.

shaklee3 8y ago

No, they're not. Straight front the horse's mouth: https://software.intel.com/en-us/mkl/features/benchmarks https://software.intel.com/en-us/mkl/features/benchmarks In no case will you see it get close to 10TFLOPS. GPUs easily do this, and can approach 100TFLOPS with tensor cores.

Cybiote 8y ago
I don't dispute this. If you can remove memory transfer overhead and meet the requirements I mentioned, then GPUs will be much better. But there are many problems that do not fit that (SIMD per warp) regime. In short, GPUs are no panacea and have their own trade-offs and bottleneck sensitivities, just like everything else.
- shaklee3 8y ago
  I agree that they are no panacea, but we also have an example of the Intel Xeon phi, which did have high speed hbm memory. That too did not compete with GPUs. I think the majority of it has to do with the GPU chip itself being extremely simple, with no Branch prediction, no branch difference, very simple caching behavior, etc. They are able to allocate a much larger portion of the die to just throw fma units at it.