3 ms·

This can be vectorised for CPU execution (which is nice!), but it will not solve the problem for GPUs. Moving just that scalar-vector multiplication to a GPU w

by Athas 9y ago

This can be vectorised for CPU execution (which is nice!), but it will not solve the problem for GPUs. Moving just that scalar-vector multiplication to a GPU would be detrimental to performance, because the run-time would be dominated by the cost of moving the 'b' array to the GPU, followed by moving the 'a' array back. You get into all kinds of issues with where to leave data that has no obvious right answer. It also does not help with problems that are not as embarrassingly parallel, but could still benefit from vectorisation, such as summations (or reductions in general), or things that contain their own inner control flow (think a Mandelbrot set, where you have a larger outer parallel loop with an inner serial while-loop).

yarg 9y ago
That actually depends upon the architecture. If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's caches. It'll be interesting to see what sort of context-switch performance comes out of the Zen APU devices when they are released (next year I believe). I'd be somewhat surprised if Intel and NVidia aren't working on similar things.
- Athas 9y ago
  > If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's caches. Shared memory by itself will not have a significant performance impact here (although it simplifies the data movement substantially). For a scalar-vector multiplication, you are strongly memory bound, and so you still have to wait for all the memory to be retrieved over the bus. You are correct if both CPU and GPU are behind the same bus (such as if the GPU is on the same die), but modern GPUs support shared virtual memory even for non-integrated GPUs.

by Athas 9y ago

yarg 9y ago

That actually depends upon the architecture. If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's caches. It'll be interesting to see what sort of context-switch performance comes out of the Zen APU devices when they are released (next year I believe). I'd be somewhat surprised if Intel and NVidia aren't working on similar things.

Athas 9y ago
> If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's caches. Shared memory by itself will not have a significant performance impact here (although it simplifies the data movement substantially). For a scalar-vector multiplication, you are strongly memory bound, and so you still have to wait for all the memory to be retrieved over the bus. You are correct if both CPU and GPU are behind the same bus (such as if the GPU is on the same die), but modern GPUs support shared virtual memory even for non-integrated GPUs.