> If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's

> If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's caches. Shared memory by itself will not have a significant performance impact here (although it simplifies the data movement substantially). For a scalar-vector multiplication, you are strongly memory bound, and so you still have to wait for all the memory to be retrieved over the bus. You are correct if both CPU and GPU are behind the same bus (such as if the GPU is on the same die), but modern GPUs support shared virtual memory even for non-integrated GPUs.