That's my experience in a nutshell. The cost of doing a cudamemcpy() far outweighs the advantages for computationally small tasks. The surprising bit for me was

That's my experience in a nutshell. The cost of doing a cudamemcpy() far outweighs the advantages for computationally small tasks. The surprising bit for me was what's classed as a small task. Decompressing a 5MP jpg then applying various filters is too lightweight a task to benefit. I thought that would be more or less a perfect GPU task, not so. Running on OpenCL, the CPU with vector instructions horses a small GPU performance wise for this problem.