3 ms·
Considering the source material is a file, that needs to be loaded up by the CPU first, and the operation itself is dead simple, the real constraint is RAM band
by kaetemi 6y ago
Considering the source material is a file, that needs to be loaded up by the CPU first, and the operation itself is dead simple, the real constraint is RAM bandwidth.
I would guess that the time of getting the data out to the GPU and back, is about the same as just calculating the difference on the CPU.
So, the quickest solution is probably to use the CPUs vector operations, and read ahead into the registers so you can exhaust the memory bandwidth fully.
- amelius 6y agoI recently tried this with a 19 megabyte .tiff file in PyTorch. Uploading the tensor took (i think) around 1 or 2 seconds, wall-clock time. So I quickly dismissed the GPU solution to my problem. But I'm still wondering... is this normal?
- kaetemi 6y agoYou've got your GPU program that needs to be compiled to actual GPU code at startup as well. That takes a bit. The actual file transfer is comparable to the GPU doing a memcpy. (You could also use pinned host memory if the GPU program is just doing a one-off read.) GPU processing for simple operations really shines if your data is already on the GPU and your code is hot. Otherwise, it's more useful if the process is significantly complex enough to make the overhead of a cold startup insignificant.
- Kuinox 6y agoGPU can now directly read the file on the SSD.
- plasticchris 6y agoCitation needed? I know things have been announced but what's available now?
- harias 6y agoGPUDirect has been available for quite sometime now: https://developer.nvidia.com/gpudirect https://developer.nvidia.com/gpudirect
- deleted 6y ago[deleted]
- plasticchris 6y agoThanks! I forgot storage was supported by that.