3 ms·
Nvidia has been working on this a lot lately. Check out CUDA Graphs. It's not there yet, but they're working on reducing kernel launch times. You still have
by 01100011 7y ago
Nvidia has been working on this a lot lately. Check out CUDA Graphs. It's not there yet, but they're working on reducing kernel launch times. You still have the overhead of moving data to/from the GPU, but if you can come up with a complicated workflow that can run on the GPU it may be worth it.