3 ms·
This only seems to applicable to shaders? GPGPU in most contexts is referring to code divorced from any graphics pipeline, like CUDA or ROCm. CUDA has had async
by oivey 6y ago
This only seems to applicable to shaders? GPGPU in most contexts is referring to code divorced from any graphics pipeline, like CUDA or ROCm. CUDA has had asynchronous and concurrent kernel calls for a long time. How are asynchronous compute shaders relevant in that context?
- monocasa 6y agoCompute shaders are divorced from the graphics pipeline as well. As far as the hardware is concerned, CUDA/ROCm and compute shaders are the same thing.
- Jasper_ 6y agoCompute shaders and CUDA have very different execution and driver models. Just take a look at the way memory is managed in each.
- monocasa 6y agoCan you be specific? Everything I've seen looks like a user space host driver difference. Also, I the case of "asynchronous shaders" it's a command list dispatching feature, which is equally applicable to CUDA/ROCm and compute shaders.
- oivey 6y agoCUDA has had asynchronous command lists (multiple, actually, via streams) forever. What does AMD’s special hardware enable? Is it exposed in ROCm somehow? Edit: Maybe I should be more precise with my language. Using CUDA streams, you can queue up and execute multiple kernels (basically commands?) in parallel. Kernels invoked on the same stream are serialized. I have no idea what the exact analog is in the compute shader world.
- monocasa 6y ago"Asynchronous shaders" doesn't mean that host code is unblocked on job dispatch (that's been the case for nearly all GPU APIs for going on 40 years). It's having multiple hardware queues with different priorities to be able to have other work to do when a kernel/shader is only using a subset of the SMs/CUs. So it's a hardware balancing feature for non heterogeneous workloads. Nvidia hardware won't dispatch multiple kernels simultaneously, all of the streams stuff you're seeing is hints to the user space driver in how it orders the work in it's single hardware command queue.
- oivey 6y agoIs that not what this is? https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#concurrent-kernel-execution https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.... I thought this allowed kernels to run in parallel if both kernels could fit in the open resources. It doesn't have a priority system, and I'm guessing isn't smart enough to reorder kernels to fit together better. Further Googling to be sure that they mean parallel AND concurrent didn't turn up much, but Nvidia does appear to mean both.
- my123 6y agocudaStreamCreateWithPriority is what you'd like to use for creating a stream with a specifically defined priority. The GPU is able to run up to 128 kernels in parallel on modern NVIDIA GPUs, concurrently.
- oivey 6y agoAhh ok. I’ve had a similar conversation with a graphics guy that strongly paralleled this conversation, and I was confused then, too. Seemingly the asynchronous compute issues are a limitation when it only comes to shaders I guess? I write some CUDA but no shaders, and certainly not so much that I’d make super concrete claims about more esoteric features like streams.
- Jasper_ 6y agoHave you seen CUDA? It's C++, where half of it runs on the CPU, and half of it runs on the GPU. Memory transfers are automatically scheduled for you! No, obviously the hardware doesn't magically transform when you run a CUDA program, but there's a lot of features in the hardware that just aren't exercised by compute shaders, and the 'user-space host driver' is very different. > Also, I the case of "asynchronous shaders" it's a command list dispatching feature, which is equally applicable to CUDA/ROCm and compute shaders. Async compute affects how you have to synchronize your code, and what your hazards are. If you have a queue of items, you know when one starts that the other is done. With async compute, you can give up that guarantee in exchange for overlapping work. NVIDIA still doesn't have this in modern GPUs.
- my123 6y agoYou have async compute on NVIDIA GPUs. See this slide deck about it: https://developer.download.nvidia.com/CUDA/training/StreamsAndConcurrencyWebinar.pdf https://developer.download.nvidia.com/CUDA/training/StreamsA... (for the implementation in Fermi, it's even more flexible since then)