3 ms·
You can have multiple CUDA kernels running simultaneously on the same GPU, but you have no direct control over which SM(X) core is assigned to which kernel AFAI
by m_mueller 12y ago
You can have multiple CUDA kernels running simultaneously on the same GPU, but you have no direct control over which SM(X) core is assigned to which kernel AFAIK. So it actually works pretty similarly to multi threaded programming on CPU if you take away thread affinity. In general I find a good way to approximate a top-of the line NVIDIA GPU is to think of it as an ~8 core with a vector length of 192 for single precision and 96 (half of full length) for double precision. It has a high memory bandwidth which has the limitation of requiring memory accesses to 32 neighbours simultaneously in order to make full use of the performance. The CUDA programming model is particularly set up in a way that the programmer doesn't have to handle this manually - (s)he just needs to be aware of it. I.e. you program everything scalar, map the thread indices to your data accesses and make sure that the first thread index (x) maps to the fastest varying index of your data.
- kllrnohj 12y ago> In general I find a good way to approximate a top-of the line NVIDIA GPU is to think of it as an ~8 core with a vector length of 192 for single precision and 96 (half of full length) for double precision. But that's not entirely correct either. Yes you can use it like that, but you can also use it as a single core with a vector length of 1536. In the context of over-simplification these are better thought of as single-core processors. The reason being if you have method foo() that you need to run 10,0000 times, it doesn't matter if you use 1 thread, 2 threads, or 8 threads - the total time it will take to complete the work will be identical. This is very different from an 8-core CPU where using 8 threads will be 8x faster than using 1 thread (blah blah won't be perfectly linear etc, etc).