2 ms·
In a way, there are many more independent cores in an Nvidia GPU than #SMs, since the SIMD (warp) is only 32 threads wide, and >1k threads can be active at a ti
by desertrider12 6y ago
In a way, there are many more independent cores in an Nvidia GPU than #SMs, since the SIMD (warp) is only 32 threads wide, and >1k threads can be active at a time per SM. Each warp has its own independent program counter. So if you really wanted to, you could have a kernel with blocks that are Nx32, and code like:
if(blockIdx.x == 0)
doThing1();
else if(blockIdx.x == 1)
doThing2();
...
etc., and the doThings could run at the same time, even on the same SM. Not super practical but it's allowed.