2 ms·
I'm jealous. Dan is reading my twitter and re-posting stuff here :) https://twitter.com/majek04/status/1267388797278400512 https://twitter.com/majek04/status/1
by majke 6y ago
I'm jealous. Dan is reading my twitter and re-posting stuff here :)
https://twitter.com/majek04/status/1267388797278400512 https://twitter.com/majek04/status/1267388797278400512
You should definitely follow me on twitter :)
Anyway, the question is about optimally using GPGPU resources. It's desired to keep both ALU units and memory units busy, and it's not trivial what ratio of memory vs alu operations should there be. This paper addresses it _very_ nicely.
> Substituting the hardware parameters listed in Table 4.1 into the solutions above suggests that hiding arithmetic latency requires 24 warps per SM and hiding memory latency requires 30 warps per SM if on the Maxwell GPU – which are similar numbers. This is despite the dramatic difference in latencies –6 cycles in one case and 368 cycles in another – and in contrast with the prevailing common wisdom.
The paper doesn't answer if there is a disadvantage in having too many warps, but considering that the max warps/block/sm is 32, we can say that you need 32 warps / 1024 threads per SM for optimal performance most often.
- rrss 6y agoNote the results you quoted are for a test case in which all instructions are back-to-back dependent, which is good for testing the hardware and developing performance models, but not representative of well-tuned GPU codes. > doesn't answer if there is a disadvantage in having too many warps, There are disadvantages in having too many warps - the first is that the more warps per block you have, the fewer registers/thread you can use and the lower your achievable instruction level parallelism. > but considering that the max warps/block/sm is 32, we can say that you need 32 warps / 1024 threads per SM for optimal performance most often I don't think any well-tuned GPU kernels use 32 warps/block, most are at 4 to 8 warps/block. See also: https://github.com/NervanaSystems/maxas/wiki/SGEMM https://github.com/NervanaSystems/maxas/wiki/SGEMM https://www.nvidia.com/content/GTC-2010/pdfs/2238_GTC2010.pdf https://www.nvidia.com/content/GTC-2010/pdfs/2238_GTC2010.pd...
- majke 6y agoThis is great. Thanks