3 ms·
> NV has put a great deal of recent work into getting their system to execute regular multi-threaded code with the same sort of speedup as they have had with SI
by dragontamer 6y ago
> NV has put a great deal of recent work into getting their system to execute regular multi-threaded code with the same sort of speedup as they have had with SIMD workloads.
I think you're talking about "Independent Thread Scheduling", which has been deployed since Volta.
https://docs.nvidia.com/cuda/volta-tuning-guide/index.html#sm-independent-thread-scheduling https://docs.nvidia.com/cuda/volta-tuning-guide/index.html#s...
https://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf https://images.nvidia.com/content/volta-architecture/pdf/vol...
You're close... but wrong in a few key areas.
What NVidia has done is make it POSSIBLE to run multithreaded code without deadlocks. There's still the thread-divergence performance problem.
That is to say: you can now do more classical lock/unlock paradigms on NVidia hardware, but the code is serialized and becomes sequential. One 32-wide SIMD unit will execute only 1x thread at a time in that case. (That is: 1/32 utilization. The SM is designed to handle 32 SIMD threads but is only executing one of them at a time)
So you still have the thread-divergence performance problem. On older GPUs, such divergence would have been a DEADLOCK instead. So you've gone from 0% performance utilization to 3% utilization, which means your code actually will finish.
But it doesn't mean that your code actually has any degree of worthwhile performance. Indeed: CPUs have proven that time-and-time again, per-thread branch-prediction and prefetching is best for that kind of code.
If you have a great deal of divergent if/else statements in your code, you'll want to run on CPUs. NVidia GPUs can now handle those situations better (ie: without deadlocking if those if/else statements contain thread-synchronization statements), but they aren't actually high-performance in those use cases.
- ncmncm 6y agoI am corrected. I made the mistake of believing what the project manager said about it.