4 ms·
In general: * hyperthreading is useless for purely-numerical code (the inner loop of math/science software), and for this kind of code it is also often harmful
by o11c 3y ago
In general:
* hyperthreading is useless for purely-numerical code (the inner loop of math/science software), and for this kind of code it is also often harmful per the next point
* hyperthreading is harmful for code that is highly cache-sensitive within the scheduling quantum (though multiple cores may share caches too, and for the degenerate case of RAM-as-cache-for-swap people generally think in terms of "too many threads means more thrashing"; hyperthreading really isn't special there). This is more likely for "busy" tasks, but "background" tasks are likely to have no problem here.
* hyperthreading is beneficial for OO-like code that does a lot of pointer-chasing (which almost all software is, at least in part)
* hyperthreading might beat threads in separate cores for contended atomics
If you're trying to optimize particular code, it's generally better to eliminate the horrible memory patterns in the first place than expect a handwaved hardware "solution". Where it wins is for broader code that is too spread out for you to focus your attention on.
And of course, there is still a lot of code that is fundamentally single-threaded (at least in the critical path), and if allowed to hog the CPU will not see any benefit from hyperthreading.
For a long time hyperthreading was any easy win on desktop for all the various single-threaded background tasks. Now, with increasing core counts, specialized E- vs P-cores (and focus on power management in general), and Spectre/Meltdown, I'm not sure.
- ajross 3y agoThis is oversimplifying. Pipeline stalls aren't the same thing as "horrible memory patterns". Not all workloads can be optimized like that at all, some very legitimate workloads are inherently random access (think RAM caching and databases!) and we still want to run them on CPUs tuned for wide issue of in-register instructions. And when your very legitimate workload does stall, inevitably, it's nice if the CPU has another thread able to issue while it waits. In fact hyperthreading benefits almost all but the most heavily tuned workloads. Pretty much everyone stalls on a routine enough basis to make it worthwhile. Now, in the post-spectre world where you need to spend transistors on firewalling/tagging to prevent information leaks, maybe it's not as big a benefit. But it's mischaracterizing history to claim that it was never worthwhile.
- juggertao 3y agoYet GPUs which take HT to the next level by having thousands of "hyper-threads" work very well for scientific computing.
- pjmlp 3y agoTheir design is completly different, it isn't shared execution units like it happens on CPUs.
- juggertao 3y agoBut it is. GPUs have many more threads in flight than execution units.
- pjmlp 3y agoThreads groups get exclusive resources in SIMT execution pipelines.
- juggertao 3y agoAnd at memory stall they are exchanged with other waiting thread groups. Just like HT.
- RealHusky 3y agoScheduling algorithm is different. CPUs target low latency (they switch often). GPUs target high troughput (they switch rarely, only when needed). High troughput algorithms dont have problem with a lot of threads. Low latency algorithms have problem with a lot of threads (they need lot of cache memory because of constant switching).