4 ms·

Could someone here with experience tell me about how important hyper-threading is in whatever code they have experience with? In my prehistoric experience we g

Could someone here with experience tell me about how important hyper-threading is in whatever code they have experience with? In my prehistoric experience we got support from Intel to put hyper-threading code in a game around 2001-- that would have been with a Xeon processor. Our mandate was clear but unwritten, have a particular level of the game perform much better on an Intel CPU then any other x86 and then talk about it. As I recall we really had to jump through hoops to get a meaningful performance improvement. It was more like we slightly crippled non hyper-threading cpus on a handful of assets. We wanted to sell the game everywhere. It was like, if you looked at a particular rock in a particular level you really could measure the difference. It probably got a lot better with the i7, but experiences like that make me take new CPU features with a grain of salt. Hopefully someone can share more contemporary experience with hyper-threading.

jiggawatts 3y ago
Hyperthreading works really well when the same code is running on each core, but the data dependencies are different. Databases and ray tracing are probably the two best examples of this, but any parallel code that does a lot of random memory access or "pointer chasing" will benefit a lot. With typical database servers, hyperthreading provides about 1.4x the performance "for free" because it allows execution units to do something useful while another thread is waiting for memory. Where it isn't useful is typical game engines, except in some corner-cases as you've noticed. Typical game engines like to have dedicated cores with the highest possible performance. Hyperthreading improves overall throughput at the expense of single-threaded throughput. Similarly, game engines tend to run task-per-thread with different code, which tends to pollute the L1 code cache too much. So if two different threads are running on the same core but are doing different things, they'll fight over this precious resource. Intel measured something like a +20% overall improvement averaged out across a wide range of workloads. A few had regressions, and a few (like ray tracing) had huge benefits. IMHO, hyperthreading makes the most sense for servers, or laptops with a small number of cores (2-4). For high-end gaming, 16 cores with HT off will likely have the best performance.
- SleepyMyroslav 3y ago
  AMD chips with hyperthreading are the reality of console gaming. It works pretty well for couple of generations and anything from before that should be viewed as ancient history not reflecting current state of the art of games and engines.
fulladder 3y ago
Xeon was/is their workstation line, isn't it? Just curious how you got into a situation where you wanted to optimize a game for that type of hardware. Do gamers actually have Xeon hardware? (Why?)
- Clubber 3y ago
  Mainly ECC ram and I believe they're better at floating point calculations, but I'm not an expert. You can have multiple processors on a board too. Here's a high level explanation. https://www.hp.com/us-en/shop/tech-takes/why-should-i-upgrade-to-intel-xeon-processor https://www.hp.com/us-en/shop/tech-takes/why-should-i-upgrad... Here's Reddit's take https://www.reddit.com/r/explainlikeimfive/comments/66vy2r/eli5_why_are_xeon_processors_better_suited_for/ https://www.reddit.com/r/explainlikeimfive/comments/66vy2r/e...
shaggie76 3y ago
I dimly recall 10-15 years ago that we saw approximately 10% faster running texture filtering on HT cores in addition to real cores -- I'm hazy on the number and I don't remember which CPU we tested with but it seemed as though for a fetch-heavy workload SMT can help hide a cache-miss on one thread by doing ALU on the twin core. Since it didn't cost anything and we already scheduled work across real cores we didn't complain about the gains being marginal from HT cores and left it enabled. More recently we did some DirectX 12 MT-rendering tests; since there's more considerable overhead for breaking up the work into more batches so we wanted to be sure so we checked carefully. On a i7-12700K we found that distributing Dx12 work to SMT or E-Cores had no benefit, even in a contrived rendering stress-test. However, on an unnamed game console, we found that scheduling additional work to the SMT cores improved draw performance by maybe 7%. YMMV but my impression is the workload needs stall on memory enough to get signfican gains by interleaving the ALU from another thread and this might benefit low-power systems like laptops or mobile phones more than workstations.
o11c 3y ago
In general: * hyperthreading is useless for purely-numerical code (the inner loop of math/science software), and for this kind of code it is also often harmful per the next point * hyperthreading is harmful for code that is highly cache-sensitive within the scheduling quantum (though multiple cores may share caches too, and for the degenerate case of RAM-as-cache-for-swap people generally think in terms of "too many threads means more thrashing"; hyperthreading really isn't special there). This is more likely for "busy" tasks, but "background" tasks are likely to have no problem here. * hyperthreading is beneficial for OO-like code that does a lot of pointer-chasing (which almost all software is, at least in part) * hyperthreading might beat threads in separate cores for contended atomics If you're trying to optimize particular code, it's generally better to eliminate the horrible memory patterns in the first place than expect a handwaved hardware "solution". Where it wins is for broader code that is too spread out for you to focus your attention on. And of course, there is still a lot of code that is fundamentally single-threaded (at least in the critical path), and if allowed to hog the CPU will not see any benefit from hyperthreading. For a long time hyperthreading was any easy win on desktop for all the various single-threaded background tasks. Now, with increasing core counts, specialized E- vs P-cores (and focus on power management in general), and Spectre/Meltdown, I'm not sure.
- ajross 3y ago
  This is oversimplifying. Pipeline stalls aren't the same thing as "horrible memory patterns". Not all workloads can be optimized like that at all, some very legitimate workloads are inherently random access (think RAM caching and databases!) and we still want to run them on CPUs tuned for wide issue of in-register instructions. And when your very legitimate workload does stall, inevitably, it's nice if the CPU has another thread able to issue while it waits. In fact hyperthreading benefits almost all but the most heavily tuned workloads. Pretty much everyone stalls on a routine enough basis to make it worthwhile. Now, in the post-spectre world where you need to spend transistors on firewalling/tagging to prevent information leaks, maybe it's not as big a benefit. But it's mischaracterizing history to claim that it was never worthwhile.
- juggertao 3y ago
  Yet GPUs which take HT to the next level by having thousands of "hyper-threads" work very well for scientific computing.

jiggawatts 3y ago

Hyperthreading works really well when the same code is running on each core, but the data dependencies are different. Databases and ray tracing are probably the two best examples of this, but any parallel code that does a lot of random memory access or "pointer chasing" will benefit a lot. With typical database servers, hyperthreading provides about 1.4x the performance "for free" because it allows execution units to do something useful while another thread is waiting for memory. Where it isn't useful is typical game engines, except in some corner-cases as you've noticed. Typical game engines like to have dedicated cores with the highest possible performance. Hyperthreading improves overall throughput at the expense of single-threaded throughput. Similarly, game engines tend to run task-per-thread with different code, which tends to pollute the L1 code cache too much. So if two different threads are running on the same core but are doing different things, they'll fight over this precious resource. Intel measured something like a +20% overall improvement averaged out across a wide range of workloads. A few had regressions, and a few (like ray tracing) had huge benefits. IMHO, hyperthreading makes the most sense for servers, or laptops with a small number of cores (2-4). For high-end gaming, 16 cores with HT off will likely have the best performance.

SleepyMyroslav 3y ago
AMD chips with hyperthreading are the reality of console gaming. It works pretty well for couple of generations and anything from before that should be viewed as ancient history not reflecting current state of the art of games and engines.

fulladder 3y ago

Xeon was/is their workstation line, isn't it? Just curious how you got into a situation where you wanted to optimize a game for that type of hardware. Do gamers actually have Xeon hardware? (Why?)

Clubber 3y ago
Mainly ECC ram and I believe they're better at floating point calculations, but I'm not an expert. You can have multiple processors on a board too. Here's a high level explanation. https://www.hp.com/us-en/shop/tech-takes/why-should-i-upgrade-to-intel-xeon-processor https://www.hp.com/us-en/shop/tech-takes/why-should-i-upgrad... Here's Reddit's take https://www.reddit.com/r/explainlikeimfive/comments/66vy2r/eli5_why_are_xeon_processors_better_suited_for/ https://www.reddit.com/r/explainlikeimfive/comments/66vy2r/e...

shaggie76 3y ago

I dimly recall 10-15 years ago that we saw approximately 10% faster running texture filtering on HT cores in addition to real cores -- I'm hazy on the number and I don't remember which CPU we tested with but it seemed as though for a fetch-heavy workload SMT can help hide a cache-miss on one thread by doing ALU on the twin core. Since it didn't cost anything and we already scheduled work across real cores we didn't complain about the gains being marginal from HT cores and left it enabled. More recently we did some DirectX 12 MT-rendering tests; since there's more considerable overhead for breaking up the work into more batches so we wanted to be sure so we checked carefully. On a i7-12700K we found that distributing Dx12 work to SMT or E-Cores had no benefit, even in a contrived rendering stress-test. However, on an unnamed game console, we found that scheduling additional work to the SMT cores improved draw performance by maybe 7%. YMMV but my impression is the workload needs stall on memory enough to get signfican gains by interleaving the ALU from another thread and this might benefit low-power systems like laptops or mobile phones more than workstations.

o11c 3y ago

In general: * hyperthreading is useless for purely-numerical code (the inner loop of math/science software), and for this kind of code it is also often harmful per the next point * hyperthreading is harmful for code that is highly cache-sensitive within the scheduling quantum (though multiple cores may share caches too, and for the degenerate case of RAM-as-cache-for-swap people generally think in terms of "too many threads means more thrashing"; hyperthreading really isn't special there). This is more likely for "busy" tasks, but "background" tasks are likely to have no problem here. * hyperthreading is beneficial for OO-like code that does a lot of pointer-chasing (which almost all software is, at least in part) * hyperthreading might beat threads in separate cores for contended atomics If you're trying to optimize particular code, it's generally better to eliminate the horrible memory patterns in the first place than expect a handwaved hardware "solution". Where it wins is for broader code that is too spread out for you to focus your attention on. And of course, there is still a lot of code that is fundamentally single-threaded (at least in the critical path), and if allowed to hog the CPU will not see any benefit from hyperthreading. For a long time hyperthreading was any easy win on desktop for all the various single-threaded background tasks. Now, with increasing core counts, specialized E- vs P-cores (and focus on power management in general), and Spectre/Meltdown, I'm not sure.

ajross 3y ago
This is oversimplifying. Pipeline stalls aren't the same thing as "horrible memory patterns". Not all workloads can be optimized like that at all, some very legitimate workloads are inherently random access (think RAM caching and databases!) and we still want to run them on CPUs tuned for wide issue of in-register instructions. And when your very legitimate workload does stall, inevitably, it's nice if the CPU has another thread able to issue while it waits. In fact hyperthreading benefits almost all but the most heavily tuned workloads. Pretty much everyone stalls on a routine enough basis to make it worthwhile. Now, in the post-spectre world where you need to spend transistors on firewalling/tagging to prevent information leaks, maybe it's not as big a benefit. But it's mischaracterizing history to claim that it was never worthwhile.
juggertao 3y ago
Yet GPUs which take HT to the next level by having thousands of "hyper-threads" work very well for scientific computing.