4 ms·

The dropping hyperthreading is interesting. I'm hoping x64 will trend in the other direction - I'd like four or eight threads scheduled across a pool of executi

The dropping hyperthreading is interesting. I'm hoping x64 will trend in the other direction - I'd like four or eight threads scheduled across a pool of execution units as a way of hiding memory latency. Sort of a step towards the GPU model.

jandrewrogers 2y ago
This has not been successful historically because software developers struggle to design code that has mechanical sympathy with these architectural models. These types of processors will run idiomatic CPU code with mediocre efficiency, which provides an element of backward compatibility, but achieving their theoretical efficiency requires barrel-processing style data structures and algorithms, which isn't something widely taught or known to software developers. I like these architectures but don't expect it to happen. Every attempt to produce a CPU that works this way has done poorly because software developers don't know how to use them properly. The hardware companies have done the "build it and they will come" thing multiple times and it has never panned out for them. You would need a killer app that would strongly incentivize software developers to learn how design efficient code for these architectures.
- bri3d 2y ago
  I think another huge issue is the combination of general-purpose/multi-tenant computing, especially with virtualization in the mix, and cache. Improvements gained from masking memory latency by issuing instructions from more thread execution contexts are suddenly lost when each of those thread contexts is accessing a totally different part of memory and evicting the cache line that the next thread wanted anyway. In many ways barrel processing and hyperthreading are remarkably similar to VLIW - they're two very different sides of the same coin. At the end of the day, both work well for situations where multiple simultaneous instruction streams need to access relatively small amounts of adjacent data. Realtime signal processing, dedicated databases, and gaming are obvious applications here, which I think is why VLIW has done so well in DSP and hyperthreading did well in gaming and databases. Once the simultaneous instruction streams are accessing completely disparate parts of main memory, it all blows up. Plus, in a multi-tenant environment, the security issues inherent to sharing even more execution state context (and therefore side channels) between domains also become untenable fairly quickly.
- pradn 2y ago
  Hyperthreading is also useful for simply making a little progress on more threads. Even if you're not pinning a core by making full use of two hyperthreads, you can handle double the threads at once. Now, I don't know how important it is, but I assume that for desktop applications, this could lead to a snappier experience. Most code on the desktop is just waiting for memory loads and context switching. Of course, the big elephant in the room is security - timing attacks on shared cores is a big problem. Sharing anything is a big problem for security conscious customers. Maybe it's the case of the server leading the client here.
  - immibis 2y ago
    Just this morning I benchmarked my 7960X on RandomX (i.e. Monero mining). That's a 24-core CPU. With 48 cores, it gets about 10kH/s and uses about 100 watts extra. With 48 threads, about 15kH/s and 150 watts. It does make sense with the nature of RandomX. Another benchmark I've done is .onion vanity address mining. Here it's about a 20% improvement in total throughput when using hyperthreading. It's definitely not useless. However, I didn't compare to a scenario with hyperthreading disabled in the BIOS. Are you telling me the threads get 20-50% faster, each, with it disabled?
Covzire 2y ago
I've been disabling HT for security reasons on all my machines ever since the first vulnerabilities appeared, with no discernible negative impact on daily usage.
- groos 2y ago
  The gain from HT for doing large builds has been 20% at most. Daily usage is indistinguishable, as you say.
  - MisterTea 2y ago
    HT really only benefits you if you spend a lot of time waiting on IO. E.g. file serving where you're waiting on slow spinning disks.
    - coolspot 2y ago
      IO wait happens on OS kernel level, not on CPU level. RAM fetch latency is what happens on CPU level.
- binkHN 2y ago
  For security reasons, OpenBSD disables hyperthreading by default.
- duffyjp 2y ago
  I'm running a quite old CPU currently, a 6-core Haswell-E i7-5930K. Disabling HT gave me a huge boost in gaming workloads. I'm basically never doing anything that pegs the entire CPU so getting that extra 10-15% IPC for single threaded tasks is huge. And like you said, the vulnerability consideration makes HT a hard sell for workloads that would benefit (hypervisors). That i7 is a huge downgrade from what I was running before (long story) so I'm looking forward to Arrow Lake and I like everything I've read. In addition to removing HT, they're supporting Native JEDEC DDR5-6400 Memory so XMP won't be necessary. I've never liked XMP/Expo...
AzzyHN 2y ago
To summarize Intel's reasoning, the extra hardware (and I guess associated firmware or whatever it'd be called) required to manage hyperthreading in their P-cores takes up package space and power, meaning the core can't boost as high. And since hyperthreading only increases IPC by 30% (according to Intel), they're planning on making up the loss of threads with more E-cores. But we'll have to see how that turns out, especially since Intel's first chiplet design (the Core Ultra series 1) had performance degradations compared to their 13th Gen mobile counterparts
killerstorm 2y ago
That's what SPARC and Power architectures do now, with 8 threads per core. The only issues with these architectures is that they are priced as "high-end server stuff" while x64 is priced like a commodity.

jandrewrogers 2y ago

This has not been successful historically because software developers struggle to design code that has mechanical sympathy with these architectural models. These types of processors will run idiomatic CPU code with mediocre efficiency, which provides an element of backward compatibility, but achieving their theoretical efficiency requires barrel-processing style data structures and algorithms, which isn't something widely taught or known to software developers. I like these architectures but don't expect it to happen. Every attempt to produce a CPU that works this way has done poorly because software developers don't know how to use them properly. The hardware companies have done the "build it and they will come" thing multiple times and it has never panned out for them. You would need a killer app that would strongly incentivize software developers to learn how design efficient code for these architectures.

bri3d 2y ago
I think another huge issue is the combination of general-purpose/multi-tenant computing, especially with virtualization in the mix, and cache. Improvements gained from masking memory latency by issuing instructions from more thread execution contexts are suddenly lost when each of those thread contexts is accessing a totally different part of memory and evicting the cache line that the next thread wanted anyway. In many ways barrel processing and hyperthreading are remarkably similar to VLIW - they're two very different sides of the same coin. At the end of the day, both work well for situations where multiple simultaneous instruction streams need to access relatively small amounts of adjacent data. Realtime signal processing, dedicated databases, and gaming are obvious applications here, which I think is why VLIW has done so well in DSP and hyperthreading did well in gaming and databases. Once the simultaneous instruction streams are accessing completely disparate parts of main memory, it all blows up. Plus, in a multi-tenant environment, the security issues inherent to sharing even more execution state context (and therefore side channels) between domains also become untenable fairly quickly.
pradn 2y ago
Hyperthreading is also useful for simply making a little progress on more threads. Even if you're not pinning a core by making full use of two hyperthreads, you can handle double the threads at once. Now, I don't know how important it is, but I assume that for desktop applications, this could lead to a snappier experience. Most code on the desktop is just waiting for memory loads and context switching. Of course, the big elephant in the room is security - timing attacks on shared cores is a big problem. Sharing anything is a big problem for security conscious customers. Maybe it's the case of the server leading the client here.
- immibis 2y ago
  Just this morning I benchmarked my 7960X on RandomX (i.e. Monero mining). That's a 24-core CPU. With 48 cores, it gets about 10kH/s and uses about 100 watts extra. With 48 threads, about 15kH/s and 150 watts. It does make sense with the nature of RandomX. Another benchmark I've done is .onion vanity address mining. Here it's about a 20% improvement in total throughput when using hyperthreading. It's definitely not useless. However, I didn't compare to a scenario with hyperthreading disabled in the BIOS. Are you telling me the threads get 20-50% faster, each, with it disabled?

Covzire 2y ago

I've been disabling HT for security reasons on all my machines ever since the first vulnerabilities appeared, with no discernible negative impact on daily usage.

groos 2y ago
The gain from HT for doing large builds has been 20% at most. Daily usage is indistinguishable, as you say.
- MisterTea 2y ago
  HT really only benefits you if you spend a lot of time waiting on IO. E.g. file serving where you're waiting on slow spinning disks.
  - coolspot 2y ago
    IO wait happens on OS kernel level, not on CPU level. RAM fetch latency is what happens on CPU level.
binkHN 2y ago
For security reasons, OpenBSD disables hyperthreading by default.
duffyjp 2y ago
I'm running a quite old CPU currently, a 6-core Haswell-E i7-5930K. Disabling HT gave me a huge boost in gaming workloads. I'm basically never doing anything that pegs the entire CPU so getting that extra 10-15% IPC for single threaded tasks is huge. And like you said, the vulnerability consideration makes HT a hard sell for workloads that would benefit (hypervisors). That i7 is a huge downgrade from what I was running before (long story) so I'm looking forward to Arrow Lake and I like everything I've read. In addition to removing HT, they're supporting Native JEDEC DDR5-6400 Memory so XMP won't be necessary. I've never liked XMP/Expo...

AzzyHN 2y ago

To summarize Intel's reasoning, the extra hardware (and I guess associated firmware or whatever it'd be called) required to manage hyperthreading in their P-cores takes up package space and power, meaning the core can't boost as high. And since hyperthreading only increases IPC by 30% (according to Intel), they're planning on making up the loss of threads with more E-cores. But we'll have to see how that turns out, especially since Intel's first chiplet design (the Core Ultra series 1) had performance degradations compared to their 13th Gen mobile counterparts

killerstorm 2y ago

That's what SPARC and Power architectures do now, with 8 threads per core. The only issues with these architectures is that they are priced as "high-end server stuff" while x64 is priced like a commodity.