4 ms·

How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get bett

How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your <insert processor here> by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.

tgma 1y ago
It has a lot to do with your workload as well as if not moreso than the chip architecture. The primary trade-off is the cache utilization when executing two sets of instruction streams.
- hinkley 1y ago
  That's likely the primary factor, but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU.
  - gruez 1y ago
    >but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU. That doesn't make any sense. Disabling SMT likely saves negligible amount of power, but disables any performance to be gained from the other thread. If there's thermal budget available, it's better to spend it by shoving more work onto the second thread than to leave it disabled. If anything, due to voltage/frequency curves, it might even be better to run your CPU at lower clocks but with SMT enabled to make up for it (assuming it's amenable to your workloads), than it is to run with SMT disabled.
  - tgma 1y ago
    May be true for FMA or AVX2 or similar stuff. Outside vector units that sounds implausible. Obviously multi core thermal throttling is a thing but that would by far dominate. Hyperthreading should have minimal impact there.
twoodfin 1y ago
For whatever it’s worth, operational database systems (many users/connections, unpredictable access patterns) are beneficiaries of modern hyperthreading. I’m familiar with one such system where the throughput benefit is ~15%, which is a big deal for a BIOS flag. IBM’s POWER would have been discontinued a decade ago were it not for transactional database systems, and that architecture is heavily invested in SMT, up to 8-way(!)
- jiggawatts 1y ago
  I've noticed an overreliance on throughput as measured during 100% load as the performance metric, which has resulted in hardware vendors "optimising to the test" at the expense of other, arguably more important metrics. For example: single-user latency when the server is just 50% loaded.
  - twoodfin 1y ago
    That’s more than fair. In the system I’m most familiar with, however, the benefits of hyperthreading for throughput extend to the 50-70% utilization band where p99 latency is not stressed.
  - hinkley 1y ago
    Or p98 time for requests. Throughput and latency are usually at odds with each other.
- tom_ 1y ago
  Why do they need so many threads? This really feels like they just designed the cpu poorly, in that it can't extract enough parallelism out of the instruction stream already. (Intel and AMD stopped at 2! Apparently more wasn't worth it for them. Presumably because the cpu was doing enough of the right thing already.)
  - wmf 1y ago
    Power does have higher memory latency because of OMI and it supports more sockets. But I think the main motivation for SMT8 is low-IPC spaghetti code.
duped 1y ago
For me today it's definitely a pessimation because I have enough well-meaning applications that spawn `nproc` worker threads. Which would be fine if they're the only process running, but they're not.
- toast0 1y ago
  Wouldn't that be about the same badness without hyperthreads? If you're oversubscribed, there might be some benefit to having fewer tasks, but maybe you get some good throughput with two different application's threads running on opposite hyperthreads.
  - hinkley 1y ago
    Oversubscribing also leads to process migration, which these days leads to memory read delays.
- hinkley 1y ago
  I wrote a little tool for our services that could do basic expression based off of nproc based on an environment variable at startup time. You could do one thread for every two cores, three threads for every 2 cores, one thread per core ± 1, or both (2n + 1). Unfortunately the sweet spot based on our memory usage always came out to 1:1, except for a while when we had a memory leak that was surprisingly hard to fix, and we ran n - 1 for about 4 months while a bunch of work and exploratory testing were done. We had to tune in other places to maximize throughput.
FpUser 1y ago
In the old days it had made the difference between my multimedia game like application not working at all with hyperthreading off to working just fine with it on.
- hinkley 1y ago
  Yeah when it was one core versus 1.3 cores that's fair. But 3 core machines often did better (or at least more consistently run to run) with HT disabled.
BrendanLong 1y ago
To be fair, in most of these tests hyperthreading did provide a significant benefit (in the general CPU stress test, the hyperthreads increased performance by ~66%). It's just confusing that utilization metrics treat hyperthread usage the same as full physical cores.
tom_ 1y ago
Total throughout has always seemed better with it switched on for me, even for stuff that isn't hyper threading friendly. You get a free 10% at least.
toast0 1y ago
Going from 1 core to 2 hyperthreads was a big bonus in interactivity. But I think it was easy to get early systems to show worse throughput. I think there's two kinds of loads where hyperthreads aren't more likely to hurt than help. If you've got a tight loop that uses all the processor execution resources, you're not gaining anything by splitting that in two, it just makes things harder. Or if your load is mostly bound by memory bandwidth without a lot of compute... having more threads probably means you're that much more oversubscribed on i/o and caching. But a lot of loads are grab some stuff from memory and then do some compute, rinse and repeat. There's a lot of potential for idle time while waiting on a load, being able to run something else during that time makes a lot of sense. It's worth checking how your load performs with hyperthreads off, but I think default on is probably the right choice.
- sroussey 1y ago
  Definitely measure both ways and decide. For many years (still?) it was faster to run your database with hyper threading turned off and your app server with it turned on.
esseph 1y ago
Intel vs AMD, you'll get a different answer on the hyperthreading question. https://www.tomshardware.com/pc-components/cpus/zen-4-smt-focused-testing-suggests-intel-made-a-mistake-ditching-hyper-threading-on-lunar-lake https://www.tomshardware.com/pc-components/cpus/zen-4-smt-fo...
loeg 1y ago
HT provides a significant benefit to many workloads. The use cases that benefit from actually disabling HT are likely working around pessimal OS scheduler or application thread use. (After all, even with it enabled, you're free to not use the sibling cores.) Otherwise, it is an overgeneralization to say that disabling it will benefit arbitrary workloads.
- hedora 1y ago
  There’s some argument that you should jam stuff on to as few hyperthread pairs as possible to improve energy efficiency and cache locality. Of course, if the CPU governor is set to “performance” or “game mode”, then the OS should use as many pairs as possible instead (unless thermal throttling matters; computers are hard).
- robocat 1y ago
  > use cases that benefit from actually disabling HT Other benefits: per-CPU software licencing sometimes, and security on servers that share CPU with multiple clients.
- mkbosmans 1y ago
  Especially in HPC there are lots of workloads that do not benefit from SMT. Such workloads are almost always bottlenecked on either memory bandwidth or vector execution ports. These are exactly the resources that are shared between the sibling threads. So now you have a choice of either disabling SMT in the bios, or make sure the application correctly interprets the CPU topology and only spawns one thread per physical core. The former is often the easier option, both from software development and system administration perspective.
  - PunchyHamster 1y ago
    HT cores can still run OS stuff in that case as that isn't really in contention with those. Tho I can see someone not wanting to bother with pinning
  - skeezyboy 1y ago
    >Especially in HPC there are lots of workloads that do not benefit from SMT...So now you have a choice of either disabling SMT in the bios Thats madness. Theyre cheaper than their all-core equivalent. Why even buy one in the first place if HT slows down the CPU? Youre still better off with them enabled.
bee_rider 1y ago
Those weird Xeon Phi accelerators had 4 threads per core, and IIRC needed at least 2 running to get full performance. They were sort of niche, though. I guess in general parallelism inside a core will either be extracted by the computer automatically with instruction-level-parallelism, or the programmer can tell it about independent tasks, using hyperthreads. So the hyperthread implementations are optimistic about how much progrmmers care about performance, haha.
- mkbosmans 1y ago
  Sort of niche indeed. In addition to needing SMT to get full performance, there were a lot of other small details you needed to get right on Xeon Phi to get close to the advertised performance. Think of AVX512 and the HBM. For practical applications, it never really delivered.

tgma 1y ago

It has a lot to do with your workload as well as if not moreso than the chip architecture. The primary trade-off is the cache utilization when executing two sets of instruction streams.

hinkley 1y ago
That's likely the primary factor, but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU.
- gruez 1y ago
  >but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU. That doesn't make any sense. Disabling SMT likely saves negligible amount of power, but disables any performance to be gained from the other thread. If there's thermal budget available, it's better to spend it by shoving more work onto the second thread than to leave it disabled. If anything, due to voltage/frequency curves, it might even be better to run your CPU at lower clocks but with SMT enabled to make up for it (assuming it's amenable to your workloads), than it is to run with SMT disabled.
- tgma 1y ago
  May be true for FMA or AVX2 or similar stuff. Outside vector units that sounds implausible. Obviously multi core thermal throttling is a thing but that would by far dominate. Hyperthreading should have minimal impact there.

twoodfin 1y ago

For whatever it’s worth, operational database systems (many users/connections, unpredictable access patterns) are beneficiaries of modern hyperthreading. I’m familiar with one such system where the throughput benefit is ~15%, which is a big deal for a BIOS flag. IBM’s POWER would have been discontinued a decade ago were it not for transactional database systems, and that architecture is heavily invested in SMT, up to 8-way(!)

jiggawatts 1y ago
I've noticed an overreliance on throughput as measured during 100% load as the performance metric, which has resulted in hardware vendors "optimising to the test" at the expense of other, arguably more important metrics. For example: single-user latency when the server is just 50% loaded.
- twoodfin 1y ago
  That’s more than fair. In the system I’m most familiar with, however, the benefits of hyperthreading for throughput extend to the 50-70% utilization band where p99 latency is not stressed.
- hinkley 1y ago
  Or p98 time for requests. Throughput and latency are usually at odds with each other.
tom_ 1y ago
Why do they need so many threads? This really feels like they just designed the cpu poorly, in that it can't extract enough parallelism out of the instruction stream already. (Intel and AMD stopped at 2! Apparently more wasn't worth it for them. Presumably because the cpu was doing enough of the right thing already.)
- wmf 1y ago
  Power does have higher memory latency because of OMI and it supports more sockets. But I think the main motivation for SMT8 is low-IPC spaghetti code.

duped 1y ago

For me today it's definitely a pessimation because I have enough well-meaning applications that spawn `nproc` worker threads. Which would be fine if they're the only process running, but they're not.

toast0 1y ago
Wouldn't that be about the same badness without hyperthreads? If you're oversubscribed, there might be some benefit to having fewer tasks, but maybe you get some good throughput with two different application's threads running on opposite hyperthreads.
- hinkley 1y ago
  Oversubscribing also leads to process migration, which these days leads to memory read delays.
hinkley 1y ago
I wrote a little tool for our services that could do basic expression based off of nproc based on an environment variable at startup time. You could do one thread for every two cores, three threads for every 2 cores, one thread per core ± 1, or both (2n + 1). Unfortunately the sweet spot based on our memory usage always came out to 1:1, except for a while when we had a memory leak that was surprisingly hard to fix, and we ran n - 1 for about 4 months while a bunch of work and exploratory testing were done. We had to tune in other places to maximize throughput.

FpUser 1y ago

In the old days it had made the difference between my multimedia game like application not working at all with hyperthreading off to working just fine with it on.

hinkley 1y ago
Yeah when it was one core versus 1.3 cores that's fair. But 3 core machines often did better (or at least more consistently run to run) with HT disabled.

BrendanLong 1y ago

To be fair, in most of these tests hyperthreading did provide a significant benefit (in the general CPU stress test, the hyperthreads increased performance by ~66%). It's just confusing that utilization metrics treat hyperthread usage the same as full physical cores.

tom_ 1y ago

Total throughout has always seemed better with it switched on for me, even for stuff that isn't hyper threading friendly. You get a free 10% at least.

toast0 1y ago

Going from 1 core to 2 hyperthreads was a big bonus in interactivity. But I think it was easy to get early systems to show worse throughput. I think there's two kinds of loads where hyperthreads aren't more likely to hurt than help. If you've got a tight loop that uses all the processor execution resources, you're not gaining anything by splitting that in two, it just makes things harder. Or if your load is mostly bound by memory bandwidth without a lot of compute... having more threads probably means you're that much more oversubscribed on i/o and caching. But a lot of loads are grab some stuff from memory and then do some compute, rinse and repeat. There's a lot of potential for idle time while waiting on a load, being able to run something else during that time makes a lot of sense. It's worth checking how your load performs with hyperthreads off, but I think default on is probably the right choice.

sroussey 1y ago
Definitely measure both ways and decide. For many years (still?) it was faster to run your database with hyper threading turned off and your app server with it turned on.

esseph 1y ago

Intel vs AMD, you'll get a different answer on the hyperthreading question. https://www.tomshardware.com/pc-components/cpus/zen-4-smt-focused-testing-suggests-intel-made-a-mistake-ditching-hyper-threading-on-lunar-lake https://www.tomshardware.com/pc-components/cpus/zen-4-smt-fo...

loeg 1y ago

HT provides a significant benefit to many workloads. The use cases that benefit from actually disabling HT are likely working around pessimal OS scheduler or application thread use. (After all, even with it enabled, you're free to not use the sibling cores.) Otherwise, it is an overgeneralization to say that disabling it will benefit arbitrary workloads.

hedora 1y ago
There’s some argument that you should jam stuff on to as few hyperthread pairs as possible to improve energy efficiency and cache locality. Of course, if the CPU governor is set to “performance” or “game mode”, then the OS should use as many pairs as possible instead (unless thermal throttling matters; computers are hard).
robocat 1y ago
> use cases that benefit from actually disabling HT Other benefits: per-CPU software licencing sometimes, and security on servers that share CPU with multiple clients.
mkbosmans 1y ago
Especially in HPC there are lots of workloads that do not benefit from SMT. Such workloads are almost always bottlenecked on either memory bandwidth or vector execution ports. These are exactly the resources that are shared between the sibling threads. So now you have a choice of either disabling SMT in the bios, or make sure the application correctly interprets the CPU topology and only spawns one thread per physical core. The former is often the easier option, both from software development and system administration perspective.
- PunchyHamster 1y ago
  HT cores can still run OS stuff in that case as that isn't really in contention with those. Tho I can see someone not wanting to bother with pinning
- skeezyboy 1y ago
  >Especially in HPC there are lots of workloads that do not benefit from SMT...So now you have a choice of either disabling SMT in the bios Thats madness. Theyre cheaper than their all-core equivalent. Why even buy one in the first place if HT slows down the CPU? Youre still better off with them enabled.

bee_rider 1y ago

Those weird Xeon Phi accelerators had 4 threads per core, and IIRC needed at least 2 running to get full performance. They were sort of niche, though. I guess in general parallelism inside a core will either be extracted by the computer automatically with instruction-level-parallelism, or the programmer can tell it about independent tasks, using hyperthreads. So the hyperthread implementations are optimistic about how much progrmmers care about performance, haha.

mkbosmans 1y ago
Sort of niche indeed. In addition to needing SMT to get full performance, there were a lot of other small details you needed to get right on Xeon Phi to get close to the advertised performance. Think of AVX512 and the HBM. For practical applications, it never really delivered.