5 ms·

Are there any scheduling experts on here? So I've always been under the impression that HT just schedules instructions on the unused functional units. So say y

Are there any scheduling experts on here? So I've always been under the impression that HT just schedules instructions on the unused functional units. So say your CPU has 100 adders, and when you resolve all your dependencies for incoming instructions, you can only use 60 of those adders when running instructions in parallel (out of order), so the HT/logical core uses the other 40 adders for another thread (so the HT cores get a lower priority than the standard cores; hence why operating system that are HT aware can be more efficient by scheduling lower priority threads on the HT cores). Is that correct or am I way off? (HT wasn't a thing when I took architecture class. I had a dual AthlonXP back then, where I had two physically separate processors).

w0utert 8y ago
Yes I think it more or less works that way. A hyperthread has its own set of registers and stack and everything, but for doing any actual work it shares the same functional blocks as any other thread running on the same core. I do think that the hyperthreading parts increase the numbers of some functional blocks like load/store units and such compared to non-hyperthreading parts though, to reduce contention (someone correct me if I'm wrong). What this means is that hyperthreading does not really work so well on sustained, homogenuous workloads. For example doing very heavy computation on 8 threads of a 4-core CPU with 8 hardware threads, can actually reduce performance, because all threads will be contending for the same functional blocks of the CPU.
- bartread 8y ago
  > For example doing very heavy computation on 8 threads of a 4-core CPU with 8 hardware threads, can actually reduce performance, because all threads will be contending for the same functional blocks of the CPU. We have exactly this scenario on a server we're running (non-virtualised) and, if it didn't involve rebooting the machine, I'd already have run a test to benchmark whether we get better performance with or without hyperthreading. Unfortunately our test environments aren't similar enough to be representative in terms of testing yet, but this is going to change in the next few weeks so I'll finally be able to run my test without needing to take down the production box.
  - stock_toaster 8y ago
    You could try offlining the HT cores in the linux scheduler, and see if it makes a difference (no reboot required). Example: https://www.golinuxhub.com/2018/01/how-to-disable-or-enable-hyper.html https://www.golinuxhub.com/2018/01/how-to-disable-or-enable-...
    - bartread 8y ago
      Unfortunately we're running Windows Server 2016. I may be wrong - be nice if I was, actually - but I thought it required a reboot.
- pizza234 8y ago
  > hyperthreading does not really work so well on sustained, homogenuous workloads what would distinguish this description from, say, compression and compiling? based on the experience on my machine (which of course is limited), HT does give a boost in those two cases, which could be classified as homogeneous and sustained.
  - dragontamer 8y ago
    Compiling isn't homogeneous IMO. Lots of branches, lots of waiting of RAM, lots of corner cases. Lots of ways to get those hyperthreads working in parallel. Compression would be closer to homogeneous. But something REALLY homogeneous is like, Prime95. You're only hitting SSE and/or AVX instructions over-and-over again. All threads try to only use AVX instructions, so hyperthreading doesn't help too much.
    - berti 8y ago
      HT also doesn't help as much on FP workloads simply because there are less FP execution units to share, e.g. Skylake: https://en.wikichip.org/wiki/intel/microarchitectures/skylake_(client)#Individual_Core https://en.wikichip.org/wiki/intel/microarchitectures/skylak...
  - nordsieck 8y ago
    >> hyperthreading does not really work so well on sustained, homogenuous workloads > what would distinguish this description from, say, compression and compiling? The real difference is between integer and floating point workloads. Integer workloads typically have lots of branch mis-predicts and cache misses which cause pipeline stalls. Any time you've got pipeline stalls, hyper-threading will be great. Floating Point workloads typically are just huge amounts of number crunching with very simple access patterns. There are few branches and memory accesses typically have a consistent stride and so are pre-fetch friendly. Typically, this kind of workload is memory bandwidth limited because your CPU can run full bore without any bubbles in the pipeline. Hyper-threading isn't much use in this case: if any functional units are going unused, it's only because the CPU can't vacuum data up fast enough. This is one of the big reasons GPUs are so popular for doing FP workloads: they have a ton of functional units and they are paired with very high speed on-board memory. In the grandparent's defense, floating point workloads tend to be sustained and homogenous. They are, however, not the only kind of sustained, homogenous workloads, as you so accurately point out.
- Sohcahtoa82 8y ago
  > What this means is that hyperthreading does not really work so well on sustained, homogenuous workloads. For example doing very heavy computation on 8 threads of a 4-core CPU with 8 hardware threads, can actually reduce performance, because all threads will be contending for the same functional blocks of the CPU. When I got my i7-3770K (4 cores, 8 threads) years ago, I was into POV-Ray rendering. I did a test using only 1, 2, 4, and 8 threads. As expected, 2 threads was double the speed of 1, 4 threads was double the speed of 2, but 8 threads was only about 15% faster than 4. Thinking about it now, I wonder what power consumption looked like.
cesarb 8y ago
AFAIK no, HT siblings do not get a lower priority. The core's resources are shared equally between the SMT threads. Operating systems that are SMT aware try to avoid this sharing when possible (a non-SMT-aware OS could schedule a pair of runnable threads on one core while another core is idle; a SMT-aware OS would use the idle core instead, so both threads get a full core for themselves).
brudgers 8y ago
I am certainly not an expert. My high level understanding of hyper-threading comes from having a Pentium 4 with it back in the early 00's [1]. Things may have changed since then, so I could be wrong...anyway, a simple model of hyperthreading would be a CPU with two operations: load and execute. Each operation takes one CPU cycle. Hyperthreading is possible when both load and execute can run each and every clock cycle. Cycle 0: Thread 1 loads Cycle 1: Thread 1 executes & thread 2 loads Cycle 2: Thread 1 loads & thread 2 executes Cycle 3: etc. The advantage of hyperthreading is that there is little or no context switching overhead. At least relative to traditional hardware threads. With a real CPU there are more steps in the pipeline and each step takes a variable amount of time. Later CPU's take advantage of those available cycles with parallel, speculative and out of order execution...but those are generally abstractions over a single thread. [1]: https://en.wikipedia.org/wiki/Pentium_4 https://en.wikipedia.org/wiki/Pentium_4
gpderetta 8y ago
HT can help fill unused execution units (which are in the order of a dozen not hundreds unfortunately) as they are over-provisioned compared to what a single thread could possibly use [1]. But the biggest win is when HT can fill pipelines holes caused by a single thread waiting for some long latency operation (almost invariably memory accesses). Normally out of order execution can fill these holes by extracting parallelism from a single thread of execution, but that's not always possible when multiple high or unpredictable latency operations are chained together.

w0utert 8y ago

Yes I think it more or less works that way. A hyperthread has its own set of registers and stack and everything, but for doing any actual work it shares the same functional blocks as any other thread running on the same core. I do think that the hyperthreading parts increase the numbers of some functional blocks like load/store units and such compared to non-hyperthreading parts though, to reduce contention (someone correct me if I'm wrong). What this means is that hyperthreading does not really work so well on sustained, homogenuous workloads. For example doing very heavy computation on 8 threads of a 4-core CPU with 8 hardware threads, can actually reduce performance, because all threads will be contending for the same functional blocks of the CPU.

bartread 8y ago
> For example doing very heavy computation on 8 threads of a 4-core CPU with 8 hardware threads, can actually reduce performance, because all threads will be contending for the same functional blocks of the CPU. We have exactly this scenario on a server we're running (non-virtualised) and, if it didn't involve rebooting the machine, I'd already have run a test to benchmark whether we get better performance with or without hyperthreading. Unfortunately our test environments aren't similar enough to be representative in terms of testing yet, but this is going to change in the next few weeks so I'll finally be able to run my test without needing to take down the production box.
- stock_toaster 8y ago
  You could try offlining the HT cores in the linux scheduler, and see if it makes a difference (no reboot required). Example: https://www.golinuxhub.com/2018/01/how-to-disable-or-enable-hyper.html https://www.golinuxhub.com/2018/01/how-to-disable-or-enable-...
  - bartread 8y ago
    Unfortunately we're running Windows Server 2016. I may be wrong - be nice if I was, actually - but I thought it required a reboot.
pizza234 8y ago
> hyperthreading does not really work so well on sustained, homogenuous workloads what would distinguish this description from, say, compression and compiling? based on the experience on my machine (which of course is limited), HT does give a boost in those two cases, which could be classified as homogeneous and sustained.
- dragontamer 8y ago
  Compiling isn't homogeneous IMO. Lots of branches, lots of waiting of RAM, lots of corner cases. Lots of ways to get those hyperthreads working in parallel. Compression would be closer to homogeneous. But something REALLY homogeneous is like, Prime95. You're only hitting SSE and/or AVX instructions over-and-over again. All threads try to only use AVX instructions, so hyperthreading doesn't help too much.
  - berti 8y ago
    HT also doesn't help as much on FP workloads simply because there are less FP execution units to share, e.g. Skylake: https://en.wikichip.org/wiki/intel/microarchitectures/skylake_(client)#Individual_Core https://en.wikichip.org/wiki/intel/microarchitectures/skylak...
- nordsieck 8y ago
  >> hyperthreading does not really work so well on sustained, homogenuous workloads > what would distinguish this description from, say, compression and compiling? The real difference is between integer and floating point workloads. Integer workloads typically have lots of branch mis-predicts and cache misses which cause pipeline stalls. Any time you've got pipeline stalls, hyper-threading will be great. Floating Point workloads typically are just huge amounts of number crunching with very simple access patterns. There are few branches and memory accesses typically have a consistent stride and so are pre-fetch friendly. Typically, this kind of workload is memory bandwidth limited because your CPU can run full bore without any bubbles in the pipeline. Hyper-threading isn't much use in this case: if any functional units are going unused, it's only because the CPU can't vacuum data up fast enough. This is one of the big reasons GPUs are so popular for doing FP workloads: they have a ton of functional units and they are paired with very high speed on-board memory. In the grandparent's defense, floating point workloads tend to be sustained and homogenous. They are, however, not the only kind of sustained, homogenous workloads, as you so accurately point out.
Sohcahtoa82 8y ago
> What this means is that hyperthreading does not really work so well on sustained, homogenuous workloads. For example doing very heavy computation on 8 threads of a 4-core CPU with 8 hardware threads, can actually reduce performance, because all threads will be contending for the same functional blocks of the CPU. When I got my i7-3770K (4 cores, 8 threads) years ago, I was into POV-Ray rendering. I did a test using only 1, 2, 4, and 8 threads. As expected, 2 threads was double the speed of 1, 4 threads was double the speed of 2, but 8 threads was only about 15% faster than 4. Thinking about it now, I wonder what power consumption looked like.

cesarb 8y ago

AFAIK no, HT siblings do not get a lower priority. The core's resources are shared equally between the SMT threads. Operating systems that are SMT aware try to avoid this sharing when possible (a non-SMT-aware OS could schedule a pair of runnable threads on one core while another core is idle; a SMT-aware OS would use the idle core instead, so both threads get a full core for themselves).

brudgers 8y ago

I am certainly not an expert. My high level understanding of hyper-threading comes from having a Pentium 4 with it back in the early 00's [1]. Things may have changed since then, so I could be wrong...anyway, a simple model of hyperthreading would be a CPU with two operations: load and execute. Each operation takes one CPU cycle. Hyperthreading is possible when both load and execute can run each and every clock cycle. Cycle 0: Thread 1 loads Cycle 1: Thread 1 executes & thread 2 loads Cycle 2: Thread 1 loads & thread 2 executes Cycle 3: etc. The advantage of hyperthreading is that there is little or no context switching overhead. At least relative to traditional hardware threads. With a real CPU there are more steps in the pipeline and each step takes a variable amount of time. Later CPU's take advantage of those available cycles with parallel, speculative and out of order execution...but those are generally abstractions over a single thread. [1]: https://en.wikipedia.org/wiki/Pentium_4 https://en.wikipedia.org/wiki/Pentium_4

gpderetta 8y ago

HT can help fill unused execution units (which are in the order of a dozen not hundreds unfortunately) as they are over-provisioned compared to what a single thread could possibly use [1]. But the biggest win is when HT can fill pipelines holes caused by a single thread waiting for some long latency operation (almost invariably memory accesses). Normally out of order execution can fill these holes by extracting parallelism from a single thread of execution, but that's not always possible when multiple high or unpredictable latency operations are chained together.