6 ms·

>This suggests a long-term compromise solution where threads within a process can use hyperthreading to share a core, but threads in different processes can't.

by mww09 8y ago

>This suggests a long-term compromise solution where threads within a process can use hyperthreading to share a core, but threads in different processes can't. Given that hyperthreads share L1 cache, this might also be better for performance. Intuitively this may sound logical, however in practice it's often not the case. For many workloads putting two threads of the same program on a core ends up being worse than co-locating with threads from different programs. The reason is that two threads of the same program will often end up executing similar instruction streams (a really good example is when both are using vector instructions (these registers are shared between the two hyperthreads)).

amelius 8y ago
> The reason is that two threads of the same program will often end up executing similar instruction streams Why is that bad?
- xeroaura 8y ago
  Taking a guess, but since they are running similar streams, they have similar loads at a specific time. Competition between main thread and hyperthread could hurt performance instead?
- uryga 8y ago
  "both are using vector instructions (these registers are shared between the two hyperthreads)" So I guessing GP meant there's going to be contention for those registers, and thus no speedup?
- jfoutz 8y ago
  Different instruction streams use different registers. It’s like sharing a bathroom. I can shower while you brush your teeth. There’s more contention when we’re both trying to shower.
- CHY872 8y ago
  Your processor has a certain number of execution units which can actually execute individual instructions, maybe like 4 floating point units, maybe 8 arithmetic ones, and maybe 1 that can do vector processing (these numbers are not real, but are like, good enough for sake of message). So the idea with SMT is that most of the time, lots of the execution units are unused because the thread a) isn't using them at all (e.g. a process to do encryption won't use the floating point units) and/or b) can't use them all because of how the program's written (for example, if I say 'load a random memory address, then add it to a register, then load another random memory address, then add it, etc' I'm going to be spending most of my time waiting for memory to be loaded. SMT basically means that you run another program at the same time, so even if the encryption process can't use the floating point units, maybe there's another process that we can schedule that will. However, imagine my encryption process can use 6 of the 8 arithmetic units. If I have 2 encryption processes scheduled on the same core, I have demand for 12 when there are only 8. So now I have contention for resources, and I won't see a speedup from using SMT. Other comments mention registers and not execution units: I'm suspicious of this, since modern processors have many registers (for Skylake, 250+) which they remap between aggressively as part of pipelining. Maybe this is different for the SIMD units. That said, I haven't looked at this stuff since university so could well be wrong on the execution unit vs register comparison.
  - blattimwind 8y ago
    The contention would actually not be on a per-EU level, but one level higher up. The reservation station has a bunch (~5-8) of ports and typically multiple EUs are connected to one port. Can't use one port for two different things at the same time. Here's a simplified block diagram of a Skylake core: https://en.wikichip.org/wiki/intel/microarchitectures/skylake_(client)#Individual_Core https://en.wikichip.org/wiki/intel/microarchitectures/skylak...
    - CHY872 8y ago
      Thanks! Yeah figured I'd be wrong somewhere in there!
      - Twirrim 8y ago
        There's also some funkiness around the CPU cache, at least at one stage. If your two HT threads are working on the same data, there was a chance you'd get some great cache performance out of it. However the hyperthread when faced with a cache miss, can cause the cache to get evicted to be replaced with the data it needs. Under those circumstances, performance takes quite a nose dive as both threads are stomping over each other somewhat. Hyperthreading can be a real mixed bag for performance, though generally good and a lot of engineering effort has gone in to making it shine. As ever it's strongly advisable that people benchmark real world conditions on a server, and it's worth giving a shot with hyperthreading turned on and off.
  - gpderetta 8y ago
    You are pretty much right. A couple of things to add: even hand optimized asm code wont be able to use all ports all the time with a single instruction steam; the biggest win for hyperthreading is filling the pipeline bubbles caused by memory loads out of L1 (there is only so mach that OoO scheduling can do on your average load)
  - amelius 8y ago
    So on a RISC architecture, this would happen even for non-similar programs, because the number of instructions is smaller? Or would they just duplicate the processing units?
ajross 8y ago
In practice it sometimes is the case, though. SMT/hyperthreading is complicated. If you have a workload dominated by non-local DRAM fetches, it's a huge win because when the CPU pipeline is stalled on one thread it can still issue instructions from the other. If you have a workload dominated by L1 cache bandwidth, the opposite is true because the threads compete for the same resource. On balance, on typical workloads, it's a win. But there are real-world problems for which turning it off is a legitimate performance choice.
- greglindahl 8y ago
  Typical workloads? What's that? People run hugely diverse workloads on cpus, and they change over time.
  - ajross 8y ago
    Building software, serving web pages, executing database queries, running a DOM layout, managing game logic... I mean, come on. You knew what I meant. Those are all tasks with "medium" cache residency and "occasional" stalls on DRAM. Anything that does a bunch of different things with a big-ish world of data. Conversely: finding a task that is L1-cache-bound but does not frequently have to stall for memory is much harder. The only ones off the top of my head are streaming tasks like software video decode.
    - greglindahl 8y ago
      Oh, you meant typical for you. One task that is L1 cache bound and does not frequently stall for memory (if you code it up well) is matrix multiply.
      - kbenson 8y ago
        > Oh, you meant typical for you. I'm pretty sure those are meant to be, and I think are, "typical" for the general purpose CPU in use, and thus the general case. Both mobile and desktop CPUs will be doing DOM layout, DB queries (whether to SQLite or the registry or just the filesystem), and possibly computing game logic on a regular basis.

by mww09 8y ago

amelius 8y ago

> The reason is that two threads of the same program will often end up executing similar instruction streams Why is that bad?

xeroaura 8y ago
Taking a guess, but since they are running similar streams, they have similar loads at a specific time. Competition between main thread and hyperthread could hurt performance instead?
uryga 8y ago
"both are using vector instructions (these registers are shared between the two hyperthreads)" So I guessing GP meant there's going to be contention for those registers, and thus no speedup?
jfoutz 8y ago
Different instruction streams use different registers. It’s like sharing a bathroom. I can shower while you brush your teeth. There’s more contention when we’re both trying to shower.
CHY872 8y ago
Your processor has a certain number of execution units which can actually execute individual instructions, maybe like 4 floating point units, maybe 8 arithmetic ones, and maybe 1 that can do vector processing (these numbers are not real, but are like, good enough for sake of message). So the idea with SMT is that most of the time, lots of the execution units are unused because the thread a) isn't using them at all (e.g. a process to do encryption won't use the floating point units) and/or b) can't use them all because of how the program's written (for example, if I say 'load a random memory address, then add it to a register, then load another random memory address, then add it, etc' I'm going to be spending most of my time waiting for memory to be loaded. SMT basically means that you run another program at the same time, so even if the encryption process can't use the floating point units, maybe there's another process that we can schedule that will. However, imagine my encryption process can use 6 of the 8 arithmetic units. If I have 2 encryption processes scheduled on the same core, I have demand for 12 when there are only 8. So now I have contention for resources, and I won't see a speedup from using SMT. Other comments mention registers and not execution units: I'm suspicious of this, since modern processors have many registers (for Skylake, 250+) which they remap between aggressively as part of pipelining. Maybe this is different for the SIMD units. That said, I haven't looked at this stuff since university so could well be wrong on the execution unit vs register comparison.
- blattimwind 8y ago
  The contention would actually not be on a per-EU level, but one level higher up. The reservation station has a bunch (~5-8) of ports and typically multiple EUs are connected to one port. Can't use one port for two different things at the same time. Here's a simplified block diagram of a Skylake core: https://en.wikichip.org/wiki/intel/microarchitectures/skylake_(client)#Individual_Core https://en.wikichip.org/wiki/intel/microarchitectures/skylak...
  - CHY872 8y ago
    Thanks! Yeah figured I'd be wrong somewhere in there!
    - Twirrim 8y ago
      There's also some funkiness around the CPU cache, at least at one stage. If your two HT threads are working on the same data, there was a chance you'd get some great cache performance out of it. However the hyperthread when faced with a cache miss, can cause the cache to get evicted to be replaced with the data it needs. Under those circumstances, performance takes quite a nose dive as both threads are stomping over each other somewhat. Hyperthreading can be a real mixed bag for performance, though generally good and a lot of engineering effort has gone in to making it shine. As ever it's strongly advisable that people benchmark real world conditions on a server, and it's worth giving a shot with hyperthreading turned on and off.
- gpderetta 8y ago
  You are pretty much right. A couple of things to add: even hand optimized asm code wont be able to use all ports all the time with a single instruction steam; the biggest win for hyperthreading is filling the pipeline bubbles caused by memory loads out of L1 (there is only so mach that OoO scheduling can do on your average load)
- amelius 8y ago
  So on a RISC architecture, this would happen even for non-similar programs, because the number of instructions is smaller? Or would they just duplicate the processing units?

ajross 8y ago

In practice it sometimes is the case, though. SMT/hyperthreading is complicated. If you have a workload dominated by non-local DRAM fetches, it's a huge win because when the CPU pipeline is stalled on one thread it can still issue instructions from the other. If you have a workload dominated by L1 cache bandwidth, the opposite is true because the threads compete for the same resource. On balance, on typical workloads, it's a win. But there are real-world problems for which turning it off is a legitimate performance choice.

greglindahl 8y ago
Typical workloads? What's that? People run hugely diverse workloads on cpus, and they change over time.
- ajross 8y ago
  Building software, serving web pages, executing database queries, running a DOM layout, managing game logic... I mean, come on. You knew what I meant. Those are all tasks with "medium" cache residency and "occasional" stalls on DRAM. Anything that does a bunch of different things with a big-ish world of data. Conversely: finding a task that is L1-cache-bound but does not frequently have to stall for memory is much harder. The only ones off the top of my head are streaming tasks like software video decode.
  - greglindahl 8y ago
    Oh, you meant typical for you. One task that is L1 cache bound and does not frequently stall for memory (if you code it up well) is matrix multiply.
    - kbenson 8y ago
      > Oh, you meant typical for you. I'm pretty sure those are meant to be, and I think are, "typical" for the general purpose CPU in use, and thus the general case. Both mobile and desktop CPUs will be doing DOM layout, DB queries (whether to SQLite or the registry or just the filesystem), and possibly computing game logic on a regular basis.