5 ms·

Linked-list chasing (as in vlovich123's example) isn't bandwidth-limited as much as it is latency: SMT helps here because you're able to enqueue multiple wait-s

by geocar 6y ago

Linked-list chasing (as in vlovich123's example) isn't bandwidth-limited as much as it is latency: SMT helps here because you're able to enqueue multiple wait-states "simultaneously".

jeffbee 6y ago
Anything you can do with SMT you can do without SMT using ILP instead. SMT does not grant a CPU extra cache-filling resources.
- geocar 6y ago
  SMT is easier to program. Walking a single linked-list is nothing; A garbage collector walks lots of them that tend to fan out for a bit.
  - jeffbee 6y ago
    SMT gives you some probability that thread X and thread Y are under-using the resources of the CPU but if you lose that bet you get antagonism. On Intel CPUs in particular there are only 2 slots for filling cache lines, and filling them randomly from main memory takes many cycles, so two threads can easily get starved. If thread X is chasing a linked list and thread Y is compressing an MPEG then it's brilliant.
  - gpsar 6y ago
    Software SMT is even better for concurrent traversals as you are not limited by the number of hw contexts https://www.linkedin.com/pulse/dont-stall-multitask-georgios-psaropoulos https://www.linkedin.com/pulse/dont-stall-multitask-georgios...
- dragontamer 6y ago
  Depends on the implementation of SMT. I know that POWER9 for example, reserves the superslices for different threads as you go from SMT1 (1-thread per core) up to SMT4 (4-threads per core). Thread#0 and Thread#1 get Slice0, while Thread#2 and Thread#3 get Slice1. -------- For an even more extreme example, Bulldozer's implementation of SMT / Cores / whatever you wanna call it... the L1 cache was split between the two cores. So you'd rather have 2-threads doing two different things (even if they shared the same decoder and significantly shared the same resources), because you'd "magically" get access to more L1 cache. Even on modern Skylake / Zen systems, some SMT resources are locked to one thread or the other. So you do in fact get more resources from SMT. On the other hand: you're absolutely right in that pointer-chasing can be done in parallel in a single thread due to ILP (and that ILP is probably even preferable to using more threads). Still, the fact that SMT systems are implemented so differently and so weirdly at times... it means that we can't really make general rules about SMT systems.
  - jeffbee 6y ago
    Yeah, thanks for adding details. SMT on Skylake and later is quirky, people don't appreciate that the CPU replicates some things, partitions others, and fights over some. The static partitioning in particular means that if you try to disable HT by taking all of the odd core numbers offline[+] you just wasted half of your L1 cache and iTLB entries. +: I know this sounds like an obviously-wrong approach to the problem, but I came across it at a large, well-known, public company.
vlovich123 6y ago
I agree it’s not bandwidth limited. But the CPU is stuck waiting around for the results of that operation. So offloading it to dedicated Hw let’s you do an important background task without needing the OS and CPU to spend any wall-clock time on it. Same reason you have DMAs and other coprocessors/accelerators in the first place. A good such solution will increase effective memory bandwidth (offloading heavy background-worthy tasks for the coprocessor) but also latency-sensitive tasks (eg walking a large set of linked lists, you don’t need to pull in a 256 byte cache line at a time and can instead pull the 8 bytes for the pointer). I don’t pretend that this is easy or an unbridled obvious idea. I recognize there are smarter people than me who have considered this (and from the posts clearly is being explored commercially). I am attracted to the idea on first principles that the CPU is served well by specialized coprocessors that have a performance profile better suited for some specific subset of operations. For example, memcpy is the easiest one. We still don’t have a simple (good) memcpy instruction/accelerator even though we know that an insanely large amount of work the CPU ends up doing is copying memory around in some way or another (yes Intel has one that isn’t useful/good performing).
- geocar 6y ago
  > I am attracted to the idea on first principles that the CPU is served well by specialized coprocessors that have a performance profile better suited for some specific subset of operations. It certainly sounds promising, that's for sure, e.g. https://link.springer.com/content/pdf/10.1007%2Fs00778-019-00581-w.pdf https://link.springer.com/content/pdf/10.1007%2Fs00778-019-0...

by geocar 6y ago

Linked-list chasing (as in vlovich123's example) isn't bandwidth-limited as much as it is latency: SMT helps here because you're able to enqueue multiple wait-states "simultaneously".

jeffbee 6y ago

Anything you can do with SMT you can do without SMT using ILP instead. SMT does not grant a CPU extra cache-filling resources.

geocar 6y ago
SMT is easier to program. Walking a single linked-list is nothing; A garbage collector walks lots of them that tend to fan out for a bit.
- jeffbee 6y ago
  SMT gives you some probability that thread X and thread Y are under-using the resources of the CPU but if you lose that bet you get antagonism. On Intel CPUs in particular there are only 2 slots for filling cache lines, and filling them randomly from main memory takes many cycles, so two threads can easily get starved. If thread X is chasing a linked list and thread Y is compressing an MPEG then it's brilliant.
- gpsar 6y ago
  Software SMT is even better for concurrent traversals as you are not limited by the number of hw contexts https://www.linkedin.com/pulse/dont-stall-multitask-georgios-psaropoulos https://www.linkedin.com/pulse/dont-stall-multitask-georgios...
dragontamer 6y ago
Depends on the implementation of SMT. I know that POWER9 for example, reserves the superslices for different threads as you go from SMT1 (1-thread per core) up to SMT4 (4-threads per core). Thread#0 and Thread#1 get Slice0, while Thread#2 and Thread#3 get Slice1. -------- For an even more extreme example, Bulldozer's implementation of SMT / Cores / whatever you wanna call it... the L1 cache was split between the two cores. So you'd rather have 2-threads doing two different things (even if they shared the same decoder and significantly shared the same resources), because you'd "magically" get access to more L1 cache. Even on modern Skylake / Zen systems, some SMT resources are locked to one thread or the other. So you do in fact get more resources from SMT. On the other hand: you're absolutely right in that pointer-chasing can be done in parallel in a single thread due to ILP (and that ILP is probably even preferable to using more threads). Still, the fact that SMT systems are implemented so differently and so weirdly at times... it means that we can't really make general rules about SMT systems.
- jeffbee 6y ago
  Yeah, thanks for adding details. SMT on Skylake and later is quirky, people don't appreciate that the CPU replicates some things, partitions others, and fights over some. The static partitioning in particular means that if you try to disable HT by taking all of the odd core numbers offline[+] you just wasted half of your L1 cache and iTLB entries. +: I know this sounds like an obviously-wrong approach to the problem, but I came across it at a large, well-known, public company.

vlovich123 6y ago

I agree it’s not bandwidth limited. But the CPU is stuck waiting around for the results of that operation. So offloading it to dedicated Hw let’s you do an important background task without needing the OS and CPU to spend any wall-clock time on it. Same reason you have DMAs and other coprocessors/accelerators in the first place. A good such solution will increase effective memory bandwidth (offloading heavy background-worthy tasks for the coprocessor) but also latency-sensitive tasks (eg walking a large set of linked lists, you don’t need to pull in a 256 byte cache line at a time and can instead pull the 8 bytes for the pointer). I don’t pretend that this is easy or an unbridled obvious idea. I recognize there are smarter people than me who have considered this (and from the posts clearly is being explored commercially). I am attracted to the idea on first principles that the CPU is served well by specialized coprocessors that have a performance profile better suited for some specific subset of operations. For example, memcpy is the easiest one. We still don’t have a simple (good) memcpy instruction/accelerator even though we know that an insanely large amount of work the CPU ends up doing is copying memory around in some way or another (yes Intel has one that isn’t useful/good performing).

geocar 6y ago
> I am attracted to the idea on first principles that the CPU is served well by specialized coprocessors that have a performance profile better suited for some specific subset of operations. It certainly sounds promising, that's for sure, e.g. https://link.springer.com/content/pdf/10.1007%2Fs00778-019-00581-w.pdf https://link.springer.com/content/pdf/10.1007%2Fs00778-019-0...