3 ms·

Even with no cache misses, SMT can still be a win, since it can increase instruction level parallelism - a single thread can may not have enough instructions be

by grogers 17y ago

Even with no cache misses, SMT can still be a win, since it can increase instruction level parallelism - a single thread can may not have enough instructions being able to execute in parallel to completely fill the pipeline. But with SMT, multiple threads have the chance to fill all available execution units which is much more likely. There are probably pathological cases where disabling SMT would be an improvement but I think in general, it should always improve throughput. The main question is whether using the transistors to go towards SMT would be better spent elsewhere. But already the cores on the chip take a very small percent of the die - most of it is cache. And adding SMT doesn't increase the core size very much (a couple percent IIRC), so it seems like a no brainer to me as long as the OS scheduler can do a decent job.

AlisdairO 17y ago
It often depends on how related the datasets of the two hyperthreads are. If both HTs are working on the same data, then it can often be a win for the reason you state - increased instruction level parallelism, and of course the opportunity to do work while one thread is stalled on a non-overlappable miss. If the two threads are not so related, they can easily trash each others cache and have a substantially negative impact upon performance. Ulrich Drepper has a pretty good discussion on page 29 of this doc ( http://people.redhat.com/drepper/cpumemory.pdf http://people.redhat.com/drepper/cpumemory.pdf ). It boils down to this: Code that has a low cache hit rate can often benefit from HT, as it can usually be parallelised into two threads that have a not-much-worse cache hit rate, and there will be plenty of gaps to fill. More optimised code stands to lose quite significantly, as parallelising code with a high cache hit rate to (a) not trash each others cache, and (b) still have a very high hit rate, is very hard.
- grogers 17y ago
  Note that his example is about a processor that can only execute one thread at a time, switching on a cache miss. This is not SMT. The main problem is if the working set size of the single threaded code fits just in cache, but the combined working set size of both threads does not. In this case the threads are competing for the shared L1 cache instead of doing useful work.
  - AlisdairO 17y ago
    That's not quite right - his example models execution as one thread at a time, switching on a cache miss because most of the extra work that gets done by SMT is when you get a cache miss, freeing up the rest of the processor. While extra work can also be done when there isn't a stall (for example, when the processor is unable to sufficiently parallelise execution of the one thread), his argument is that this is a relatively insignificant fraction of the gain produced by HT, so his model is sufficiently accurate. Now, I don't know practically whether this is true or not, but ulrich drepper ought to :-).

by grogers 17y ago

AlisdairO 17y ago

It often depends on how related the datasets of the two hyperthreads are. If both HTs are working on the same data, then it can often be a win for the reason you state - increased instruction level parallelism, and of course the opportunity to do work while one thread is stalled on a non-overlappable miss. If the two threads are not so related, they can easily trash each others cache and have a substantially negative impact upon performance. Ulrich Drepper has a pretty good discussion on page 29 of this doc ( http://people.redhat.com/drepper/cpumemory.pdf http://people.redhat.com/drepper/cpumemory.pdf ). It boils down to this: Code that has a low cache hit rate can often benefit from HT, as it can usually be parallelised into two threads that have a not-much-worse cache hit rate, and there will be plenty of gaps to fill. More optimised code stands to lose quite significantly, as parallelising code with a high cache hit rate to (a) not trash each others cache, and (b) still have a very high hit rate, is very hard.

grogers 17y ago
Note that his example is about a processor that can only execute one thread at a time, switching on a cache miss. This is not SMT. The main problem is if the working set size of the single threaded code fits just in cache, but the combined working set size of both threads does not. In this case the threads are competing for the shared L1 cache instead of doing useful work.
- AlisdairO 17y ago
  That's not quite right - his example models execution as one thread at a time, switching on a cache miss because most of the extra work that gets done by SMT is when you get a cache miss, freeing up the rest of the processor. While extra work can also be done when there isn't a stall (for example, when the processor is unable to sufficiently parallelise execution of the one thread), his argument is that this is a relatively insignificant fraction of the gain produced by HT, so his model is sufficiently accurate. Now, I don't know practically whether this is true or not, but ulrich drepper ought to :-).