4 ms·

When you say "lousy performance", do you mean HT lowers performance, or that it's not a great speed-up? I benchmarked HT using POV-Ray back when I first got an

When you say "lousy performance", do you mean HT lowers performance, or that it's not a great speed-up? I benchmarked HT using POV-Ray back when I first got an i7-3770 (4 cores, 8 threads) since it was my first multicore/multithread CPU. I did a benchmark with 1 thread, 2 threads, 4 threads, 8, and 16 threads. As expected, 2 was double the speed of 1, and 4 was double the speed of two. However, 8 was merely 10% faster than 4, and as expected, 16 was no faster than 8. Knowing how HT works, the fact that 8 threads was only 10% faster than 4 should be expected. POV-Ray rendering is almost entirely floating point math, so it can't take advantage of HT very well. I have heard of HT possibly lowering performance, but only in contrived examples. But HT was never meant to be a replacement for additional cores. It merely allows a single core to be used more effectively. There are probably some contrived examples of HT increasing performance by 50+% by running one thread with all integer math and the other with all floating point.

rbanffy 3y ago
> However, 8 was merely 10% faster than 4 Since all threads usually share the cache (early Intel implementations hard-partitioned L1 and, IIRC, L2), cache misses can easily limit throughput. The more a thread causes misses, the less of a performance increase you'll get from a higher SMT. Your observations indicate the cache is already limiting throughput and adding an extra thread to an already used core causes it to spend more time waiting for L3 or (most likely) main memory.
10000truths 3y ago
Port contention is not the only source of pipeline stalls. HT can also be used to run the second thread when the first one is stalled on a dependent memory access or branch misprediction.
_a_a_a_ 3y ago
Both. I have most definitely seen performance start to decrease after all physical cause get used and SMT starts. It was not a contrived example, it was using MS SQL. Now to be fair I don't understand the mechanism behind this and it may have been to do with the machine being limited in (say) memory channels because somebody put together a very strange spec for the purchase, or any number of other reasons that I don't know of, but I did observe this consistently
yencabulator 3y ago
Consider this scenario: Say one execution thread's working memory footprint just barely fits in cache. Now with 2 execution threads sharing that same slice of cache, you have more cache misses and trashing. And cache misses can have disastrous effect on performance. It doesn't matter that there's CPU available for hyperthreading to use during the cache misses. Visually, say we have 4 slots of cache available, we'll mark cache misses with capital letters, and we have an algorithm that goes like time goes this way--> ABCDabcdabcdabcdabcd Now put two threads there AB C D A EF G H E ^--cache now contains DAHE ^--cache now contains CDGH ^--cache now contains BCFG Related, I've seen situations where algorithm 1 wins every microbenchmark, while algorithm 2 is faster in the real app. Why? Because alg1 needed more icache space than the simpler alg2, and in the real app there was more icache pressure than in the microbenchmark.
KolenCh 3y ago
Not contrived examples. In scientific computing it is quite common and for example it is common practice to set OMP_NUM_THREADS=no of physical cores. As an overly simplified rule of thumb, for compute bound applications one should uses number of threads equals to no of physical cores; and for IO bound applications, set to no of logical cores.

rbanffy 3y ago

> However, 8 was merely 10% faster than 4 Since all threads usually share the cache (early Intel implementations hard-partitioned L1 and, IIRC, L2), cache misses can easily limit throughput. The more a thread causes misses, the less of a performance increase you'll get from a higher SMT. Your observations indicate the cache is already limiting throughput and adding an extra thread to an already used core causes it to spend more time waiting for L3 or (most likely) main memory.

10000truths 3y ago

Port contention is not the only source of pipeline stalls. HT can also be used to run the second thread when the first one is stalled on a dependent memory access or branch misprediction.

_a_a_a_ 3y ago

Both. I have most definitely seen performance start to decrease after all physical cause get used and SMT starts. It was not a contrived example, it was using MS SQL. Now to be fair I don't understand the mechanism behind this and it may have been to do with the machine being limited in (say) memory channels because somebody put together a very strange spec for the purchase, or any number of other reasons that I don't know of, but I did observe this consistently

yencabulator 3y ago

Consider this scenario: Say one execution thread's working memory footprint just barely fits in cache. Now with 2 execution threads sharing that same slice of cache, you have more cache misses and trashing. And cache misses can have disastrous effect on performance. It doesn't matter that there's CPU available for hyperthreading to use during the cache misses. Visually, say we have 4 slots of cache available, we'll mark cache misses with capital letters, and we have an algorithm that goes like time goes this way--> ABCDabcdabcdabcdabcd Now put two threads there AB C D A EF G H E ^--cache now contains DAHE ^--cache now contains CDGH ^--cache now contains BCFG Related, I've seen situations where algorithm 1 wins every microbenchmark, while algorithm 2 is faster in the real app. Why? Because alg1 needed more icache space than the simpler alg2, and in the real app there was more icache pressure than in the microbenchmark.

KolenCh 3y ago

Not contrived examples. In scientific computing it is quite common and for example it is common practice to set OMP_NUM_THREADS=no of physical cores. As an overly simplified rule of thumb, for compute bound applications one should uses number of threads equals to no of physical cores; and for IO bound applications, set to no of logical cores.