5 ms·

> Hyperthreads aren't real CPU cores I can't think of a good principled reason to say SMT cores aren't "real". I'm assuming the answer isn't "because they sha

> Hyperthreads aren't real CPU cores I can't think of a good principled reason to say SMT cores aren't "real". I'm assuming the answer isn't "because they share some computing stuff". What I think you'd call "real" cores also share resources like L2/L3 caches, sometimes DMA engines, etc. And IIRC, each Intel SMT (hyper-thread) unit has its on instruction pointer and (non-SIMD?) register set.

toast0 4y ago
> And IIRC, each Intel SMT (hyper-thread) unit has its on instruction pointer and (non-SIMD?) register set. I believe the SMTs share the register rename storage, which I'd say is the register set more than the 'architectural registers' But I'd say the reason they're not real is because they don't increase the maximum instructions per clock. With many loads, they do increase the average instructions per clock, but they don't let you do anymore work if you've got a fully tuned load that uses all the computing resources.
- eloff 4y ago
  In reality you never have a finely tuned load that uses all the resources all of the time. Processors spend a lot of time waiting for memory, so hyperthreading allows for better utilization of compute. It’s rare to have a workload that benefits from turning it off, and in those cases it’s usually because the hyperthread is hurting the cache hit rate enough to offset the gains.
  - hinkley 4y ago
    Both threads are waiting on the same memory pipeline. The actual performance has always lagged behind the theory
    - eloff 4y ago
      But not the same memory operations. Unless your code fully saturates the memory bandwidth, which is rare, you get some gains here.
      - AdrianB1 4y ago
        It is almost never about saturating the memory bandwidth, but waiting to load instructions and data from memory. That wait time counted in computer cycles is huge.
        eloff 4y ago
        Yes, that’s precisely why hyperthreading is such a good deal.
        hinkley 4y ago
        “Such” a good deal is 2-30% in benchmarks (and they don’t say if that’s with cache leak protections turned on or off). In previous generations it was more like -20-20%. If one thread is having issues with L1 and L2 cache, splitting that evenly with a completely different workflow isn’t going to help. If cache contention weren’t a problem, and it was just a matter of jumping into previously unseen instructions and data (cold cache), you’d expect to see 50-300% numbers from hyperthreading, precisely because of how long the stalls are.
    - kevin_thibedeau 4y ago
      If one thread has to hit main memory but the other one can get what it needs from L1 cache they aren't competing for the same resources.
  - MauranKilom 4y ago
    I recently checked whether an ugly hack I implemented years ago for 30% performance gains ("halve the default OpenMP thread count at start of computation and restore it afterwards" -> effectively, disable hyperthreading) was still necessary. It's now apparently a 3x performance gain... All because I'm saturating memory bandwidth. I don't know whether to be happy or sad...
    - eloff 4y ago
      Ouch, looks like they’re clobbering each others cache. Also, well done. It looks like you optimized the hell out of your code.
- FpUser 4y ago
  Many years ago a customer on a single core computer had problems of heavy video stutter in my product. It was multithreaded and ran as I recall 5 threads (GUI, video decoding, 3d pipeline, device control and computation). Others did not have this problem. After investigation it turned out that said customer had hyperthreading disabled in BIOS. Enabling it fixed the problem instantly. So "real" or not but from my experience HT does work to the benefit.
- hinkley 4y ago
  I don’t know how many hardware revisions Intel went through where every benchmark said hyperthreading was slower than turning it off, but it was a lot. It became Lucy’s football at some point. And in these days of post-Dennard scaling, you have thermal throttling, so idle cycles aren’t actually idle, they’re allowing the heat sink to catch up with heat production.
- zamadatix 4y ago
  I agree with this stance. It's the same as branch prediction, out of order execution, or anything else about computing efficiently.
bakugo 4y ago
> I can't think of a good principled reason to say SMT cores aren't "real". It really depends on your definition of "real", yes you can treat them like "real" independent cores but that's not ideal for performance because under the hood they're not actually independent. Your operating system is aware of this and will often avoid scheduling two tasks onto the same physical core unless it has to. If you have 20 logical cores (10 physical ones with hyperthreading) and 10 tasks to execute, the OS scheduler will usually allocate one task to each physical core and leave the other logical core idle.
- Sohcahtoa82 4y ago
  I did an experiment and effectively proved this to myself many years ago. I had just upgraded to an i7-3770K (8 threads, 4 cores) from a Core 2 Quad (4 threads, 4 cores). I did a POV-Ray render several times using 1, 2, 4, and 8 threads. 2 was nearly double the speed of 1, 4 was nearly double the speed of 2, but 8 was only about 15% faster than 4. To ensure I wasn't bottlenecking RAM at that level, I tried again with 2 threads, but forced both threads onto a single core, and it was only 15% faster than 1 thread. That was all the proof I needed of how you really can't treat two CPU threads as two cores. That said, I've never personally found an instance where allowing a processes to span all CPU threads actually reduced performance, and I'm not sure I've ever seen a real-world case where it does. It's usually something contrived.
  - duffyjp 4y ago
    When I bought my current desktop the two CPUs I was looking at were the i7-9700K and i9-9900K. This was the generation where intel went from 6 cores with HT to 8 cores w/o HT for the i7. The i9 has 8 cores with HT. I liked the idea of the i7 because without HT you have a real picture of how utilized your CPU is. If it says 100% on a core it's 100%. I ended up going with the i9 though when a promotion was too good to pass up. When I got it I ran Cinebench both with and without HT enabled in the BIOS and it made a decent impact so I left it on.
  - jeffbee 4y ago
    That's one use case. SMT is not designed to address it. POV-Ray will saturate your execution resources in the core and adding another frontend doesn't change the facts. SMT is designed to hide memory load latency. In this use case it is completely brilliant. You will get 2x speedups with hyperthreading when randomly accessing memory.
fmajid 4y ago
On AMD, yes, but Intel Hyperthreading is not real SMT.

toast0 4y ago

> And IIRC, each Intel SMT (hyper-thread) unit has its on instruction pointer and (non-SIMD?) register set. I believe the SMTs share the register rename storage, which I'd say is the register set more than the 'architectural registers' But I'd say the reason they're not real is because they don't increase the maximum instructions per clock. With many loads, they do increase the average instructions per clock, but they don't let you do anymore work if you've got a fully tuned load that uses all the computing resources.

eloff 4y ago
In reality you never have a finely tuned load that uses all the resources all of the time. Processors spend a lot of time waiting for memory, so hyperthreading allows for better utilization of compute. It’s rare to have a workload that benefits from turning it off, and in those cases it’s usually because the hyperthread is hurting the cache hit rate enough to offset the gains.
- hinkley 4y ago
  Both threads are waiting on the same memory pipeline. The actual performance has always lagged behind the theory
  - eloff 4y ago
    But not the same memory operations. Unless your code fully saturates the memory bandwidth, which is rare, you get some gains here.
    - AdrianB1 4y ago
      It is almost never about saturating the memory bandwidth, but waiting to load instructions and data from memory. That wait time counted in computer cycles is huge.
      - eloff 4y ago
        Yes, that’s precisely why hyperthreading is such a good deal.
        hinkley 4y ago
        “Such” a good deal is 2-30% in benchmarks (and they don’t say if that’s with cache leak protections turned on or off). In previous generations it was more like -20-20%. If one thread is having issues with L1 and L2 cache, splitting that evenly with a completely different workflow isn’t going to help. If cache contention weren’t a problem, and it was just a matter of jumping into previously unseen instructions and data (cold cache), you’d expect to see 50-300% numbers from hyperthreading, precisely because of how long the stalls are.
  - kevin_thibedeau 4y ago
    If one thread has to hit main memory but the other one can get what it needs from L1 cache they aren't competing for the same resources.
- MauranKilom 4y ago
  I recently checked whether an ugly hack I implemented years ago for 30% performance gains ("halve the default OpenMP thread count at start of computation and restore it afterwards" -> effectively, disable hyperthreading) was still necessary. It's now apparently a 3x performance gain... All because I'm saturating memory bandwidth. I don't know whether to be happy or sad...
  - eloff 4y ago
    Ouch, looks like they’re clobbering each others cache. Also, well done. It looks like you optimized the hell out of your code.
FpUser 4y ago
Many years ago a customer on a single core computer had problems of heavy video stutter in my product. It was multithreaded and ran as I recall 5 threads (GUI, video decoding, 3d pipeline, device control and computation). Others did not have this problem. After investigation it turned out that said customer had hyperthreading disabled in BIOS. Enabling it fixed the problem instantly. So "real" or not but from my experience HT does work to the benefit.
hinkley 4y ago
I don’t know how many hardware revisions Intel went through where every benchmark said hyperthreading was slower than turning it off, but it was a lot. It became Lucy’s football at some point. And in these days of post-Dennard scaling, you have thermal throttling, so idle cycles aren’t actually idle, they’re allowing the heat sink to catch up with heat production.
zamadatix 4y ago
I agree with this stance. It's the same as branch prediction, out of order execution, or anything else about computing efficiently.

bakugo 4y ago

> I can't think of a good principled reason to say SMT cores aren't "real". It really depends on your definition of "real", yes you can treat them like "real" independent cores but that's not ideal for performance because under the hood they're not actually independent. Your operating system is aware of this and will often avoid scheduling two tasks onto the same physical core unless it has to. If you have 20 logical cores (10 physical ones with hyperthreading) and 10 tasks to execute, the OS scheduler will usually allocate one task to each physical core and leave the other logical core idle.

Sohcahtoa82 4y ago
I did an experiment and effectively proved this to myself many years ago. I had just upgraded to an i7-3770K (8 threads, 4 cores) from a Core 2 Quad (4 threads, 4 cores). I did a POV-Ray render several times using 1, 2, 4, and 8 threads. 2 was nearly double the speed of 1, 4 was nearly double the speed of 2, but 8 was only about 15% faster than 4. To ensure I wasn't bottlenecking RAM at that level, I tried again with 2 threads, but forced both threads onto a single core, and it was only 15% faster than 1 thread. That was all the proof I needed of how you really can't treat two CPU threads as two cores. That said, I've never personally found an instance where allowing a processes to span all CPU threads actually reduced performance, and I'm not sure I've ever seen a real-world case where it does. It's usually something contrived.
- duffyjp 4y ago
  When I bought my current desktop the two CPUs I was looking at were the i7-9700K and i9-9900K. This was the generation where intel went from 6 cores with HT to 8 cores w/o HT for the i7. The i9 has 8 cores with HT. I liked the idea of the i7 because without HT you have a real picture of how utilized your CPU is. If it says 100% on a core it's 100%. I ended up going with the i9 though when a promotion was too good to pass up. When I got it I ran Cinebench both with and without HT enabled in the BIOS and it made a decent impact so I left it on.
- jeffbee 4y ago
  That's one use case. SMT is not designed to address it. POV-Ray will saturate your execution resources in the core and adding another frontend doesn't change the facts. SMT is designed to hide memory load latency. In this use case it is completely brilliant. You will get 2x speedups with hyperthreading when randomly accessing memory.

fmajid 4y ago

On AMD, yes, but Intel Hyperthreading is not real SMT.