5 ms·

do you mean the L2 or L3 cache? Hyperthreading is there to help reduce the times the CPU is stalled. If you run less active threads, the cache efficiency woul

by xxs 8y ago

do you mean the L2 or L3 cache? Hyperthreading is there to help reduce the times the CPU is stalled. If you run less active threads, the cache efficiency would the same in that case, hypethreading or not.

rbanffy 8y ago
L1 and L2 (IIRC) are usually tied to a single physical core and, also IIRC, are partitioned between logical cores when HT is enabled. With HT disabled, cache per logical core doubles (because there is only one) and cache misses go down. Many HPC installs run with HT disabled because of that (YMMV depending on specifics of your workload - if your hot data fits in the cache, you'll be happy). Now that we mention it, I wonder if HT can be enabled or disabled per core. If that's possible, the machine would look like it has some logical cores with more cache than others and we could use cache miss counters as input to decide where a process should run.
- xxs 8y ago
  >With HT disabled, cache per logical core doubles (because there is only one) and cache misses go down. This is too naive to look at in such a way. Yes if there is less scheduling on the core (less concurrency) the likehood of misses is lower, but the idea is to pay less for each miss doing effectively something else meanwhile instead of being stalled for memory. >Now that we mention it, I wonder if HT can be enabled or disabled per core. OS scheduler can do that. Normally the guidance for hyperthreading is: consider each "thread" as a separate core. If an OS chooses to use one core at a time and not schedule anything, the hypertrheadind won't be used. That is imagine logical core0&core 1 represent the same physical one, if the OS schedules only to core0, hyperthreading is not in play. Notes: controlling scheduling on linux is done via taskset. And cores info is available via "cat /proc/cpuinfo"
  - rbanffy 8y ago
    > OS scheduler can do that True, but the scheduler can't tell the CPU to allocate all of the partitioned cache to a thread if it the silicon thinks it belongs to another logical core, so you end with effectively half of your cache dark. If L1 and L2 are pooled between the logical cores, it makes no difference, but I'm not sure how HT behaves WRT to partitioning caches on current generations of Intel parts and I remember it used to be like that (because I turned off HT on a couple dozen boxes to improve single thread performance, which was what was bothering me).
    - xxs 8y ago
      L1&L2 are exclusive to each designated core. L3 is shared (hence my very 1st question); also updated the info how to force scheduling on your own.
      - rbanffy 8y ago
        Got back from Intel's website. It seems the cache was partitioned on early HT implementations, but now L1 and L2 are shared between the logical cores and, therefore, it should make no difference to disable HT.
  - pjmlp 8y ago
    Back when the first CPUs with HT came into the market I got a developer workstation with them. After trying out WebSphere developer tools (RAD) with HT enabled and disabled for several days, I eventually came to the conclusion that with them disabled everything just felt faster. Also it is not clear if the security issues caused by HT, which affect other vendors as well, could ever be properly sorted out.
    - zrm 8y ago
      The first HT processors had smaller caches and that was before operating systems had SMT-aware schedulers. The place where SMT really seems to benefit is in particular applications that can use arbitrarily many threads. This also points to a solution to the security issue, because you get most of the benefit even if you only schedule threads from the same process group together on the same core and it would be possible for programs to specify whether it's safe to do that (e.g. yes for a render, no for a web browser). That would also improve SMT performance in general since SMT makes the performance of some programs worse, and they could use the same mechanism to opt out of it.
      - pjmlp 8y ago
        Yes I do agree the first HT generations were quite lame. Regarding threads, I used to be very keen on them when they became available everywhere, yet nowadays I see them as yet another attack vector. It is not only the security issues, also the overall application stability, as a buggy thread can bring the whole application down.
- marmaduke 8y ago
  > Many HPC installs run with HT disabled the ones I use (and admin) don't. Slurm for example handles threads vs cores explicitly, and smart defaults can be set.
  - mroche 8y ago
    Do you see any performance differences when enabling/disabling? Our cluster uses Sandy Bridge through Skylake chips and all of our nodes have HT disabled. When things were tested (before I got here) there apparently was a non-trivial difference for the applications running. Curious what your results are.
    - Iwan-Zotow 8y ago
      We're running mostly Monte Carlo workload. For MC it is a clear win, 20 plus pct bump. On LA, I would guess otherwise
    - marmaduke 8y ago
      The only significant issues I’ve seen is that many libraries using threading are or were a little naive in that they’d take the core count to be thread count. Better libraries know the difference But I’m not aware of any benchmarks that pin one execution thread per physical core and compare with and without hyperthreading. I guess I have some homework to do, but then it’s a lot of effort when the results could depend on generation, clock speed etc.
- rayiner 8y ago
  The caches aren’t “partitioned” between logical cores. You have two cores generating memory requests, and the combined working set could be larger than the caches, but there is no partitioning going on, in the sense of allocating cache lines to one thread or the other.
- dragontamer 8y ago
  > also IIRC, are partitioned between logical cores when HT is enabled. A lot of things are partitioned. But my understanding is that L1 and L2 cache is competitively shared. Which means if one thread uses 512 bytes of cache, the OTHER thread can use the full 32kB for itself. There are other benefits: Both hyperthreads can share L1-code cache for instance, if there's a tight loop that is shared between the two threads. Lower-level things like the retirement engine (aka: shadow registers / out-of-order registers) are partitioned. I dunno whether or not the branch predictor is shared or partitioned. But... things like that.

by xxs 8y ago

rbanffy 8y ago

L1 and L2 (IIRC) are usually tied to a single physical core and, also IIRC, are partitioned between logical cores when HT is enabled. With HT disabled, cache per logical core doubles (because there is only one) and cache misses go down. Many HPC installs run with HT disabled because of that (YMMV depending on specifics of your workload - if your hot data fits in the cache, you'll be happy). Now that we mention it, I wonder if HT can be enabled or disabled per core. If that's possible, the machine would look like it has some logical cores with more cache than others and we could use cache miss counters as input to decide where a process should run.

xxs 8y ago
>With HT disabled, cache per logical core doubles (because there is only one) and cache misses go down. This is too naive to look at in such a way. Yes if there is less scheduling on the core (less concurrency) the likehood of misses is lower, but the idea is to pay less for each miss doing effectively something else meanwhile instead of being stalled for memory. >Now that we mention it, I wonder if HT can be enabled or disabled per core. OS scheduler can do that. Normally the guidance for hyperthreading is: consider each "thread" as a separate core. If an OS chooses to use one core at a time and not schedule anything, the hypertrheadind won't be used. That is imagine logical core0&core 1 represent the same physical one, if the OS schedules only to core0, hyperthreading is not in play. Notes: controlling scheduling on linux is done via taskset. And cores info is available via "cat /proc/cpuinfo"
- rbanffy 8y ago
  > OS scheduler can do that True, but the scheduler can't tell the CPU to allocate all of the partitioned cache to a thread if it the silicon thinks it belongs to another logical core, so you end with effectively half of your cache dark. If L1 and L2 are pooled between the logical cores, it makes no difference, but I'm not sure how HT behaves WRT to partitioning caches on current generations of Intel parts and I remember it used to be like that (because I turned off HT on a couple dozen boxes to improve single thread performance, which was what was bothering me).
  - xxs 8y ago
    L1&L2 are exclusive to each designated core. L3 is shared (hence my very 1st question); also updated the info how to force scheduling on your own.
    - rbanffy 8y ago
      Got back from Intel's website. It seems the cache was partitioned on early HT implementations, but now L1 and L2 are shared between the logical cores and, therefore, it should make no difference to disable HT.
- pjmlp 8y ago
  Back when the first CPUs with HT came into the market I got a developer workstation with them. After trying out WebSphere developer tools (RAD) with HT enabled and disabled for several days, I eventually came to the conclusion that with them disabled everything just felt faster. Also it is not clear if the security issues caused by HT, which affect other vendors as well, could ever be properly sorted out.
  - zrm 8y ago
    The first HT processors had smaller caches and that was before operating systems had SMT-aware schedulers. The place where SMT really seems to benefit is in particular applications that can use arbitrarily many threads. This also points to a solution to the security issue, because you get most of the benefit even if you only schedule threads from the same process group together on the same core and it would be possible for programs to specify whether it's safe to do that (e.g. yes for a render, no for a web browser). That would also improve SMT performance in general since SMT makes the performance of some programs worse, and they could use the same mechanism to opt out of it.
    - pjmlp 8y ago
      Yes I do agree the first HT generations were quite lame. Regarding threads, I used to be very keen on them when they became available everywhere, yet nowadays I see them as yet another attack vector. It is not only the security issues, also the overall application stability, as a buggy thread can bring the whole application down.
marmaduke 8y ago
> Many HPC installs run with HT disabled the ones I use (and admin) don't. Slurm for example handles threads vs cores explicitly, and smart defaults can be set.
- mroche 8y ago
  Do you see any performance differences when enabling/disabling? Our cluster uses Sandy Bridge through Skylake chips and all of our nodes have HT disabled. When things were tested (before I got here) there apparently was a non-trivial difference for the applications running. Curious what your results are.
  - Iwan-Zotow 8y ago
    We're running mostly Monte Carlo workload. For MC it is a clear win, 20 plus pct bump. On LA, I would guess otherwise
  - marmaduke 8y ago
    The only significant issues I’ve seen is that many libraries using threading are or were a little naive in that they’d take the core count to be thread count. Better libraries know the difference But I’m not aware of any benchmarks that pin one execution thread per physical core and compare with and without hyperthreading. I guess I have some homework to do, but then it’s a lot of effort when the results could depend on generation, clock speed etc.
rayiner 8y ago
The caches aren’t “partitioned” between logical cores. You have two cores generating memory requests, and the combined working set could be larger than the caches, but there is no partitioning going on, in the sense of allocating cache lines to one thread or the other.
dragontamer 8y ago
> also IIRC, are partitioned between logical cores when HT is enabled. A lot of things are partitioned. But my understanding is that L1 and L2 cache is competitively shared. Which means if one thread uses 512 bytes of cache, the OTHER thread can use the full 32kB for itself. There are other benefits: Both hyperthreads can share L1-code cache for instance, if there's a tight loop that is shared between the two threads. Lower-level things like the retirement engine (aka: shadow registers / out-of-order registers) are partitioned. I dunno whether or not the branch predictor is shared or partitioned. But... things like that.