3 ms·

I don't think the book is closed. Thermal and TDP downclocking are still present. It would have been nice to see the vcore and thermal values graphed as part

I don't think the book is closed. Thermal and TDP downclocking are still present. It would have been nice to see the vcore and thermal values graphed as part of the benchmark. Do they increase faster for AVX-512 vs the other instruction sets? I've had problems in the past with Sandybridge, where AVX hit thermal throttles before SSE. I ended up having to disable them in my build because of it. Presumably, the same behaviour would be seen here now that the vector unit is wider and there are more densely packed transistors flipping.

wmf 6y ago
Doing work faster is almost always going to consume more power and if you're already at the power or temperature limit (which is how most CPUs/GPUs operate now) then the frequency will have to reduce. This isn't automatically bad; ultimately what matters is performance. Did you really see lower performance with AVX than with SSE?
- CoolGuySteve 6y ago
  The problem is that the downclocking affects other cores. So a performance improvement in this one task can hurt performance on other threads, which is what happened to me.
  - wmf 6y ago
    Yeah, I think Intel is trying to fix this with Speed Select but it's complex enough that no one will probably use it.
  - gameswithgo 6y ago
    it does bot affect other cores on all cpus, most newer ones it doesn’t
  - BeeOnRope 6y ago
    At least on most new cores, the frequency is per-core. This isn't true on, for example, some Skylake client cores - but these don't have much SIMD related downclocking either.
    - makomk 6y ago
      The license-based downclocking is per-core on most new chips, but according to the linked blog post that doesn't really exist on Ice Lake anyway. Pretty sure thermal and TDP-based downclocking are across the entire package (all cores and the GPU) which in some sense might make them more objectionable than license-based downclocking.
- donor20 6y ago
  Isn't the problem if you have just a small % of your workload or some random binaries on your system doing a view 512 instructions they bonk the rest of your performance? If I had a little background deamon that used 512 because they were cool or in the hotchips presentation, and that bonked my overall system performance that would be annoying. It's also annoying because intel benchmarks with no mitigations. So what can happen is you think you should be seeing X performance, and then with mitigations applied and some 512 instructions hitting you are Y performance.
  - BeeOnRope 6y ago
    Yes, that has been the problem – but the TLDR from this post is that it is no longer a meaningful problem at least on the Ice Lake client CPUs: since AVX-512 no longer causes almost any downclocking.
BeeOnRope 6y ago
The book is definitely not closed, but the other limits are somehow less problematic than the license-based downclocking. You you use more power (and get hotter temps: these are exactly proportional, so you can mostly just talk about them as one) with wider vectors because you are doing more work. When you look at it on a per-element basis, you use less power per element with wider vectors. E.g., you might use 1 pJ per element for 256-bit FMA but only 0.8 for 512-bit FMA. Of course, since you can do 2x as many total elements in 512-bits on a 2 FMA machine, you can be both more efficient but use more total power, so you can get TDP or thermal limits with 512-bit code that you wouldn't on 256-bit, but it should still per faster and more efficient per element. All of this assumes you can usefully use the 2x more work with the larger vectors. Sometimes the scaling is worse: e.g., for lots of short arrays, when a lookup table is involved, when additional shuffling or transposition is required with larger vectors, etc. In that case you could end up less efficient with larger vectors.
tarlinian 6y ago
To reiterate, the problem with the existing license based downclocking is that a few AVX-512 operations can drop your frequency for subsequent scalar loads on the same core so you need to carefully analyze the overall application to make sure that you have enough AVX-512 work over which you can amortize the loss in performance in the rest of your code that is affected by the frequency drop. If the only issue with AVX-512 is thermal downclocking because you end up using more power, it's almost definitely because you are getting more work done per time. A few AVX-512 instructions in a mostly scalar workload is not going to significantly increase power dissipation and therefore should not induce thermal downclocking, while a heavily utilized AVX-512 kernel will burn power, but should also be doing work twice as fast per instruction.

wmf 6y ago

Doing work faster is almost always going to consume more power and if you're already at the power or temperature limit (which is how most CPUs/GPUs operate now) then the frequency will have to reduce. This isn't automatically bad; ultimately what matters is performance. Did you really see lower performance with AVX than with SSE?

CoolGuySteve 6y ago
The problem is that the downclocking affects other cores. So a performance improvement in this one task can hurt performance on other threads, which is what happened to me.
- wmf 6y ago
  Yeah, I think Intel is trying to fix this with Speed Select but it's complex enough that no one will probably use it.
- gameswithgo 6y ago
  it does bot affect other cores on all cpus, most newer ones it doesn’t
- BeeOnRope 6y ago
  At least on most new cores, the frequency is per-core. This isn't true on, for example, some Skylake client cores - but these don't have much SIMD related downclocking either.
  - makomk 6y ago
    The license-based downclocking is per-core on most new chips, but according to the linked blog post that doesn't really exist on Ice Lake anyway. Pretty sure thermal and TDP-based downclocking are across the entire package (all cores and the GPU) which in some sense might make them more objectionable than license-based downclocking.
donor20 6y ago
Isn't the problem if you have just a small % of your workload or some random binaries on your system doing a view 512 instructions they bonk the rest of your performance? If I had a little background deamon that used 512 because they were cool or in the hotchips presentation, and that bonked my overall system performance that would be annoying. It's also annoying because intel benchmarks with no mitigations. So what can happen is you think you should be seeing X performance, and then with mitigations applied and some 512 instructions hitting you are Y performance.
- BeeOnRope 6y ago
  Yes, that has been the problem – but the TLDR from this post is that it is no longer a meaningful problem at least on the Ice Lake client CPUs: since AVX-512 no longer causes almost any downclocking.

BeeOnRope 6y ago

The book is definitely not closed, but the other limits are somehow less problematic than the license-based downclocking. You you use more power (and get hotter temps: these are exactly proportional, so you can mostly just talk about them as one) with wider vectors because you are doing more work. When you look at it on a per-element basis, you use less power per element with wider vectors. E.g., you might use 1 pJ per element for 256-bit FMA but only 0.8 for 512-bit FMA. Of course, since you can do 2x as many total elements in 512-bits on a 2 FMA machine, you can be both more efficient but use more total power, so you can get TDP or thermal limits with 512-bit code that you wouldn't on 256-bit, but it should still per faster and more efficient per element. All of this assumes you can usefully use the 2x more work with the larger vectors. Sometimes the scaling is worse: e.g., for lots of short arrays, when a lookup table is involved, when additional shuffling or transposition is required with larger vectors, etc. In that case you could end up less efficient with larger vectors.

tarlinian 6y ago

To reiterate, the problem with the existing license based downclocking is that a few AVX-512 operations can drop your frequency for subsequent scalar loads on the same core so you need to carefully analyze the overall application to make sure that you have enough AVX-512 work over which you can amortize the loss in performance in the rest of your code that is affected by the frequency drop. If the only issue with AVX-512 is thermal downclocking because you end up using more power, it's almost definitely because you are getting more work done per time. A few AVX-512 instructions in a mostly scalar workload is not going to significantly increase power dissipation and therefore should not induce thermal downclocking, while a heavily utilized AVX-512 kernel will burn power, but should also be doing work twice as fast per instruction.