5 ms·

Hopefully this behavior change will help improve AVX-512 uptake and end the somewhat ridiculous conception people have that the instructions are entirely useles

Hopefully this behavior change will help improve AVX-512 uptake and end the somewhat ridiculous conception people have that the instructions are entirely useless. Intel's HotChips presentation on Icelake-SP also indicated that the behavior will be significantly better on server chips, but the behavior is more instruction dependent with 3 license levels and only 512-bit instructions that utilize the FMA unit being subject to downclocking (by ~15-17% as opposed to ~27-29% on SKX derived chips). (Intel Hotchips slide: https://images.anandtech.com/doci/15984/202008171757161.jpg https://images.anandtech.com/doci/15984/202008171757161.jpg)

BeeOnRope 6y ago
Makes sense - the main difference between ICL and ICL-X would seem to be 2 FMA units.
Dylan16807 6y ago
It definitely helps. And it looks like they've reduced how often a single instruction will cause a lockup as the core shifts to a different power level. But until they've eliminated that issue, it's still scary to toss in a few AVX-512 instructions.
- FrankBooth 6y ago
  There is no "lockup". Quit spreading misinformation.
  - zlynx 6y ago
    It is definitely a problem of defective hardware. But there are plenty of people out there with both Intel and Ryzen systems who disable every C-State and every Turbo setting because any change in power levels causes their system to crash. Lockup, bluescreen, kernel panic or just computation errors. Some of that is from overclocking. Some is from old and/or defective power supplies. Some is from motherboard VRMs. And some, like the original Ryzen 1700X, is from bad SOC-internal power management. At any rate, I have read forum posts reporting system failures caused by AVX. Either overcurrent or clockrate changes crashing it. So it seems to have support. I wouldn't call that "misinformation."
    - theevilsharpie 6y ago
      > But there are plenty of people out there with both Intel and Ryzen systems who disable every C-State and every Turbo setting because any change in power levels causes their system to crash. There are plenty of people with both Intel and Ryzen systems that are straight-up broken and don't power on at all. Defects happen. Improperly designed systems happen. Misconfigurations happen. While those situations are unfortunate for the small percentage of people experiencing them, they shouldn't be used to judged the platform's capabilities as a whole. > So it seems to have support. I wouldn't call that "misinformation." Using anonymous, largely unverifiable anecdotes posted on web forums as evidence for a population-wide problem is a textbook case of selection bias. And if you're OK with that, let me throw in my own anecdote. All of my systems, which include: * Skylake, Coffee Lake, Haswell/Broadwell, Sandy Bridge/Ivy Bridge, and Westmere/Nehalem Intel processors, * Zen 2, K10, and K8 AMD processors, ...have been able to reliably execute supported vector instructions (SSE, AVX, etc.) for extended periods of time without any problems.
  - Dylan16807 6y ago
    https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html > There is a transition period (the rightmost of the two shaded regions, in orange14) of ~11 μs15 where the CPU is halted: no samples occur during this period16. For fun, I’ll call this a frequency transition. It stops executing for 35 thousand cycles. I call that a "lockup" "as it shifts".
    - tarlinian 6y ago
      This isn't the real problem though...the problem with the existing AVX-512 implementations is the "relaxation time" causes subsequent scalar code to be slow. From later in the same post: > Here, we have the worst case scenario of transitions packed as closely as possible, but we lose only ~20 μs (for 2 transitions) out of 760 μs, less than a 3% impact. The impact of running at the lower frequency is much higher: 2.8 vs 3.2 GHz: a 12.5% impact in the case that the lowered frequency was not useful (i.e., because the wide SIMD payload represents a vanishingly small part of the total work). Interestingly enough, this is another feature that is supposed to have been improved on server Icelake. The frequency transition halt time is now pretty much negligible. The "core frequency transition block time" goes from ~12 us on CLX (similar to the number quoted above) to ~0 us on ICX. (Slide with frequency transition info: https://images.anandtech.com/doci/15984/202008171754441.jpg https://images.anandtech.com/doci/15984/202008171754441.jpg)
      - Dylan16807 6y ago
        Either way, it's a big problem that certain single instructions can cause this transition. When the transition is based on a usage threshold of heavy instructions, it's not so bad. And with this revision, more of the transitions are based on threshold. But there are still some instructions that cause an immediate frequency change, if I'm reading the articles right.
        tarlinian 6y ago
        No...the whole point is that the single instruction induced halt for downclocking isn't a real issue. Even in the pathological case where you insert a single instruction spaced 760 us apart in order to induce the maximum number of clock shifts, the total performance degradation due to the clock halts is only 3% (the frequency drop that is induced by the use of these instructions has a much larger impact.) Furthermore, on Icelake-SP, the halted time due to frequency transitions is supposed to go to 0, which makes this aspect of the problem entirely irrelevant. Yes, if you insert a single 512-bit FMA that runs every so often in your code you will get a 15% performance hit from the lower frequency, but that's much less likely than the old case where people who were trying to use AVX-512 for memcpy and the like would slow down scalar code.
The_rationalist 6y ago
I generally agree but https://www.phoronix.com/scan.php?page=news_item&px=LLVM-Clang-10-AVX512-Change https://www.phoronix.com/scan.php?page=news_item&px=LLVM-Cla...
- gameswithgo 6y ago
  compilers are not at a point where they do a great job of leveraging SIMD in general, and definitely not where they leverage AVX-512, but hand written intrinsics with AVX-512 can attain amazing performance.
CoolGuySteve 6y ago
I don't think the book is closed. Thermal and TDP downclocking are still present. It would have been nice to see the vcore and thermal values graphed as part of the benchmark. Do they increase faster for AVX-512 vs the other instruction sets? I've had problems in the past with Sandybridge, where AVX hit thermal throttles before SSE. I ended up having to disable them in my build because of it. Presumably, the same behaviour would be seen here now that the vector unit is wider and there are more densely packed transistors flipping.
- wmf 6y ago
  Doing work faster is almost always going to consume more power and if you're already at the power or temperature limit (which is how most CPUs/GPUs operate now) then the frequency will have to reduce. This isn't automatically bad; ultimately what matters is performance. Did you really see lower performance with AVX than with SSE?
  - CoolGuySteve 6y ago
    The problem is that the downclocking affects other cores. So a performance improvement in this one task can hurt performance on other threads, which is what happened to me.
    - wmf 6y ago
      Yeah, I think Intel is trying to fix this with Speed Select but it's complex enough that no one will probably use it.
    - gameswithgo 6y ago
      it does bot affect other cores on all cpus, most newer ones it doesn’t
    - BeeOnRope 6y ago
      At least on most new cores, the frequency is per-core. This isn't true on, for example, some Skylake client cores - but these don't have much SIMD related downclocking either.
ksec 6y ago
> end the somewhat ridiculous conception people have that the instructions are entirely useless. I don't think anyone ever said that. What people are really saying, AVX-512; in its current form, limited by Clockspeed, Thermals, and most importantly market segmentation. And given all these three things are only slightly improved in the current 3 years roadmap render the instruction useless. ( But latest HotChip information seems to suggest they are well aware of it. So Roadmap can change ) And even more importantly that is in reference to current stack of Intel issues.
- quietbritishjim 6y ago
  I think some subtlety has been lost on me. The parent comment said that people had the "ridiculous conception ... that the instructions are entirely useless". Then you said that no one thinks that, followed by some stuff I'll admit I didn't really understand, leading to the conclusion "... render the instruction useless". Are you disagreeing with the parent comment or not? Are you maybe saying that the instruction is useless in chips that are in computers right now but will be useful in chips that will be released in future?
- paulmd 6y ago
  > I don't think anyone ever said that. Yeah, Torvalds said pretty much exactly that, in his typically boisterous form. That it is "only useful in special cases", "only exists to make Intel look better on benchmarks", and that "the transistors would be better spent elsewhere". https://www.phoronix.com/scan.php?page=news_item&px=Linus-Torvalds-On-AVX-512 https://www.phoronix.com/scan.php?page=news_item&px=Linus-To...
Const-me 6y ago
> the instructions are entirely useless I mostly agree, but for me the frequency scaling is almost irrelevant. Here’s a link, expand “Other settings” and you’ll see: https://store.steampowered.com/hwsurvey https://store.steampowered.com/hwsurvey If you implement AVX2, the code will run on 76% of PCs. When you want good performance, such optimization is a good use of limited resources. AVX512 is only at 0.42%. That’s why it’s useless to implement AVX512 except for servers where you have full control over hardware. And with current state of Intel, not necessarily a good idea even for servers, it’s quite possible in a couple of years you’ll want to switch to AMD.
- killerstorm 6y ago
  Eh? Compilers can generate two versions of code -- AVX and non-AVX - and dynamically choose depending on CPU capabilities. It was this way for ages.
  - Const-me 6y ago
    > Compilers can generate two versions of code From what? Automatic vectorization is very limited, it’s only good for pure vertical algorithms. > and dynamically choose depending on CPU capabilities Only Intel compiler does that, the rest of them don’t. Sometimes I ship multiple versions of binaries. Other times I do runtime dispatch myself, it’s only couple lines of code, __cpuid() then cache some function pointers, or abstract class pointers. It’s even possible to compile these implementations from the same C++ source, using templates, macros, and/or something else.

BeeOnRope 6y ago

Makes sense - the main difference between ICL and ICL-X would seem to be 2 FMA units.

Dylan16807 6y ago

It definitely helps. And it looks like they've reduced how often a single instruction will cause a lockup as the core shifts to a different power level. But until they've eliminated that issue, it's still scary to toss in a few AVX-512 instructions.

FrankBooth 6y ago
There is no "lockup". Quit spreading misinformation.
- zlynx 6y ago
  It is definitely a problem of defective hardware. But there are plenty of people out there with both Intel and Ryzen systems who disable every C-State and every Turbo setting because any change in power levels causes their system to crash. Lockup, bluescreen, kernel panic or just computation errors. Some of that is from overclocking. Some is from old and/or defective power supplies. Some is from motherboard VRMs. And some, like the original Ryzen 1700X, is from bad SOC-internal power management. At any rate, I have read forum posts reporting system failures caused by AVX. Either overcurrent or clockrate changes crashing it. So it seems to have support. I wouldn't call that "misinformation."
  - theevilsharpie 6y ago
    > But there are plenty of people out there with both Intel and Ryzen systems who disable every C-State and every Turbo setting because any change in power levels causes their system to crash. There are plenty of people with both Intel and Ryzen systems that are straight-up broken and don't power on at all. Defects happen. Improperly designed systems happen. Misconfigurations happen. While those situations are unfortunate for the small percentage of people experiencing them, they shouldn't be used to judged the platform's capabilities as a whole. > So it seems to have support. I wouldn't call that "misinformation." Using anonymous, largely unverifiable anecdotes posted on web forums as evidence for a population-wide problem is a textbook case of selection bias. And if you're OK with that, let me throw in my own anecdote. All of my systems, which include: * Skylake, Coffee Lake, Haswell/Broadwell, Sandy Bridge/Ivy Bridge, and Westmere/Nehalem Intel processors, * Zen 2, K10, and K8 AMD processors, ...have been able to reliably execute supported vector instructions (SSE, AVX, etc.) for extended periods of time without any problems.
- Dylan16807 6y ago
  https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html > There is a transition period (the rightmost of the two shaded regions, in orange14) of ~11 μs15 where the CPU is halted: no samples occur during this period16. For fun, I’ll call this a frequency transition. It stops executing for 35 thousand cycles. I call that a "lockup" "as it shifts".
  - tarlinian 6y ago
    This isn't the real problem though...the problem with the existing AVX-512 implementations is the "relaxation time" causes subsequent scalar code to be slow. From later in the same post: > Here, we have the worst case scenario of transitions packed as closely as possible, but we lose only ~20 μs (for 2 transitions) out of 760 μs, less than a 3% impact. The impact of running at the lower frequency is much higher: 2.8 vs 3.2 GHz: a 12.5% impact in the case that the lowered frequency was not useful (i.e., because the wide SIMD payload represents a vanishingly small part of the total work). Interestingly enough, this is another feature that is supposed to have been improved on server Icelake. The frequency transition halt time is now pretty much negligible. The "core frequency transition block time" goes from ~12 us on CLX (similar to the number quoted above) to ~0 us on ICX. (Slide with frequency transition info: https://images.anandtech.com/doci/15984/202008171754441.jpg https://images.anandtech.com/doci/15984/202008171754441.jpg)
    - Dylan16807 6y ago
      Either way, it's a big problem that certain single instructions can cause this transition. When the transition is based on a usage threshold of heavy instructions, it's not so bad. And with this revision, more of the transitions are based on threshold. But there are still some instructions that cause an immediate frequency change, if I'm reading the articles right.
      - tarlinian 6y ago
        No...the whole point is that the single instruction induced halt for downclocking isn't a real issue. Even in the pathological case where you insert a single instruction spaced 760 us apart in order to induce the maximum number of clock shifts, the total performance degradation due to the clock halts is only 3% (the frequency drop that is induced by the use of these instructions has a much larger impact.) Furthermore, on Icelake-SP, the halted time due to frequency transitions is supposed to go to 0, which makes this aspect of the problem entirely irrelevant. Yes, if you insert a single 512-bit FMA that runs every so often in your code you will get a 15% performance hit from the lower frequency, but that's much less likely than the old case where people who were trying to use AVX-512 for memcpy and the like would slow down scalar code.

The_rationalist 6y ago

I generally agree but https://www.phoronix.com/scan.php?page=news_item&px=LLVM-Clang-10-AVX512-Change https://www.phoronix.com/scan.php?page=news_item&px=LLVM-Cla...

gameswithgo 6y ago
compilers are not at a point where they do a great job of leveraging SIMD in general, and definitely not where they leverage AVX-512, but hand written intrinsics with AVX-512 can attain amazing performance.

CoolGuySteve 6y ago

I don't think the book is closed. Thermal and TDP downclocking are still present. It would have been nice to see the vcore and thermal values graphed as part of the benchmark. Do they increase faster for AVX-512 vs the other instruction sets? I've had problems in the past with Sandybridge, where AVX hit thermal throttles before SSE. I ended up having to disable them in my build because of it. Presumably, the same behaviour would be seen here now that the vector unit is wider and there are more densely packed transistors flipping.

wmf 6y ago
Doing work faster is almost always going to consume more power and if you're already at the power or temperature limit (which is how most CPUs/GPUs operate now) then the frequency will have to reduce. This isn't automatically bad; ultimately what matters is performance. Did you really see lower performance with AVX than with SSE?
- CoolGuySteve 6y ago
  The problem is that the downclocking affects other cores. So a performance improvement in this one task can hurt performance on other threads, which is what happened to me.
  - wmf 6y ago
    Yeah, I think Intel is trying to fix this with Speed Select but it's complex enough that no one will probably use it.
  - gameswithgo 6y ago
    it does bot affect other cores on all cpus, most newer ones it doesn’t
  - BeeOnRope 6y ago
    At least on most new cores, the frequency is per-core. This isn't true on, for example, some Skylake client cores - but these don't have much SIMD related downclocking either.

ksec 6y ago

> end the somewhat ridiculous conception people have that the instructions are entirely useless. I don't think anyone ever said that. What people are really saying, AVX-512; in its current form, limited by Clockspeed, Thermals, and most importantly market segmentation. And given all these three things are only slightly improved in the current 3 years roadmap render the instruction useless. ( But latest HotChip information seems to suggest they are well aware of it. So Roadmap can change ) And even more importantly that is in reference to current stack of Intel issues.

quietbritishjim 6y ago
I think some subtlety has been lost on me. The parent comment said that people had the "ridiculous conception ... that the instructions are entirely useless". Then you said that no one thinks that, followed by some stuff I'll admit I didn't really understand, leading to the conclusion "... render the instruction useless". Are you disagreeing with the parent comment or not? Are you maybe saying that the instruction is useless in chips that are in computers right now but will be useful in chips that will be released in future?
paulmd 6y ago
> I don't think anyone ever said that. Yeah, Torvalds said pretty much exactly that, in his typically boisterous form. That it is "only useful in special cases", "only exists to make Intel look better on benchmarks", and that "the transistors would be better spent elsewhere". https://www.phoronix.com/scan.php?page=news_item&px=Linus-Torvalds-On-AVX-512 https://www.phoronix.com/scan.php?page=news_item&px=Linus-To...

Const-me 6y ago

> the instructions are entirely useless I mostly agree, but for me the frequency scaling is almost irrelevant. Here’s a link, expand “Other settings” and you’ll see: https://store.steampowered.com/hwsurvey https://store.steampowered.com/hwsurvey If you implement AVX2, the code will run on 76% of PCs. When you want good performance, such optimization is a good use of limited resources. AVX512 is only at 0.42%. That’s why it’s useless to implement AVX512 except for servers where you have full control over hardware. And with current state of Intel, not necessarily a good idea even for servers, it’s quite possible in a couple of years you’ll want to switch to AMD.

killerstorm 6y ago
Eh? Compilers can generate two versions of code -- AVX and non-AVX - and dynamically choose depending on CPU capabilities. It was this way for ages.
- Const-me 6y ago
  > Compilers can generate two versions of code From what? Automatic vectorization is very limited, it’s only good for pure vertical algorithms. > and dynamically choose depending on CPU capabilities Only Intel compiler does that, the rest of them don’t. Sometimes I ship multiple versions of binaries. Other times I do runtime dispatch myself, it’s only couple lines of code, __cpuid() then cache some function pointers, or abstract class pointers. It’s even possible to compile these implementations from the same C++ source, using templates, macros, and/or something else.