4 ms·

> The AVX-512 instructions not only weren’t enabled by default, they couldn’t be enabled at all unless you went out of your way to disable the efficiency cores

by hajile 5y ago

> The AVX-512 instructions not only weren’t enabled by default, they couldn’t be enabled at all unless you went out of your way to disable the efficiency cores completely. They also wouldn’t benefit your workload unless disabling those extra cores was offset by the AVX-512 instructions on the remaining cores. If your code benefits from AVX-512, it'll probably benefit from turning off the efficiency cores too. Sixteen 256-bit AVX channels are the same as eight 512-bit channels in theoretical throughput. Because there are fewer load/store commands and fewer bunches of setup code to run, overall theoretical efficiency should be higher. Power efficiency was a killer at 14nm. The cores would downclock into oblivion when AVX-512 executed. Given the node shrink and shutting off half the cores, I don't see why this would happen here. Doing the same calculations with less power means a lot for the kinds of workloads that actually use AVX-512. Sure, idle performance may go down a bit, but once again, if you're running the kind of application that benefits from this, that's also probably not a top consideration either. The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread.

PragmaticPulp 5y ago
> If your code benefits from AVX-512, it'll probably benefit from turning off the efficiency cores too. Maybe if you have a poorly parallelized problem where the bulk of the code is spent in AVX-512 instructions and you don’t ever do anything else on the computer, but that’s going to be rare for any consumer. > The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread. “It wouldn’t be hard” is a huge understatement. This is OS-level support, not something Intel can just do inside the CPU. And operating system vendors and developers are still catching up to basic Alder Like scheduler support without AVX-512 handing, so it’s not reasonable to suggest this would be easy.
- pantalaimon 5y ago
  There have been reports that it was possible to enable AVX512 on the big cores with the little ones enabled [0] by toggling some bits in the MSR. For a first step it would have been enough to manually pin threads to the big cores for adventurous users, but Linux already tacks which threads use AVX (since it comes with a clock penalty, so you want to isolate them if possible), it's not unreasonable to think that auto-setting CPU mask based on that should be possible too. [0] https://lkml.org/lkml/2021/12/2/1059 https://lkml.org/lkml/2021/12/2/1059
- gnufx 5y ago
  I don't see how "poorly parallelized" is relevant compared with, say, a decent OpenMP BLAS which is likely to benefit from any reduced power consumption that allows a faster clock. In general on compute nodes it's worth turning off cores dynamically which aren't being used by jobs, like on a half-full node.
amluto 5y ago
> The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread. Having actually worked on this, it’s quite a bit more complicated, although it is possible. For better or for worse, though, Intel made a decision that all cores would expose the same feature set, and now this is baked into Linux and probably other OSes at the ABI level, and changing it would be a mess. A correct heterogeneous ABI (and frankly a good AVX512 ABI at all) would require explicit opt-in, per process, for AVX512. Opting in would change the signal frame format, the available features, and the set of allowed cores.
- tmccrary55 5y ago
  Boost your benchmarks with AVX-512 by subscribing to Intel+
  - amluto 5y ago
    What benchmarks? Not a lot of real programs actually use AVX512, and a lot of the ones that did discovered that it made performance worse, so they stopped. (The issue is that AVX512 (the actual 512-bit parts, not the associated EVEX and masking extensions) may well be excellent for long-running vector-math-heavy usage, but the cost of switching AVX512 on and off is extreme, and using it for things like memcpy() and strcmp() is pretty much always a loss except in silly microbenchmarks.) To be clear, I don't like the type of product line differentiation that Intel does, and I think Intel should have supported proper heterogenous ISA systems so that AVX512 and related technologies on client systems would make sense, but I don't think any of this is nefarious.
    - hajile 5y ago
      > What benchmarks? Not a lot of real programs actually use AVX512, and a lot of the ones that did discovered that it made performance worse, so they stopped. This paints a picture of what happened to Intel's fab process rather than AVX-512 itself. AVX-512 was proposed back in 2013. It was NOT designed for desktops. The original designs were for their Phi chips (basically turning a bunch of x86 cores into a GPU). These Phi chips ran between 1 and 1.5GHz, so power consumption and clocks were always matched up. Intel wanted to move these instructions into their HPC CPUs. The problem at hand was ultra-high turbo speeds. These speeds work because the heat is a bit spread out on the chip and a lot of pieces are disabled at any given time. With AVX-512, they had 30% of the core going wide open for the vector units (not to mention the added usage from saturating the load/store bandwidth). They wanted 10nm and then 7nm to fix these issues. At their predicted schedule, 10nm would have launched in 2015 and 7nm in 2017. Given the introduction of AVX-512 in 2013, they had plenty of time. In fact, the first official product was Knight's landing in 2016. Skylake with AVX-512 didn't launch until 2017 when they were supposed to be on 7nm. Intel was forced to backport their designs to 14nm++++++++++++ which forced a deal with the devil. They had to downclock the CPU to keep within thermal limits, but this slowed down EVERYTHING. Maybe they could have created a separate power plane for AVX, but this would be a BIG change (and probably politically infeasible). What happens with the downclocking? If you run dedicated AVX-512 loads, then maybe you should have been looking at Phi instead. If not, mixed loads suffered overall because of the lower clockspeeds. Their second revision of 10nm superfin is still a generation larger (well, probably a half-generation) than what they anticipated. There might still be downclocking, but I'd guess that it's nowhere near what previous iterations required. TL;DR -- AVX-512 was launched two nodes too early which screwed over performance. It should become acceptable either with 10nm Superfin or 7nm when it launches so the CPU doesn't have to downclock constantly.

by hajile 5y ago

PragmaticPulp 5y ago

> If your code benefits from AVX-512, it'll probably benefit from turning off the efficiency cores too. Maybe if you have a poorly parallelized problem where the bulk of the code is spent in AVX-512 instructions and you don’t ever do anything else on the computer, but that’s going to be rare for any consumer. > The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread. “It wouldn’t be hard” is a huge understatement. This is OS-level support, not something Intel can just do inside the CPU. And operating system vendors and developers are still catching up to basic Alder Like scheduler support without AVX-512 handing, so it’s not reasonable to suggest this would be easy.

pantalaimon 5y ago
There have been reports that it was possible to enable AVX512 on the big cores with the little ones enabled [0] by toggling some bits in the MSR. For a first step it would have been enough to manually pin threads to the big cores for adventurous users, but Linux already tacks which threads use AVX (since it comes with a clock penalty, so you want to isolate them if possible), it's not unreasonable to think that auto-setting CPU mask based on that should be possible too. [0] https://lkml.org/lkml/2021/12/2/1059 https://lkml.org/lkml/2021/12/2/1059
gnufx 5y ago
I don't see how "poorly parallelized" is relevant compared with, say, a decent OpenMP BLAS which is likely to benefit from any reduced power consumption that allows a faster clock. In general on compute nodes it's worth turning off cores dynamically which aren't being used by jobs, like on a half-full node.

amluto 5y ago

> The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread. Having actually worked on this, it’s quite a bit more complicated, although it is possible. For better or for worse, though, Intel made a decision that all cores would expose the same feature set, and now this is baked into Linux and probably other OSes at the ABI level, and changing it would be a mess. A correct heterogeneous ABI (and frankly a good AVX512 ABI at all) would require explicit opt-in, per process, for AVX512. Opting in would change the signal frame format, the available features, and the set of allowed cores.

tmccrary55 5y ago
Boost your benchmarks with AVX-512 by subscribing to Intel+
- amluto 5y ago
  What benchmarks? Not a lot of real programs actually use AVX512, and a lot of the ones that did discovered that it made performance worse, so they stopped. (The issue is that AVX512 (the actual 512-bit parts, not the associated EVEX and masking extensions) may well be excellent for long-running vector-math-heavy usage, but the cost of switching AVX512 on and off is extreme, and using it for things like memcpy() and strcmp() is pretty much always a loss except in silly microbenchmarks.) To be clear, I don't like the type of product line differentiation that Intel does, and I think Intel should have supported proper heterogenous ISA systems so that AVX512 and related technologies on client systems would make sense, but I don't think any of this is nefarious.
  - hajile 5y ago
    > What benchmarks? Not a lot of real programs actually use AVX512, and a lot of the ones that did discovered that it made performance worse, so they stopped. This paints a picture of what happened to Intel's fab process rather than AVX-512 itself. AVX-512 was proposed back in 2013. It was NOT designed for desktops. The original designs were for their Phi chips (basically turning a bunch of x86 cores into a GPU). These Phi chips ran between 1 and 1.5GHz, so power consumption and clocks were always matched up. Intel wanted to move these instructions into their HPC CPUs. The problem at hand was ultra-high turbo speeds. These speeds work because the heat is a bit spread out on the chip and a lot of pieces are disabled at any given time. With AVX-512, they had 30% of the core going wide open for the vector units (not to mention the added usage from saturating the load/store bandwidth). They wanted 10nm and then 7nm to fix these issues. At their predicted schedule, 10nm would have launched in 2015 and 7nm in 2017. Given the introduction of AVX-512 in 2013, they had plenty of time. In fact, the first official product was Knight's landing in 2016. Skylake with AVX-512 didn't launch until 2017 when they were supposed to be on 7nm. Intel was forced to backport their designs to 14nm++++++++++++ which forced a deal with the devil. They had to downclock the CPU to keep within thermal limits, but this slowed down EVERYTHING. Maybe they could have created a separate power plane for AVX, but this would be a BIG change (and probably politically infeasible). What happens with the downclocking? If you run dedicated AVX-512 loads, then maybe you should have been looking at Phi instead. If not, mixed loads suffered overall because of the lower clockspeeds. Their second revision of 10nm superfin is still a generation larger (well, probably a half-generation) than what they anticipated. There might still be downclocking, but I'd guess that it's nowhere near what previous iterations required. TL;DR -- AVX-512 was launched two nodes too early which screwed over performance. It should become acceptable either with 10nm Superfin or 7nm when it launches so the CPU doesn't have to downclock constantly.