4 ms·

Some of the biggest investments into general purpose Linux performance, reliability, and sustainability were made by people with "specialty engines" that Linus

by ak217 4y ago

Some of the biggest investments into general purpose Linux performance, reliability, and sustainability were made by people with "specialty engines" that Linus is talking about. The specialty applications often turn out to be important because they push the platform to excel in unexpected ways. With that said, the biggest practical problem we saw with AVX-512 was that it's so big, power hungry, and dissimilar to other components of the CPU. One tenant turning that thing on can throttle the whole chip, causing unpredictable performance and even reliability issues for everything on the machine. I imagine it's even worse on mobile/embedded, where turning it on can blow out your power and thermal budget. The alternatives (integrated GPUs/ML cores) seem to be more power efficient.

cogman10 4y ago
> and dissimilar to other components of the CPU It's not terribly dissimilar from the SSE instructions before it. The biggest weirdness about AVX is it's CPU throttling. Which, IMO, should not be a thing for the reasons you mention. I imagine that future iterations of AVX will likely ditch CPU throttling. I'm guessing it was a "our current design can't handle this and we need to ship!" sort of solution. (If they haven't already? Do current CPUs throttle AVX-512 instructions?) > I imagine it's even worse on mobile/embedded Fortunately, x86 mobile/embedded isn't terribly common, certainly not with AVX-512 support.
- Kon-Peki 4y ago
  > I imagine that future iterations of AVX will likely ditch CPU throttling Already happening: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Downclocking https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Dow...
- paulmd 4y ago
  > I imagine that future iterations of AVX will likely ditch CPU throttling. I'm guessing it was a "our current design can't handle this and we need to ship!" sort of solution. (If they haven't already? Do current CPUs throttle AVX-512 instructions?) exactly what it looks like to me, and it was already fixed in Ice Lake and probably doesn't exist at all in Zen4. Just nobody cares about Ice Lake-SP when Epyc is crushing the server market. https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre... and some measurements of Skylake-X displaying the effect, note the voltage swings in particular: https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html the whole thing looks very much to me like "oops we put not just AVX-512 but dual AVX-512 in our cores and on 14nm it can hammer the chip so hard we need more voltage... guess we're gonna have to stop and swing the voltage upwards". not only is that less of a problem with higher-efficiency nodes, but the consumer core designs drop the idea of dual-AVX 512 entirely which reduces the worst-case voltage droop as well...
janwas 4y ago
The turbo curves depend on the exact type of Xeon. Were they Bronze or Silver? Those are much worse in that regard. It is also unclear to me that AVX-512 is necessarily problematic in terms of power and thermal effects. Scalar code on many threads actually uses more energy (dispatch/control of OoO instructions being the main driver) than vector code (running far fewer instructions). The much-maligned power-up phase does not matter if running vector code for at least a millisecond. I agree dedicated HW can be more power-efficient still, assuming it does exactly what your algorithm requires. And yet they make the fragmentation and deployment issues (can I use it?) of SIMD look trivial by comparison :)
- sliken 4y ago
  I have some water cooled Xeon Gold (6100 series) chips. Generally they never throttle, unless running a code well known for heavy use of AVX 512. So despite running many codes that use all cores 100%, using AVX512 is the only way I've seen them throttle.
  - janwas 4y ago
    Thanks for the data point. Water cooled, impressive. But what is the problem here? Power is work per time. More power can be a good thing, especially if the work is useful. This is more likely in the SIMD case, which has much lower OoO cost than scalar code in terms of total work accomplished. At some point, the chip hits a limit and throttles. This means less speedup than otherwise might have been the case, but it's still a useful speedup relative to scalar, right?
nomel 4y ago
> One tenant turning that thing on can throttle the whole chip, This wasn't related to AVX-512, it was related to its implementation on that specific, somewhat old, architecture. Current AMD and next generation Intel don't have this limitation, so it shouldn't be considered when thinking of "the future".
dathinab 4y ago
> One tenant turning that thing on can throttle the whole chip, causing unpredictable performance and even reliability issues for everything on the machine This is where ironically where the double pumping of Zen 4 shines, it's less of an problem with it.

by ak217 4y ago

cogman10 4y ago

> and dissimilar to other components of the CPU It's not terribly dissimilar from the SSE instructions before it. The biggest weirdness about AVX is it's CPU throttling. Which, IMO, should not be a thing for the reasons you mention. I imagine that future iterations of AVX will likely ditch CPU throttling. I'm guessing it was a "our current design can't handle this and we need to ship!" sort of solution. (If they haven't already? Do current CPUs throttle AVX-512 instructions?) > I imagine it's even worse on mobile/embedded Fortunately, x86 mobile/embedded isn't terribly common, certainly not with AVX-512 support.

Kon-Peki 4y ago
> I imagine that future iterations of AVX will likely ditch CPU throttling Already happening: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Downclocking https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Dow...
paulmd 4y ago
> I imagine that future iterations of AVX will likely ditch CPU throttling. I'm guessing it was a "our current design can't handle this and we need to ship!" sort of solution. (If they haven't already? Do current CPUs throttle AVX-512 instructions?) exactly what it looks like to me, and it was already fixed in Ice Lake and probably doesn't exist at all in Zen4. Just nobody cares about Ice Lake-SP when Epyc is crushing the server market. https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre... and some measurements of Skylake-X displaying the effect, note the voltage swings in particular: https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html the whole thing looks very much to me like "oops we put not just AVX-512 but dual AVX-512 in our cores and on 14nm it can hammer the chip so hard we need more voltage... guess we're gonna have to stop and swing the voltage upwards". not only is that less of a problem with higher-efficiency nodes, but the consumer core designs drop the idea of dual-AVX 512 entirely which reduces the worst-case voltage droop as well...

janwas 4y ago

The turbo curves depend on the exact type of Xeon. Were they Bronze or Silver? Those are much worse in that regard. It is also unclear to me that AVX-512 is necessarily problematic in terms of power and thermal effects. Scalar code on many threads actually uses more energy (dispatch/control of OoO instructions being the main driver) than vector code (running far fewer instructions). The much-maligned power-up phase does not matter if running vector code for at least a millisecond. I agree dedicated HW can be more power-efficient still, assuming it does exactly what your algorithm requires. And yet they make the fragmentation and deployment issues (can I use it?) of SIMD look trivial by comparison :)

sliken 4y ago
I have some water cooled Xeon Gold (6100 series) chips. Generally they never throttle, unless running a code well known for heavy use of AVX 512. So despite running many codes that use all cores 100%, using AVX512 is the only way I've seen them throttle.
- janwas 4y ago
  Thanks for the data point. Water cooled, impressive. But what is the problem here? Power is work per time. More power can be a good thing, especially if the work is useful. This is more likely in the SIMD case, which has much lower OoO cost than scalar code in terms of total work accomplished. At some point, the chip hits a limit and throttles. This means less speedup than otherwise might have been the case, but it's still a useful speedup relative to scalar, right?

nomel 4y ago

> One tenant turning that thing on can throttle the whole chip, This wasn't related to AVX-512, it was related to its implementation on that specific, somewhat old, architecture. Current AMD and next generation Intel don't have this limitation, so it shouldn't be considered when thinking of "the future".

dathinab 4y ago

> One tenant turning that thing on can throttle the whole chip, causing unpredictable performance and even reliability issues for everything on the machine This is where ironically where the double pumping of Zen 4 shines, it's less of an problem with it.