8 ms·

And meanwhile after 9 years of existing intel is still not reliably adding avx 512 to flagship desktop/gaming cpu's, first because of 5+ years of using the same

And meanwhile after 9 years of existing intel is still not reliably adding avx 512 to flagship desktop/gaming cpu's, first because of 5+ years of using the same old architecture and process, now because I don't know why they disabled it when it was there. Why was it not a problem for Intel to add mmx, several sse iterations, avx, avx 2, but now 9 years of nothing, even though parallel computing in annoying to program for and depend on gpu's is only gaining in importance?

ceeplusplus 5y ago
It costs a lot of area for something that's doesn't move the needle for the majority of desktop applications (browsers, games, office apps). The AVX512 units on Skylake-SP for example are a substantial portion of the area of each core. At some point you have to consider how many cores you could fit in the area used for vector extensions and make a trade-off.
- aseipp 5y ago
  This argument IMO is rather overblown. A quick die shot of Skylake-X indicates to me that even if you completely nerfed every AVX-512 unit appropriately (including the register files) on say, an 8-core or 10-core processor, you're going to get what? 1 or 2 more cores at maximum?[1] People make it sound like 70% of every Skylake-X die just sits there and you could have 50x more cores, when it's not that simple. You can delete tons of features from a modern microprocessor to save space but that isn't always the best way forward. In any case, I think this whole argument really misses another important thing, which is that the ISA and the vector width are separate here different. If Intel would just give us AVX-512 with 256b or 128b vectors, it would lead to a very big increase in use IMO, without having to have massive register files and vector data paths complicate things. The instruction set improvements are good enough to justify this IMO. (Alder Lake makes it a more complex case since they'd need to do a lot of work to unify Gracemont/Golden Cove before they could even think about that, but I still think it would be great.) They could even just half the width of the vector load/store units relative to the vector size like AMD did with Zen 2 (e.g. 256b vecs are split into 2x128b ops.) I'll still take it [1] https://twitter.com/GPUsAreMagic/status/1256866465577394181/photo/2 https://twitter.com/GPUsAreMagic/status/1256866465577394181/...
  - gpderetta 5y ago
    Exactly. Even just one AVX512 ALU instead of the typical two would be an option (which in fact Intel has used on some *lake variants).
- Aardwolf 5y ago
  > It costs a lot of area for something that's doesn't move the needle for the majority of desktop applications But that could be a bit of a chicken and egg situation, right?
  - paulmd 5y ago
    Absolutely. It's useful in text processing, it's useful in emulation, it's useful for doing bit-math for various encodings and checksumming, etc. The newer revisions don't have any downclocking either. https://old.reddit.com/r/emulation/comments/lzfpz5/what_are_the_implications_of_avx512_for_emulation/ https://old.reddit.com/r/emulation/comments/lzfpz5/what_are_... https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre... People have made some incredibly broad assertions about the technology as a whole on the basis of a first-gen product that lingered on the market for far too long because of the 10nm woes. People also hang on Linus's words far, far too much. He's incredibly toxic and willing to opine about stuff he's been disconnected from for 20 years. Lisa Su is putting her money where her mouth is, AMD is making a billion-dollar bet on AVX-512 at a time when Intel has actually abandoned it in the consumer market entirely. Maybe she's doing it just for the server market, but that still means she thinks server-market customers are going to see an actual benefit on normal server tasks, such that it will be worth making every single customer pay more for a larger piece of silicon. It's not the kind of thing you do "just to win a couple HPC benchmarks". Sorry, I trust the person with the financial stake in this discussion a lot more than I trust Linus's spit-takes.
- janwas 5y ago
  Those are surprising claims. Browsers and their video/image codecs use SIMD. Other usages in office/productivity apps include Photoshop/Lightroom, video editing, compression, CAD.
jeffbee 5y ago
I don't know where you came up with 9 years. The very first CPU that had these features came out in May 2018. "Tiger Lake" has these features and it is/was Intel's mainstream laptop CPU for the past year or so. Adler Lake, their current generation, lacks these features but I think it's understandable because they had to add AVX, AVX2, BMI, BMI2, FMA, VAES, and a bunch of other junk to the Tremont design in order to make a big/little CPU that works seamlessly. Whether you think they should instead have made a heterogeneous design that is harder to use is another question.
- gnabgib 5y ago
  Intel proposed AVX512 in 2013[0], with first appearance on Xeon Phi x200 announced in 2013, launched in 2016[1], and then on Skylake-X released in 2017[2] [0]: https://en.wikipedia.org/wiki/AVX-512 https://en.wikipedia.org/wiki/AVX-512 [1]: https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing [2]: https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#High-end_desktop_processors_(Skylake-X) https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#Hi...
  - wtallis 5y ago
    Also consumer Skylake started shipping in 2015 with some non-functional silicon area reserved for AVX-512 register files.
    - jeffbee 5y ago
      Skylake did not have IFMA or VBMI. The first microarchitecture with both of those was Cannon Lake, Q2 2018, which practically did not exist in the market, and the first mainstream CPU with both of these was Ice Lake, Q3 2019.
  - adrian_b 5y ago
    Actually AVX-512 is considerably older than AVX. Initially AVX-512 was known as "Larrabee New Instructions". This instruction set, which included essential features, which have been missing in both earlier and later Intel ISAs, e.g. mask registers and scatter-gather instructions, was developed a few years before 2009, by a team in which many people had been brought from outside Intel. The "Larrabee New Instructions" have been disclosed publicly in 2009, then the first hardware implementation available outside Intel, "Knights Ferry" was released in May 2010. Due to poor performance against GPUs, it was available only in development systems. A year later, in 2011, Sandy Bridge was launched, the first Intel product with AVX. Even if AVX had significant improvements over SSE, it was seriously crippled in comparison with the older AVX-512 a.k.a. "Larrabee New Instructions". It would have been much better for the Intel customers if Sandy Bridge would have implemented a 256-bit version of AVX-512 instead of implementing AVX. However Intel has always attempted to implement as few improvements as possible in each CPU generation, in order to minimize their production costs and maximize their profits. This worked very well for them as long as they did not have serious competition. The next implementation of AVX-512 (using the name "Intel Many Integrated Cores Instructions"), was in Knights Corner, the first Xeon Phi, launched in Q4 2012. This version made some changes in the encoding of the instructions and it also removed some instructions intended for GPU applications. The next implementation of AVX-512, which changed again the encoding of the instructions to the one used today, and which changed its name to AVX-512, was in Knights Landing, which was launched in Q2 2016. With the launch of Skylake Server, in Q3 2017, AVX-512 appeared for the first time in mainstream Intel CPUs, but after removing some sets of instructions previously available on Xeon Phi. AVX-512 is a much more pleasant ISA than AVX, e.g. by using the mask registers it is much easier to program loops when the length and alignment of data is arbitrary. Unfortunately the support for it is unpredictable, so it is usually not worthwhile to optimize for it. Hopefully the rumors that Zen 4 supports AVX-512 are true, so its launch might be the real start of widespread use of AVX-512.
Findecanor 5y ago
AVX-512 still isn't one instruction set either. It is several that are overlapping, with different feature sets. I have seen several accounts of them being inefficient (drawing much power, large chip area, dragging down the clock of the entire chip). They have also been criticised for making context switches longer. There are some good ideas in there though. What I think Intel should do is to redesign it as a single modern instruction set for 128 and 256-bit vectors (or for scalable vector lengths such as where ARM and RISC-V are going). Then express older SSE-AVX2 instructions in the new instructions' microcode, with the goal of replacing SSE-AVX2 in the long term. Otherwise, if Intel doesn't have a instruction set that is modern and feature-complete, I think Intel would be falling behind.
- aseipp 5y ago
  You can already use the AVX-512 instructions with vectors smaller than 512 bits; that, in fact, is one of the extensions to the base microarchitecture, and is present in every desktop/server CPU that has it available (but not the original Xeon Phis, IIRC.) This is called the "Vector Length" extension. Edit: I noted you said "redesign" which is an important point. I don't disagree in principle, I just point this out to be clear you can use the new ISA with smaller vectors already, to anyone who doesn't know this. No need to throw out baby with bathwater. Maybe their "redesign" could just be ratifying this as a separate ISA feature... > drawing much power, large chip area, dragging down the clock of the entire chip The chip area thing has mostly been overblown IMO, the vector unit itself is rather large but would be anyway (you can't get away from this if you want fat vectors), and the register file is shared anyway so it's "only" 2x bigger relative to a 256b one, but still pales in comparison to the L2 or L3. The power considerations are significantly better at least as of Ice Lake/Tiger Lake, having pretty minimal impact on clock speeds/power licensing, but it's worth nothing these have only 1 FMA unit in consumer SKUs and are traditionally low core count designs. Seeing AVX-512 performance/power profiles on Sapphire Rapids or Ice Lake Xeon (dual FMAs, for one) would be more interesting but again it wouldn't match desktop SKUs in any case. It's also worth noting AVX2 has similar problems e.g. on Haswell machines where it first appeared, and would similarly cause downclocking due to thermal constraints; but at the time the blogosphere wasn't as advanced and efficient at regurgitating wives' tales with zero context or wider understanding, so you don't see it talked about as much. In any case I suspect Intel is probably thinking about moving in the direction of variable-width vector sizes. But they probably think about a lot of things. It's unclear how this will all pan out, though I do admit AVX-512 overall counts as a bit of a "fumbled bag" for them, so to speak. I just want the instructions! No bigger vectors needed.
matja 5y ago
which AVX-512? :) - https://twitter.com/InstLatX64/status/1114141314441011200/photo/1 https://twitter.com/InstLatX64/status/1114141314441011200/ph...
PragmaticPulp 5y ago
> Why was it not a problem for Intel to add mmx, several sse iterations, avx, avx 2, but now 9 years of nothing, even though parallel computing in annoying to program for and depend on gpu's is only gaining in importance? AVX512 is significantly more intensive than previous generation SIMD instruction sets. It takes up a lot of die space and it uses a massive amount of power. It can use so much power and is so complex that the clock speed of the CPU is reduced slightly when AVX512 instructions are running. This led to an exaggerated outrage from enthusiast/gaming users who didn't want their clock speed reduced temporarily just because some program somewhere on their machine executed some AVX512 instructions. To this day you can still find articles about how to disable AVX512 for this reason. AVX512 is also incompatible with having a heterogeneous architecture with smaller efficiency cores. The AVX512 part of the die is massive and power-hungry, so it's not really an option for making efficiency cores. I think Intel is making the right choice to move away from AVX512 for consumer use cases. Very few, if any, consumer-target applications would even benefit from AVX512 optimizations. Very few companies would actually want to do it, given that the optimizations wouldn't run on AMD CPUs or many Intel CPUs. It's best left as a specific optimization technique for very specific use cases where you know you're controlling the server hardware.
- Aardwolf 5y ago
  MMX was 64-bit wide, SSE 128-bit wide and AVX 256-bit wide, and they succeeded each other rapidly (when compared to the situation now). AVX512 is another doubling, so what's the hold up with this one compared to the previous two doublings? Maybe it's time for AMD to do something about this, like when they took the initiative to create x86-64
  - arcticbull 5y ago
    As the 4th doubling, that's a 16X increase over the base. 512-bit registers, ALUs, data paths, etc, are all really really physically big.
  - mcronce 5y ago
    Zen 4 is reported to have AVX-512 support. I'm not sure if that'll be included on Ryzen or not, though; only Epyc is "confirmed" as far as I know.
needusername 5y ago
> now because I don't know why they disabled it when it was there E-cores (Gracemont) don't support AVX-512, only P-cores (Golden Cove) support AVX-512
- Aardwolf 5y ago
  Sure, but they deliberately released a BIOS version that prevented people from enabling it when disabling E-cores
cl0ckt0wer 5y ago
It looks like they're shifting to efficiency as a target. In their latest 12th gen desktop CPUs, you could re-enable avx512 if you disabled the "efficiency" cores. Then they released a BIOS update with that feature removed. Has there been an instruction set that failed? Itanium maybe?
paulmd 5y ago
They disabled it because it's not compatible with their efficiency-core design for big.LITTLE - right now, you either have to put AVX-512 on both, or neither. An efficiency core with AVX-512 is basically a Xeon Phi core (and that approach actually probably would be good at the compute-heavy workloads efficiency cores are optimized for), so that's not an absurdity, but they didn't do that. So at that point the answer is "neither", and they turned it off on their big cores. But apparently they left the unit there in the silicon eating up a fairly large amount of space (I think it's around 10-20% of the core). The real question is why nobody thought of this before... did they have some hardware/software solution to handle the instruction set differences that didn't work out? The fact that it was shoddily disabled before launch, and left in programmers' guides and other documents, speaks that maybe there was something like that and somehow it didn't end up happening. (You could trap instruction errors and mark the thread as big-core-only, the Linux kernel did that at one point for AVX instructions in general (to save registers/bandwidth/etc they wouldn't save the AVX registers unless a thread actually used AVX). But there's more problems that still have to be solved first - how does software know how many AVX-aware cores are available? How does software know which core their CPUID instruction will be executed on? etc etc.) If they had a heterogenous-ISA mechanism or something that will be introduced later, why isn't it on the roadmaps for next-gen/etc. And they didn't have those mechanisms, then why is it still on the core on an extremely high-volume consumer architecture? You could say maybe it's a testbed for Sapphire Rapids, but every company has dedicated test silicon for that, and 10-20% of every single consumer chip sold is a huge waste of money. Sure, they'd have to design a core without it, but Intel absolutely has the money to do that - just like Skylake vs Skylake-X. The unit is designed to be modular so it's not hard, and they'd make it back extremely quickly. Maybe they just needed some dead silicon to spread out the thermal load a bit? Obviously dead silicon has to be dead, so it doesn't matter if you print AVX-512 or the names of the design team, it's not supposed to be powered up either way. Anyway that's my take - the "why take it back off" part is straightforward, the answer is big.LITTLE. The real interesting question is "so why is it still on the silicon".
- twoodfin 5y ago
  Feels like the answer is Conway’s Law, one way or another.
KolenCh 5y ago
To supplement other comments, AVX-512 is an umbrella of a bunch of related stuffs. Some chips has a smaller subset of AVX-512 capabilities and other have more. Linus has mentioned he want AVX-512 to die a gruesome death. AMD also doesn’t actually support AVX-512 (but when seen run it in 2 cycles of 256-bit wide instruction aka AVX-2.) The whole AVX-512 situation is just so complicated in hardwares and hence softwares (which subset to enable?) and this is probably the primary reasons for the lack of adoption, even from Intel themselves. Eg the disabled AVX-512 that you alluded to in recent Intel CPU is because of its hybrid nature and the little CPU aren’t capable of AVX-512 and since they want to same instructions to be able to run on both big and little CPU, the only move is to disable it. Anandtech tested with the little CPU disabled in BIOS, and while Intel said the AVX-512 is fused off, they actually are able to run AVX-512.
- janwas 5y ago
  Not sure why the AVX-512 situation is complicated. As paulmd commented, in practice all the various avx512* supported by more recent chips (except Phi) are supersets of the previous ones. github.com/highway compiles the same source code to AVX2 or AVX512, and checks at runtime which to use. I've seen 1.5x speedups from this (even including throttling on 5 year old Skylakes). Why not use what's there?
  - KolenCh 5y ago
    Caveat 1: My comment was intended to be a supplement to others so it isn't aimed to be self-contained nor exhaustive. Caveat 2: I am a user of Intel Xeon Phi (KNL specifically) on an HPC system. So my view is biased in this way. It's probably right that the extensions has much less variants from non-Xeon Phi line of products. But, it still carries some weights, because it is a sign that Intel was struggling to strike a balance at least in the beginning. They wasn't sure what to offer, the fact that Xeon Phi line is killed and replaced by GPU is their admission of failure there. Then vendor other than Intel (AMD) would hesitate to adopt that because even Intel wasn't sure what to offer. And if AMD doesn't pick that up, the usefulness of AVX-512 is in doubt. E.g. in the traditional way of prepping binary releases (many distribution methods still do) is to target the "highest common factor" of supported instructions of reasonably recent CPUs, meaning they would not target AVX-512 at all as the latest CPU from AMD doesn't have it (they later add fake AVX-512 capability.) Now it means that whether AVX-512 is useful to you as an end users then depends on the distributions a lot (and in practice none of the general purpose binaries are targeting this. In the rare case they build using the Intel compilers, it is very easy to ship both, which is not true if using GNU compilers.) And if it ain't going to useful for the majority of users, there's no hurry to upgrade specially for AVX-512. Then it becomes a disadvantage for Intel to increase the cost of making AVX-512 capable CPUs to compute in the lower ends. Note that there are other comments and articles on this already—AVX-512 is not cheap. It costs significant die area which is cost. That further complicates the things when Intel is introducing BIG.little design, where in the little CPU, low energy draw and low cost hence smaller die size drives towards having no AVX-512. And since the only way to guarantee any binaries can jump between the BIG and little CPU, they choose to disable AVX-512 on the BIG CPU even if its there. (AnandTech article is much more detailed about this.) P.S. I'm sure historians would give a better and more balanced accounts on why it fails, but I think it is safe to say the standard is now too fragmented, not necessarily in implementation sans Xeon Phi but in the market, it is probably remains to be a niche. P.S.2 A fun fact is that in the recent years AMD's CPU are praised by their performance to watt ratio because of the advances in fabrication tech (basically from TSMC's advances). But I did some calculations earlier that if we're only talking about FLOPS, and compares FLOPS per watt with the consideration of AVX-512 units (many Intel CPUs even have 2 AVX-512 units), it is actually comparable if not lower for Intel. (That was an old calculations and with the newest Intel getting on better fab. I'm sure the story has changed already.) But then no much people if any (I heard none, but I don't read too much except sometimes from AnandTech) has mentioned this and make the comparison "fair" to Intel. Why? Because the end users most of the time aren't going to be benefited by AVX-512 (again because of distributions) so that's often not taken into accounts, sometimes indirectly by performing benchmarks that is not AVX-512 aware.

ceeplusplus 5y ago

It costs a lot of area for something that's doesn't move the needle for the majority of desktop applications (browsers, games, office apps). The AVX512 units on Skylake-SP for example are a substantial portion of the area of each core. At some point you have to consider how many cores you could fit in the area used for vector extensions and make a trade-off.

aseipp 5y ago
This argument IMO is rather overblown. A quick die shot of Skylake-X indicates to me that even if you completely nerfed every AVX-512 unit appropriately (including the register files) on say, an 8-core or 10-core processor, you're going to get what? 1 or 2 more cores at maximum?[1] People make it sound like 70% of every Skylake-X die just sits there and you could have 50x more cores, when it's not that simple. You can delete tons of features from a modern microprocessor to save space but that isn't always the best way forward. In any case, I think this whole argument really misses another important thing, which is that the ISA and the vector width are separate here different. If Intel would just give us AVX-512 with 256b or 128b vectors, it would lead to a very big increase in use IMO, without having to have massive register files and vector data paths complicate things. The instruction set improvements are good enough to justify this IMO. (Alder Lake makes it a more complex case since they'd need to do a lot of work to unify Gracemont/Golden Cove before they could even think about that, but I still think it would be great.) They could even just half the width of the vector load/store units relative to the vector size like AMD did with Zen 2 (e.g. 256b vecs are split into 2x128b ops.) I'll still take it [1] https://twitter.com/GPUsAreMagic/status/1256866465577394181/photo/2 https://twitter.com/GPUsAreMagic/status/1256866465577394181/...
- gpderetta 5y ago
  Exactly. Even just one AVX512 ALU instead of the typical two would be an option (which in fact Intel has used on some *lake variants).
Aardwolf 5y ago
> It costs a lot of area for something that's doesn't move the needle for the majority of desktop applications But that could be a bit of a chicken and egg situation, right?
- paulmd 5y ago
  Absolutely. It's useful in text processing, it's useful in emulation, it's useful for doing bit-math for various encodings and checksumming, etc. The newer revisions don't have any downclocking either. https://old.reddit.com/r/emulation/comments/lzfpz5/what_are_the_implications_of_avx512_for_emulation/ https://old.reddit.com/r/emulation/comments/lzfpz5/what_are_... https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre... People have made some incredibly broad assertions about the technology as a whole on the basis of a first-gen product that lingered on the market for far too long because of the 10nm woes. People also hang on Linus's words far, far too much. He's incredibly toxic and willing to opine about stuff he's been disconnected from for 20 years. Lisa Su is putting her money where her mouth is, AMD is making a billion-dollar bet on AVX-512 at a time when Intel has actually abandoned it in the consumer market entirely. Maybe she's doing it just for the server market, but that still means she thinks server-market customers are going to see an actual benefit on normal server tasks, such that it will be worth making every single customer pay more for a larger piece of silicon. It's not the kind of thing you do "just to win a couple HPC benchmarks". Sorry, I trust the person with the financial stake in this discussion a lot more than I trust Linus's spit-takes.
janwas 5y ago
Those are surprising claims. Browsers and their video/image codecs use SIMD. Other usages in office/productivity apps include Photoshop/Lightroom, video editing, compression, CAD.

jeffbee 5y ago

I don't know where you came up with 9 years. The very first CPU that had these features came out in May 2018. "Tiger Lake" has these features and it is/was Intel's mainstream laptop CPU for the past year or so. Adler Lake, their current generation, lacks these features but I think it's understandable because they had to add AVX, AVX2, BMI, BMI2, FMA, VAES, and a bunch of other junk to the Tremont design in order to make a big/little CPU that works seamlessly. Whether you think they should instead have made a heterogeneous design that is harder to use is another question.

gnabgib 5y ago
Intel proposed AVX512 in 2013[0], with first appearance on Xeon Phi x200 announced in 2013, launched in 2016[1], and then on Skylake-X released in 2017[2] [0]: https://en.wikipedia.org/wiki/AVX-512 https://en.wikipedia.org/wiki/AVX-512 [1]: https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing [2]: https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#High-end_desktop_processors_(Skylake-X) https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#Hi...
- wtallis 5y ago
  Also consumer Skylake started shipping in 2015 with some non-functional silicon area reserved for AVX-512 register files.
  - jeffbee 5y ago
    Skylake did not have IFMA or VBMI. The first microarchitecture with both of those was Cannon Lake, Q2 2018, which practically did not exist in the market, and the first mainstream CPU with both of these was Ice Lake, Q3 2019.
- adrian_b 5y ago
  Actually AVX-512 is considerably older than AVX. Initially AVX-512 was known as "Larrabee New Instructions". This instruction set, which included essential features, which have been missing in both earlier and later Intel ISAs, e.g. mask registers and scatter-gather instructions, was developed a few years before 2009, by a team in which many people had been brought from outside Intel. The "Larrabee New Instructions" have been disclosed publicly in 2009, then the first hardware implementation available outside Intel, "Knights Ferry" was released in May 2010. Due to poor performance against GPUs, it was available only in development systems. A year later, in 2011, Sandy Bridge was launched, the first Intel product with AVX. Even if AVX had significant improvements over SSE, it was seriously crippled in comparison with the older AVX-512 a.k.a. "Larrabee New Instructions". It would have been much better for the Intel customers if Sandy Bridge would have implemented a 256-bit version of AVX-512 instead of implementing AVX. However Intel has always attempted to implement as few improvements as possible in each CPU generation, in order to minimize their production costs and maximize their profits. This worked very well for them as long as they did not have serious competition. The next implementation of AVX-512 (using the name "Intel Many Integrated Cores Instructions"), was in Knights Corner, the first Xeon Phi, launched in Q4 2012. This version made some changes in the encoding of the instructions and it also removed some instructions intended for GPU applications. The next implementation of AVX-512, which changed again the encoding of the instructions to the one used today, and which changed its name to AVX-512, was in Knights Landing, which was launched in Q2 2016. With the launch of Skylake Server, in Q3 2017, AVX-512 appeared for the first time in mainstream Intel CPUs, but after removing some sets of instructions previously available on Xeon Phi. AVX-512 is a much more pleasant ISA than AVX, e.g. by using the mask registers it is much easier to program loops when the length and alignment of data is arbitrary. Unfortunately the support for it is unpredictable, so it is usually not worthwhile to optimize for it. Hopefully the rumors that Zen 4 supports AVX-512 are true, so its launch might be the real start of widespread use of AVX-512.

Findecanor 5y ago

AVX-512 still isn't one instruction set either. It is several that are overlapping, with different feature sets. I have seen several accounts of them being inefficient (drawing much power, large chip area, dragging down the clock of the entire chip). They have also been criticised for making context switches longer. There are some good ideas in there though. What I think Intel should do is to redesign it as a single modern instruction set for 128 and 256-bit vectors (or for scalable vector lengths such as where ARM and RISC-V are going). Then express older SSE-AVX2 instructions in the new instructions' microcode, with the goal of replacing SSE-AVX2 in the long term. Otherwise, if Intel doesn't have a instruction set that is modern and feature-complete, I think Intel would be falling behind.

aseipp 5y ago
You can already use the AVX-512 instructions with vectors smaller than 512 bits; that, in fact, is one of the extensions to the base microarchitecture, and is present in every desktop/server CPU that has it available (but not the original Xeon Phis, IIRC.) This is called the "Vector Length" extension. Edit: I noted you said "redesign" which is an important point. I don't disagree in principle, I just point this out to be clear you can use the new ISA with smaller vectors already, to anyone who doesn't know this. No need to throw out baby with bathwater. Maybe their "redesign" could just be ratifying this as a separate ISA feature... > drawing much power, large chip area, dragging down the clock of the entire chip The chip area thing has mostly been overblown IMO, the vector unit itself is rather large but would be anyway (you can't get away from this if you want fat vectors), and the register file is shared anyway so it's "only" 2x bigger relative to a 256b one, but still pales in comparison to the L2 or L3. The power considerations are significantly better at least as of Ice Lake/Tiger Lake, having pretty minimal impact on clock speeds/power licensing, but it's worth nothing these have only 1 FMA unit in consumer SKUs and are traditionally low core count designs. Seeing AVX-512 performance/power profiles on Sapphire Rapids or Ice Lake Xeon (dual FMAs, for one) would be more interesting but again it wouldn't match desktop SKUs in any case. It's also worth noting AVX2 has similar problems e.g. on Haswell machines where it first appeared, and would similarly cause downclocking due to thermal constraints; but at the time the blogosphere wasn't as advanced and efficient at regurgitating wives' tales with zero context or wider understanding, so you don't see it talked about as much. In any case I suspect Intel is probably thinking about moving in the direction of variable-width vector sizes. But they probably think about a lot of things. It's unclear how this will all pan out, though I do admit AVX-512 overall counts as a bit of a "fumbled bag" for them, so to speak. I just want the instructions! No bigger vectors needed.

matja 5y ago

which AVX-512? :) - https://twitter.com/InstLatX64/status/1114141314441011200/photo/1 https://twitter.com/InstLatX64/status/1114141314441011200/ph...

PragmaticPulp 5y ago

> Why was it not a problem for Intel to add mmx, several sse iterations, avx, avx 2, but now 9 years of nothing, even though parallel computing in annoying to program for and depend on gpu's is only gaining in importance? AVX512 is significantly more intensive than previous generation SIMD instruction sets. It takes up a lot of die space and it uses a massive amount of power. It can use so much power and is so complex that the clock speed of the CPU is reduced slightly when AVX512 instructions are running. This led to an exaggerated outrage from enthusiast/gaming users who didn't want their clock speed reduced temporarily just because some program somewhere on their machine executed some AVX512 instructions. To this day you can still find articles about how to disable AVX512 for this reason. AVX512 is also incompatible with having a heterogeneous architecture with smaller efficiency cores. The AVX512 part of the die is massive and power-hungry, so it's not really an option for making efficiency cores. I think Intel is making the right choice to move away from AVX512 for consumer use cases. Very few, if any, consumer-target applications would even benefit from AVX512 optimizations. Very few companies would actually want to do it, given that the optimizations wouldn't run on AMD CPUs or many Intel CPUs. It's best left as a specific optimization technique for very specific use cases where you know you're controlling the server hardware.

Aardwolf 5y ago
MMX was 64-bit wide, SSE 128-bit wide and AVX 256-bit wide, and they succeeded each other rapidly (when compared to the situation now). AVX512 is another doubling, so what's the hold up with this one compared to the previous two doublings? Maybe it's time for AMD to do something about this, like when they took the initiative to create x86-64
- arcticbull 5y ago
  As the 4th doubling, that's a 16X increase over the base. 512-bit registers, ALUs, data paths, etc, are all really really physically big.
- mcronce 5y ago
  Zen 4 is reported to have AVX-512 support. I'm not sure if that'll be included on Ryzen or not, though; only Epyc is "confirmed" as far as I know.

needusername 5y ago

> now because I don't know why they disabled it when it was there E-cores (Gracemont) don't support AVX-512, only P-cores (Golden Cove) support AVX-512

Aardwolf 5y ago
Sure, but they deliberately released a BIOS version that prevented people from enabling it when disabling E-cores

cl0ckt0wer 5y ago

It looks like they're shifting to efficiency as a target. In their latest 12th gen desktop CPUs, you could re-enable avx512 if you disabled the "efficiency" cores. Then they released a BIOS update with that feature removed. Has there been an instruction set that failed? Itanium maybe?

paulmd 5y ago

They disabled it because it's not compatible with their efficiency-core design for big.LITTLE - right now, you either have to put AVX-512 on both, or neither. An efficiency core with AVX-512 is basically a Xeon Phi core (and that approach actually probably would be good at the compute-heavy workloads efficiency cores are optimized for), so that's not an absurdity, but they didn't do that. So at that point the answer is "neither", and they turned it off on their big cores. But apparently they left the unit there in the silicon eating up a fairly large amount of space (I think it's around 10-20% of the core). The real question is why nobody thought of this before... did they have some hardware/software solution to handle the instruction set differences that didn't work out? The fact that it was shoddily disabled before launch, and left in programmers' guides and other documents, speaks that maybe there was something like that and somehow it didn't end up happening. (You could trap instruction errors and mark the thread as big-core-only, the Linux kernel did that at one point for AVX instructions in general (to save registers/bandwidth/etc they wouldn't save the AVX registers unless a thread actually used AVX). But there's more problems that still have to be solved first - how does software know how many AVX-aware cores are available? How does software know which core their CPUID instruction will be executed on? etc etc.) If they had a heterogenous-ISA mechanism or something that will be introduced later, why isn't it on the roadmaps for next-gen/etc. And they didn't have those mechanisms, then why is it still on the core on an extremely high-volume consumer architecture? You could say maybe it's a testbed for Sapphire Rapids, but every company has dedicated test silicon for that, and 10-20% of every single consumer chip sold is a huge waste of money. Sure, they'd have to design a core without it, but Intel absolutely has the money to do that - just like Skylake vs Skylake-X. The unit is designed to be modular so it's not hard, and they'd make it back extremely quickly. Maybe they just needed some dead silicon to spread out the thermal load a bit? Obviously dead silicon has to be dead, so it doesn't matter if you print AVX-512 or the names of the design team, it's not supposed to be powered up either way. Anyway that's my take - the "why take it back off" part is straightforward, the answer is big.LITTLE. The real interesting question is "so why is it still on the silicon".

twoodfin 5y ago
Feels like the answer is Conway’s Law, one way or another.

KolenCh 5y ago

To supplement other comments, AVX-512 is an umbrella of a bunch of related stuffs. Some chips has a smaller subset of AVX-512 capabilities and other have more. Linus has mentioned he want AVX-512 to die a gruesome death. AMD also doesn’t actually support AVX-512 (but when seen run it in 2 cycles of 256-bit wide instruction aka AVX-2.) The whole AVX-512 situation is just so complicated in hardwares and hence softwares (which subset to enable?) and this is probably the primary reasons for the lack of adoption, even from Intel themselves. Eg the disabled AVX-512 that you alluded to in recent Intel CPU is because of its hybrid nature and the little CPU aren’t capable of AVX-512 and since they want to same instructions to be able to run on both big and little CPU, the only move is to disable it. Anandtech tested with the little CPU disabled in BIOS, and while Intel said the AVX-512 is fused off, they actually are able to run AVX-512.

janwas 5y ago
Not sure why the AVX-512 situation is complicated. As paulmd commented, in practice all the various avx512* supported by more recent chips (except Phi) are supersets of the previous ones. github.com/highway compiles the same source code to AVX2 or AVX512, and checks at runtime which to use. I've seen 1.5x speedups from this (even including throttling on 5 year old Skylakes). Why not use what's there?
- KolenCh 5y ago
  Caveat 1: My comment was intended to be a supplement to others so it isn't aimed to be self-contained nor exhaustive. Caveat 2: I am a user of Intel Xeon Phi (KNL specifically) on an HPC system. So my view is biased in this way. It's probably right that the extensions has much less variants from non-Xeon Phi line of products. But, it still carries some weights, because it is a sign that Intel was struggling to strike a balance at least in the beginning. They wasn't sure what to offer, the fact that Xeon Phi line is killed and replaced by GPU is their admission of failure there. Then vendor other than Intel (AMD) would hesitate to adopt that because even Intel wasn't sure what to offer. And if AMD doesn't pick that up, the usefulness of AVX-512 is in doubt. E.g. in the traditional way of prepping binary releases (many distribution methods still do) is to target the "highest common factor" of supported instructions of reasonably recent CPUs, meaning they would not target AVX-512 at all as the latest CPU from AMD doesn't have it (they later add fake AVX-512 capability.) Now it means that whether AVX-512 is useful to you as an end users then depends on the distributions a lot (and in practice none of the general purpose binaries are targeting this. In the rare case they build using the Intel compilers, it is very easy to ship both, which is not true if using GNU compilers.) And if it ain't going to useful for the majority of users, there's no hurry to upgrade specially for AVX-512. Then it becomes a disadvantage for Intel to increase the cost of making AVX-512 capable CPUs to compute in the lower ends. Note that there are other comments and articles on this already—AVX-512 is not cheap. It costs significant die area which is cost. That further complicates the things when Intel is introducing BIG.little design, where in the little CPU, low energy draw and low cost hence smaller die size drives towards having no AVX-512. And since the only way to guarantee any binaries can jump between the BIG and little CPU, they choose to disable AVX-512 on the BIG CPU even if its there. (AnandTech article is much more detailed about this.) P.S. I'm sure historians would give a better and more balanced accounts on why it fails, but I think it is safe to say the standard is now too fragmented, not necessarily in implementation sans Xeon Phi but in the market, it is probably remains to be a niche. P.S.2 A fun fact is that in the recent years AMD's CPU are praised by their performance to watt ratio because of the advances in fabrication tech (basically from TSMC's advances). But I did some calculations earlier that if we're only talking about FLOPS, and compares FLOPS per watt with the consideration of AVX-512 units (many Intel CPUs even have 2 AVX-512 units), it is actually comparable if not lower for Intel. (That was an old calculations and with the newest Intel getting on better fab. I'm sure the story has changed already.) But then no much people if any (I heard none, but I don't read too much except sometimes from AnandTech) has mentioned this and make the comparison "fair" to Intel. Why? Because the end users most of the time aren't going to be benefited by AVX-512 (again because of distributions) so that's often not taken into accounts, sometimes indirectly by performing benchmarks that is not AVX-512 aware.