3 ms·

Not sure why the AVX-512 situation is complicated. As paulmd commented, in practice all the various avx512* supported by more recent chips (except Phi) are supe

by janwas 5y ago

Not sure why the AVX-512 situation is complicated. As paulmd commented, in practice all the various avx512* supported by more recent chips (except Phi) are supersets of the previous ones. github.com/highway compiles the same source code to AVX2 or AVX512, and checks at runtime which to use. I've seen 1.5x speedups from this (even including throttling on 5 year old Skylakes). Why not use what's there?

KolenCh 5y ago
Caveat 1: My comment was intended to be a supplement to others so it isn't aimed to be self-contained nor exhaustive. Caveat 2: I am a user of Intel Xeon Phi (KNL specifically) on an HPC system. So my view is biased in this way. It's probably right that the extensions has much less variants from non-Xeon Phi line of products. But, it still carries some weights, because it is a sign that Intel was struggling to strike a balance at least in the beginning. They wasn't sure what to offer, the fact that Xeon Phi line is killed and replaced by GPU is their admission of failure there. Then vendor other than Intel (AMD) would hesitate to adopt that because even Intel wasn't sure what to offer. And if AMD doesn't pick that up, the usefulness of AVX-512 is in doubt. E.g. in the traditional way of prepping binary releases (many distribution methods still do) is to target the "highest common factor" of supported instructions of reasonably recent CPUs, meaning they would not target AVX-512 at all as the latest CPU from AMD doesn't have it (they later add fake AVX-512 capability.) Now it means that whether AVX-512 is useful to you as an end users then depends on the distributions a lot (and in practice none of the general purpose binaries are targeting this. In the rare case they build using the Intel compilers, it is very easy to ship both, which is not true if using GNU compilers.) And if it ain't going to useful for the majority of users, there's no hurry to upgrade specially for AVX-512. Then it becomes a disadvantage for Intel to increase the cost of making AVX-512 capable CPUs to compute in the lower ends. Note that there are other comments and articles on this already—AVX-512 is not cheap. It costs significant die area which is cost. That further complicates the things when Intel is introducing BIG.little design, where in the little CPU, low energy draw and low cost hence smaller die size drives towards having no AVX-512. And since the only way to guarantee any binaries can jump between the BIG and little CPU, they choose to disable AVX-512 on the BIG CPU even if its there. (AnandTech article is much more detailed about this.) P.S. I'm sure historians would give a better and more balanced accounts on why it fails, but I think it is safe to say the standard is now too fragmented, not necessarily in implementation sans Xeon Phi but in the market, it is probably remains to be a niche. P.S.2 A fun fact is that in the recent years AMD's CPU are praised by their performance to watt ratio because of the advances in fabrication tech (basically from TSMC's advances). But I did some calculations earlier that if we're only talking about FLOPS, and compares FLOPS per watt with the consideration of AVX-512 units (many Intel CPUs even have 2 AVX-512 units), it is actually comparable if not lower for Intel. (That was an old calculations and with the newest Intel getting on better fab. I'm sure the story has changed already.) But then no much people if any (I heard none, but I don't read too much except sometimes from AnandTech) has mentioned this and make the comparison "fair" to Intel. Why? Because the end users most of the time aren't going to be benefited by AVX-512 (again because of distributions) so that's often not taken into accounts, sometimes indirectly by performing benchmarks that is not AVX-512 aware.
- janwas 5y ago
  Oh, you are using Phi :) My viewpoint is likewise biased towards desktop and server systems. Fair point about heterogeneous cores. I suspect those aren't going away. Perhaps the industry should move to a model where threads are temporarily pinned to their core while running SIMD kernels, and re-checking the current CPU capabilities rather than storing the result. That would allow using both AVX2 and AVX-512 depending on which core is active, and doesn't require OS changes (except to OSX which AFAIK still cannot reliably pin threads), right? I totally agree that AVX-512 is helpful for power efficiency. Without it, I struggle to see how Intel/AMD are going to compete with current/future ARM and RISC-V.

by janwas 5y ago

KolenCh 5y ago

Caveat 1: My comment was intended to be a supplement to others so it isn't aimed to be self-contained nor exhaustive. Caveat 2: I am a user of Intel Xeon Phi (KNL specifically) on an HPC system. So my view is biased in this way. It's probably right that the extensions has much less variants from non-Xeon Phi line of products. But, it still carries some weights, because it is a sign that Intel was struggling to strike a balance at least in the beginning. They wasn't sure what to offer, the fact that Xeon Phi line is killed and replaced by GPU is their admission of failure there. Then vendor other than Intel (AMD) would hesitate to adopt that because even Intel wasn't sure what to offer. And if AMD doesn't pick that up, the usefulness of AVX-512 is in doubt. E.g. in the traditional way of prepping binary releases (many distribution methods still do) is to target the "highest common factor" of supported instructions of reasonably recent CPUs, meaning they would not target AVX-512 at all as the latest CPU from AMD doesn't have it (they later add fake AVX-512 capability.) Now it means that whether AVX-512 is useful to you as an end users then depends on the distributions a lot (and in practice none of the general purpose binaries are targeting this. In the rare case they build using the Intel compilers, it is very easy to ship both, which is not true if using GNU compilers.) And if it ain't going to useful for the majority of users, there's no hurry to upgrade specially for AVX-512. Then it becomes a disadvantage for Intel to increase the cost of making AVX-512 capable CPUs to compute in the lower ends. Note that there are other comments and articles on this already—AVX-512 is not cheap. It costs significant die area which is cost. That further complicates the things when Intel is introducing BIG.little design, where in the little CPU, low energy draw and low cost hence smaller die size drives towards having no AVX-512. And since the only way to guarantee any binaries can jump between the BIG and little CPU, they choose to disable AVX-512 on the BIG CPU even if its there. (AnandTech article is much more detailed about this.) P.S. I'm sure historians would give a better and more balanced accounts on why it fails, but I think it is safe to say the standard is now too fragmented, not necessarily in implementation sans Xeon Phi but in the market, it is probably remains to be a niche. P.S.2 A fun fact is that in the recent years AMD's CPU are praised by their performance to watt ratio because of the advances in fabrication tech (basically from TSMC's advances). But I did some calculations earlier that if we're only talking about FLOPS, and compares FLOPS per watt with the consideration of AVX-512 units (many Intel CPUs even have 2 AVX-512 units), it is actually comparable if not lower for Intel. (That was an old calculations and with the newest Intel getting on better fab. I'm sure the story has changed already.) But then no much people if any (I heard none, but I don't read too much except sometimes from AnandTech) has mentioned this and make the comparison "fair" to Intel. Why? Because the end users most of the time aren't going to be benefited by AVX-512 (again because of distributions) so that's often not taken into accounts, sometimes indirectly by performing benchmarks that is not AVX-512 aware.

janwas 5y ago
Oh, you are using Phi :) My viewpoint is likewise biased towards desktop and server systems. Fair point about heterogeneous cores. I suspect those aren't going away. Perhaps the industry should move to a model where threads are temporarily pinned to their core while running SIMD kernels, and re-checking the current CPU capabilities rather than storing the result. That would allow using both AVX2 and AVX-512 depending on which core is active, and doesn't require OS changes (except to OSX which AFAIK still cannot reliably pin threads), right? I totally agree that AVX-512 is helpful for power efficiency. Without it, I struggle to see how Intel/AMD are going to compete with current/future ARM and RISC-V.