5 ms·

So Linus, and this guy is saying AVX512 was a bad idea. "“The state of software out there is really not favoring going larger vectors,” Piednoel said in the vi

by Jonnax 6y ago

So Linus, and this guy is saying AVX512 was a bad idea. "“The state of software out there is really not favoring going larger vectors,” Piednoel said in the video. “In fact, you can see clearly in Cinebench for example—that is not one of my favorite benchmarks, especially for a laptop where it doesn’t make any sense—but you can see that AMD is winning the battle of throughput. It’s because they have more cores and they can afford to have more cores.”" But in other places I've seen a lot of hype for it but without much discussion on actual use cases. With now people saying that it isn't useful except to give intel leadership in benchmarks. Is anyone here actually using it and seeing a benefit?

BeeOnRope 6y ago
AVX-512 is Intel's best vector ISA to date and it is definitely useful beyond benchmarks. In fact, too low penetration of AVX-512 has been a problem, rather than too much: for a long time it has only been available on server chips, not laptop (a small fraction of laptops have gotten it recently) or destkop (outside some low-volume "extreme" parts, which were really just rebadged server parts). It would also be quite unusual that some instructions could be very useful in benchmarks that are based on real-life, heavily used applications, but not in real life. Outside of small, easily gamed benchmark that doesn't seem plausible: if the CPU is good a video encoding benchmark, it will be good at video encoding, with high probability. The main problem with AVX-512 is lack of software exploitation. Unlike with frequency boosts, increased cache sizes, better branch predictors, etc: this speedup doesn't come for free. Either compilers have to use the new instructions, or people have to use them by hand. The former has been very limited because these instructions cause a frequency drop (so-called "license based downclocking"), so compilers have mostly disabled their use by default: otherwise, a single AVX-512 instruction could cause a large impact on surrounding code which doesn't use AVX-512. So by-hand exploitation remains, and penetration has just been too low and the people with the skills to do this are limited.
- FullyFunctional 6y ago
  I'm sure AVX-512 is wonderful and you can easily produce examples where it makes loops go N times faster. That unfortunately is missing the point. Microprocessors implementation is an extremely careful balance and effectively everything is a trade off. The area, power, and complexity of features comes at a non-zero cost to code that doesn't use it. The fact that AVX-512 isn't on every processor AND the performance benefits are FAR from obvious (as the core clocks down when using it) really hinders its adoption, which in turn lowers the value; a downward spiral. Intel is in a fine mess of their own making. I'm no fan of either company and AMD gets perhaps a little too much credit as things would have looked a lot different had 10 nm succeeded on the original schedule, but I bought my first AMD processor since Athlon 64 because it's better.
  - BeeOnRope 6y ago
    I'm not missing the point as I was responding to whether AVX-512 is useful in practice. It is. It is some badly designed extensions where the earlier ones were great: it's arguably better than the earlier ones. Now adoption has been poor and there is a chicken-and-egg effect as you point out, but that's separate from the question of whether AVX-512 is useful in practice. It is. The point about tradeoffs is well taken. It's not like Intel (or any other vendor) is rolling back progress in other areas to add larger vector units: rather they try to progress along all viable axes at once.
    - FullyFunctional 6y ago
      Sorry if I misunderstood your point. > It's not like Intel (or any other vendor) is rolling back progress Performance doesn't stand still so even if they are improving, they might have improved more having spend the AVX-512 on something else. And most definitely, larger vector unit does cost you somewhere else.
    - tremon 6y ago
      To me it looks like you and the GP just have different definitions of "in practice". Your definition of "in practice" seems to exclude the (general) availability of these extensions, whereas the GP's definition specifically includes them: if, as a developer, you can't rely on certain features being available, you're forced to optimize for the non-AVX512 case anyway, and any time spent exclusively on AVX512 is additional cost for a small audience. So, whether the instructions themselves are any good then is of secondary concern because of the other problems of Intel's own making, and you both seem to be in agreement about those.
    - PaulHoule 6y ago
      AVX-512 isn't "bad in practice" but in the context of AVX-128, AVX-256, and AVX-1024 it is a very bad practice because the inconstant availability of the vector unit means that it doesn't get used, or you use a least-common denominator of what worked on Atom 10 years ago. Look on the bright side: if you don't use AVX-512 you have 10% extra heat sink and heat soak on your CPU so you can turbo higher. AVX-512 makes the chip easier to cool.
atq2119 6y ago
AVX-512 really has two parts. It widens vectors to 512 bits, but it also makes the instruction set much more orthogonal and clean. It also adds masking of lanes. These latter parts are what's really useful about AVX-512. The widening of the vectors is just an area tradeoff that depends a lot on the particular workload.
- dman 6y ago
  Unfortunately there is a clock penalty to using AVX-512 and it is only available on a small part of the chips that Intel ships (For the longest time it was only available on Xeons). AMD does not support AVX-512 yet. As a result no one really had the incentive to add support for it outside HPC where people have control of both the workload and the hardware it runs on. To further complicate matters there are multiple variants of AVX-512 and different chip families ship with different subset of the variants. In short Intel could not have done more to discourage use of AVX-512 if they tried.
  - atq2119 6y ago
    Right, that's kind of the point. They should have made the cleaned up and extended instruction set available everywhere at least for 256-bit vectors. That would have made the good, non-trade-off parts that people actually liked pervasive.
  - lagadu 6y ago
    The chip family variants problem can't be overstated, it's ridiculous. For example the laptop I bought last year with a flagship latest gen cpu has far more support than the flagship latest gen desktop cpu that I bought two months ago. For reference: https://twitter.com/InstLatX64/status/1131489307473174528/photo/1 https://twitter.com/InstLatX64/status/1131489307473174528/ph...
  - me_me_me 6y ago
    Its not magic, everything has trade-offs. The bigger and more powerful AVX instruction the more power it needs - more power results in more heat, heat that modern cpus cannot handle (reason why we dont see higher cloked cpus).
glangdale 6y ago
I've used it a fair bit, but it's too exotic to show up on enough machines to really focus on. It's not just a trick to make benchmarks run faster; that's an idiotic conspiracy theory. It's just that the release of AVX-512 happens to coincide with the 10nm disaster. Probably the biggest problem is that new Intel ISAs often are associated with downclocking (this isn't new with AVX-512 - AVX2 had it as well). Usually these problems were short-lived, but with 10nm getting stalled, they've stuck around longer. This is pure poison if like me, you generally write libraries. It's scary to look at some 60% speedup you might get from some neat rewrite for your bit of the task, but then wonder whether you're going to make the rest of the task several percent slower due to a downclock in what might only be 20% of the time. So it's hard to make the call to use AVX-512 in its early stages unless you control the whole core. Linus thinks that everything is either a kernel or a gcc execution, and happily disregards anything that doesn't fit his own workloads. AVX-512 can do Good Things for a disparate range of stuff - in-memory databases, regex/string matching, bioinformatics, CPU intensive network workloads, parsing and lexing as well as the usual scientific computing and AI workloads (many of the latter are also pretty good on GPGPU).
me_me_me 6y ago
> and this guy is saying AVX512 was a bad idea. He said that its a bad idea to put it in every xeon chip. AVX is great tool for specific high throughput oncore calculations, when offloading to gpu would be too costly. Saying AVX512 is universally bad is like saying that welding machines are bad tools since carpenters never make use of it.

by Jonnax 6y ago

BeeOnRope 6y ago

AVX-512 is Intel's best vector ISA to date and it is definitely useful beyond benchmarks. In fact, too low penetration of AVX-512 has been a problem, rather than too much: for a long time it has only been available on server chips, not laptop (a small fraction of laptops have gotten it recently) or destkop (outside some low-volume "extreme" parts, which were really just rebadged server parts). It would also be quite unusual that some instructions could be very useful in benchmarks that are based on real-life, heavily used applications, but not in real life. Outside of small, easily gamed benchmark that doesn't seem plausible: if the CPU is good a video encoding benchmark, it will be good at video encoding, with high probability. The main problem with AVX-512 is lack of software exploitation. Unlike with frequency boosts, increased cache sizes, better branch predictors, etc: this speedup doesn't come for free. Either compilers have to use the new instructions, or people have to use them by hand. The former has been very limited because these instructions cause a frequency drop (so-called "license based downclocking"), so compilers have mostly disabled their use by default: otherwise, a single AVX-512 instruction could cause a large impact on surrounding code which doesn't use AVX-512. So by-hand exploitation remains, and penetration has just been too low and the people with the skills to do this are limited.

FullyFunctional 6y ago
I'm sure AVX-512 is wonderful and you can easily produce examples where it makes loops go N times faster. That unfortunately is missing the point. Microprocessors implementation is an extremely careful balance and effectively everything is a trade off. The area, power, and complexity of features comes at a non-zero cost to code that doesn't use it. The fact that AVX-512 isn't on every processor AND the performance benefits are FAR from obvious (as the core clocks down when using it) really hinders its adoption, which in turn lowers the value; a downward spiral. Intel is in a fine mess of their own making. I'm no fan of either company and AMD gets perhaps a little too much credit as things would have looked a lot different had 10 nm succeeded on the original schedule, but I bought my first AMD processor since Athlon 64 because it's better.
- BeeOnRope 6y ago
  I'm not missing the point as I was responding to whether AVX-512 is useful in practice. It is. It is some badly designed extensions where the earlier ones were great: it's arguably better than the earlier ones. Now adoption has been poor and there is a chicken-and-egg effect as you point out, but that's separate from the question of whether AVX-512 is useful in practice. It is. The point about tradeoffs is well taken. It's not like Intel (or any other vendor) is rolling back progress in other areas to add larger vector units: rather they try to progress along all viable axes at once.
  - FullyFunctional 6y ago
    Sorry if I misunderstood your point. > It's not like Intel (or any other vendor) is rolling back progress Performance doesn't stand still so even if they are improving, they might have improved more having spend the AVX-512 on something else. And most definitely, larger vector unit does cost you somewhere else.
  - tremon 6y ago
    To me it looks like you and the GP just have different definitions of "in practice". Your definition of "in practice" seems to exclude the (general) availability of these extensions, whereas the GP's definition specifically includes them: if, as a developer, you can't rely on certain features being available, you're forced to optimize for the non-AVX512 case anyway, and any time spent exclusively on AVX512 is additional cost for a small audience. So, whether the instructions themselves are any good then is of secondary concern because of the other problems of Intel's own making, and you both seem to be in agreement about those.
  - PaulHoule 6y ago
    AVX-512 isn't "bad in practice" but in the context of AVX-128, AVX-256, and AVX-1024 it is a very bad practice because the inconstant availability of the vector unit means that it doesn't get used, or you use a least-common denominator of what worked on Atom 10 years ago. Look on the bright side: if you don't use AVX-512 you have 10% extra heat sink and heat soak on your CPU so you can turbo higher. AVX-512 makes the chip easier to cool.

atq2119 6y ago

AVX-512 really has two parts. It widens vectors to 512 bits, but it also makes the instruction set much more orthogonal and clean. It also adds masking of lanes. These latter parts are what's really useful about AVX-512. The widening of the vectors is just an area tradeoff that depends a lot on the particular workload.

dman 6y ago
Unfortunately there is a clock penalty to using AVX-512 and it is only available on a small part of the chips that Intel ships (For the longest time it was only available on Xeons). AMD does not support AVX-512 yet. As a result no one really had the incentive to add support for it outside HPC where people have control of both the workload and the hardware it runs on. To further complicate matters there are multiple variants of AVX-512 and different chip families ship with different subset of the variants. In short Intel could not have done more to discourage use of AVX-512 if they tried.
- atq2119 6y ago
  Right, that's kind of the point. They should have made the cleaned up and extended instruction set available everywhere at least for 256-bit vectors. That would have made the good, non-trade-off parts that people actually liked pervasive.
- lagadu 6y ago
  The chip family variants problem can't be overstated, it's ridiculous. For example the laptop I bought last year with a flagship latest gen cpu has far more support than the flagship latest gen desktop cpu that I bought two months ago. For reference: https://twitter.com/InstLatX64/status/1131489307473174528/photo/1 https://twitter.com/InstLatX64/status/1131489307473174528/ph...
- me_me_me 6y ago
  Its not magic, everything has trade-offs. The bigger and more powerful AVX instruction the more power it needs - more power results in more heat, heat that modern cpus cannot handle (reason why we dont see higher cloked cpus).

glangdale 6y ago

I've used it a fair bit, but it's too exotic to show up on enough machines to really focus on. It's not just a trick to make benchmarks run faster; that's an idiotic conspiracy theory. It's just that the release of AVX-512 happens to coincide with the 10nm disaster. Probably the biggest problem is that new Intel ISAs often are associated with downclocking (this isn't new with AVX-512 - AVX2 had it as well). Usually these problems were short-lived, but with 10nm getting stalled, they've stuck around longer. This is pure poison if like me, you generally write libraries. It's scary to look at some 60% speedup you might get from some neat rewrite for your bit of the task, but then wonder whether you're going to make the rest of the task several percent slower due to a downclock in what might only be 20% of the time. So it's hard to make the call to use AVX-512 in its early stages unless you control the whole core. Linus thinks that everything is either a kernel or a gcc execution, and happily disregards anything that doesn't fit his own workloads. AVX-512 can do Good Things for a disparate range of stuff - in-memory databases, regex/string matching, bioinformatics, CPU intensive network workloads, parsing and lexing as well as the usual scientific computing and AI workloads (many of the latter are also pretty good on GPGPU).

me_me_me 6y ago

> and this guy is saying AVX512 was a bad idea. He said that its a bad idea to put it in every xeon chip. AVX is great tool for specific high throughput oncore calculations, when offloading to gpu would be too costly. Saying AVX512 is universally bad is like saying that welding machines are bad tools since carpenters never make use of it.