6 ms·

Wonder if this piece by Linus about AVX512 is still relevant https://news.ycombinator.com/item?id=23809335 https://news.ycombinator.com/item?id=23809335

stingraycharles 4y ago
It’s not, as long as you’re using a recent CPU architecture. I think the slowdown problem was mostly related to Intel’s first version of AVX512, and the underlying issue has since been addressed (at least to the point that it’s not nearly as much a problem as it used to be). This is also why it’s impressive that this is only AMD’s first attempt: it appears to work really well, where it took Intel multiple attempts to get it working well.
adgjlsfhk1 4y ago
AVX-512 one major problem and a bunch of minor ones. The major problem is that most computers still don't have it. Intel tried to segment their lineup and only put AVX-512 in their high end server CPUs for the first 2 generations that had it, but as a result normal programmers didn't have access to it, compiler devs didn't have access to it, and users didn't have access to it. As a result, most compilers don't do a good job generating AVX-512 code, and most programmers think AVX-512 isn't useful. AVX-512 is great. The new instructions are incredibly useful for a wide variety of applications, but the fragmentation and segmentation by Intel has made it a total mess.
- fwsgonzo 4y ago
  Hopefully it's available in AMD CPUs from now on. I already got it with 7950X and I am looking forward to trying it out!
- ilyt 4y ago
  Eh, intel, reminds me how they arbitrarily fused virtualization on like half of desktop CPUs. But hey this time it literally gave AMD time to catch up...
janwas 4y ago
Before we consider the technical merits, let us note that Linus admits to "irrational hatred" and "bias" on this topic. It's also not clear to me how much experience he has developing and testing AVX-512. As others mentioned, throttling is basically nonexistent on Icelake (and AMD Genoa). It can hurt on Xeon Silver (so let's not use those?) and if you only sporadically use SIMD instructions (again, don't do that). I claim that just about any reasonable code which sustains SIMD instructions over several milliseconds would still be a net win even with throttling. Un-nuanced concerns about throttling are outdated and unhelpful. Perhaps I'll write up a paper on this.
- sprash 4y ago
  The hatred is probably not that "irrational". We live in an era where specialized hardware for specialized problems is required because new manufacturing processes may give us a (seemingly slowing down) increase of transistor budget but not really better switching frequencies. We will have units for matrix multiplication, video codecs, AI cores or full blown GPUs. All those units can only be fast in specialized hardware because of predictable memory access patterns and arranging memory/cache topology accordingly "solving" the problem of low switching frequency with high bandwidth. A general purpose CPU however should specialize on unpredictable memory access. This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks.
  - janwas 4y ago
    The hatred is specifically directed at "FP" which I understand to be floating-point. Makes sense inside an OS kernel but a large majority of HPC would indeed consider this irrational. I understand that dark silicon is helpful, but am not so sure that fixed-function HW is the way to go. Perhaps video _de_coding is the most convincing from your list; codec generations are 5+ years, so enough time to benefit from HW. Encoding, on the other hand, tends not to be impressive unless perhaps there is also a software component. For the rest, programmability and deployability (can we rely on it?) is a major issue. Software has often been the limiting factor. Another big concern is the 'hardware lottery'. The algorithms we develop and get are selected for, and tuned to, the current hardware. Perhaps this gets us 5x energy efficiency vs CPU/SIMD. But by painting ourselves ever further into the corner of dense linear algebra, which is definitely not the way that nature implements intelligence, we are missing out on far larger opportunities. For example: spiking nets or memristors have the potential to be 2 or 3 orders of magnitude better. Or actual sparsity, not the fixed-pattern thing (now that is a prime example of an irrelevent benchmark, because AFAIK algorithms haven't yet been able to use them well). > A general purpose CPU however should specialize on unpredictable memory access. Should it really? I think rather we should avoid such accesses whenever possible, because their energy cost now dwarfs that of computation. > This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks. I have difficulty understanding how a reasonable person can come to such a conclusion. Lemire (the author linked here) has a long series of results showing nice speedups from AVX-512. I personally have seen gains in image compression, string processing, cryptography, linear algebra, integer coding, hash tables, databases, sorting, and compression. [Opinions are my own.]

stingraycharles 4y ago

It’s not, as long as you’re using a recent CPU architecture. I think the slowdown problem was mostly related to Intel’s first version of AVX512, and the underlying issue has since been addressed (at least to the point that it’s not nearly as much a problem as it used to be). This is also why it’s impressive that this is only AMD’s first attempt: it appears to work really well, where it took Intel multiple attempts to get it working well.

adgjlsfhk1 4y ago

AVX-512 one major problem and a bunch of minor ones. The major problem is that most computers still don't have it. Intel tried to segment their lineup and only put AVX-512 in their high end server CPUs for the first 2 generations that had it, but as a result normal programmers didn't have access to it, compiler devs didn't have access to it, and users didn't have access to it. As a result, most compilers don't do a good job generating AVX-512 code, and most programmers think AVX-512 isn't useful. AVX-512 is great. The new instructions are incredibly useful for a wide variety of applications, but the fragmentation and segmentation by Intel has made it a total mess.

fwsgonzo 4y ago
Hopefully it's available in AMD CPUs from now on. I already got it with 7950X and I am looking forward to trying it out!
ilyt 4y ago
Eh, intel, reminds me how they arbitrarily fused virtualization on like half of desktop CPUs. But hey this time it literally gave AMD time to catch up...

janwas 4y ago

Before we consider the technical merits, let us note that Linus admits to "irrational hatred" and "bias" on this topic. It's also not clear to me how much experience he has developing and testing AVX-512. As others mentioned, throttling is basically nonexistent on Icelake (and AMD Genoa). It can hurt on Xeon Silver (so let's not use those?) and if you only sporadically use SIMD instructions (again, don't do that). I claim that just about any reasonable code which sustains SIMD instructions over several milliseconds would still be a net win even with throttling. Un-nuanced concerns about throttling are outdated and unhelpful. Perhaps I'll write up a paper on this.

sprash 4y ago
The hatred is probably not that "irrational". We live in an era where specialized hardware for specialized problems is required because new manufacturing processes may give us a (seemingly slowing down) increase of transistor budget but not really better switching frequencies. We will have units for matrix multiplication, video codecs, AI cores or full blown GPUs. All those units can only be fast in specialized hardware because of predictable memory access patterns and arranging memory/cache topology accordingly "solving" the problem of low switching frequency with high bandwidth. A general purpose CPU however should specialize on unpredictable memory access. This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks.
- janwas 4y ago
  The hatred is specifically directed at "FP" which I understand to be floating-point. Makes sense inside an OS kernel but a large majority of HPC would indeed consider this irrational. I understand that dark silicon is helpful, but am not so sure that fixed-function HW is the way to go. Perhaps video _de_coding is the most convincing from your list; codec generations are 5+ years, so enough time to benefit from HW. Encoding, on the other hand, tends not to be impressive unless perhaps there is also a software component. For the rest, programmability and deployability (can we rely on it?) is a major issue. Software has often been the limiting factor. Another big concern is the 'hardware lottery'. The algorithms we develop and get are selected for, and tuned to, the current hardware. Perhaps this gets us 5x energy efficiency vs CPU/SIMD. But by painting ourselves ever further into the corner of dense linear algebra, which is definitely not the way that nature implements intelligence, we are missing out on far larger opportunities. For example: spiking nets or memristors have the potential to be 2 or 3 orders of magnitude better. Or actual sparsity, not the fixed-pattern thing (now that is a prime example of an irrelevent benchmark, because AFAIK algorithms haven't yet been able to use them well). > A general purpose CPU however should specialize on unpredictable memory access. Should it really? I think rather we should avoid such accesses whenever possible, because their energy cost now dwarfs that of computation. > This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks. I have difficulty understanding how a reasonable person can come to such a conclusion. Lemire (the author linked here) has a long series of results showing nice speedups from AVX-512. I personally have seen gains in image compression, string processing, cryptography, linear algebra, integer coding, hash tables, databases, sorting, and compression. [Opinions are my own.]