13 ms·

The least interesting part about AVX-512 is the 512 bits vector width

192 points by luu 3y ago

bombela 3y ago
Summary: AVX-512 adds many instructions that can replace what used to take 3 less efficient instructions. This instruction set also double the number of available SIMD (single instruction multiple data) registers. Those instructions are very useful on 128 bits vectors. And not a lot of people actually need 512 bit vectors. Because of the number of registers and 512 bits width, it takes a lot of space in silicon. This makes it costly, and so is reserved for more expensive CPUs. Had it been limited to, or also offered in a 256 bits version, this instruction set would have most likely be included in many more CPUs. Making it much more useful.
- mtklein 3y ago
  IIRC what you’re describing here is indeed what shipped. All the AVX-512 instructions are available for 128-bit, 256-bit and 512-bit registers (xmm, ymm, zmm). If not strictly all, essentially all?
  - moonchild 3y ago
    Base avx512, on the first phis, did in fact only work for 512-bit registers (it also used a slightly different encoding). A later extension is avx512vl (included on all 'normal' cpus implementing avx512), which adds support for the instructions on smaller vector sizes. But there is no standard mode which allows the hardware to support the instructions only for 256- or 128-bit registers; they must be supported at least for 512-bit registers or else not at all.
    - mtklein 3y ago
      Oh! Yeah! I do forget about “base” avx-512 sometimes. No one but HPC folks really ever had any proximity to that. In practice if you’ve got a computer that supports avx-512, it supports VL.
    - namibj 3y ago
      You can microcode them on top of high density SRAM if need be.
    - thechao 3y ago
      The Phi’s inherited that from Larrabee: the processor simply didn’t have any 128/256 or even “real” x87 (fpu). It did have a lot more FMA variants, including an instruction that did “single cycle next step rasterization” (faddsets).
  - saagarjha 3y ago
    Right, but the issue is you can't "just" offer AVX-512 with a 256-bit vector length. You have to also offer the 512-bit options too, which has costs that your processor vendor may not be willing to pay. So you end up only getting AVX2 support.
    - borissk 3y ago
      Hmmm, there are only two desktop processor vendors. Is AVX-512 at all available on mobile? From the two desktop vendors AMD has AVX-512 support on all their AM5 CPUs. Intel has support of AVX-512 on all 11th gen CPUs and on some 12th gen CPUs. The supports is there in the silicon on all P-cores in 12/13th gen CPUs, just disabled in microcode. So AMD and Intel have already paid the cost.
      - ben-schaaf 3y ago
        AMD - until AM5 - has not been willing to pay the cost. Intel was not willing to pay the cost for their E-cores. AVX-512 is almost 10 years old at this point and because of adoption issues still can't be relied upon.
      - adrian_b 3y ago
        Previously Intel had AVX-512 support in 3 generations of mobile CPUs, Cannon Lake U, Ice Lake U, Tiger Lake H/U, but only the last generation had widespread availability. Starting with Alder Lake, Intel has dropped the AVX-512 support in non-server CPUs. On the other hand, AMD has just launched their Phoenix mobile CPUs (Ryzen x 7x40 HS or U), which have excellent AVX-512 support.
- reitzensteinm 3y ago
  The physical register file is much larger than the logical register count. A budget option could simply reduce the amount of renaming done to save space. In Intel's case, Cannon Lake did have AVX 512, but was blocked from being a mainstream part due to 10nm yields. And then their rushed efficiency core strategy effectively disabled AVX-512 just as they were getting back on track. I don't think there's an intrinsic reason you couldn't have efficiency cores run AVX-512 albeit slowly and expect we'll see just that.
  - phh 3y ago
    > And then their rushed efficiency core strategy effectively disabled AVX-512 just as they were getting back on track. I partly blame Linux for that. I remember asking at Kernel Recipes about supporting truly heterogenous multi - processor systems and got shrugged "don't buy broken hardware". Back then, it was for a Broadcom home gateway product, which has an asymmetrical dual core, one with a FPU, the other without. Since then we have seen many examples of such assymetry: most HMP smartphones have asymmetrical instruction set. Mono (and probably all JIT VMs) hit issues of varying cache length so the perfect abstraction is already gone. And now we have Intel E vs P. This is a rather hard problem, I won't pretend otherwise but the amount of dead silicon, and lost power efficiency accumulates significantly.
    - the8472 3y ago
      At least on x86 the CPUID instruction is part of the problem. Userspace can do feature-detection and then rely on that. But if it's inconsistent between cores then thread migration would cause illegal instruction faults. If the kernel tried to fix that by moving such faulting threads to P-cores that would lead to a memcpy routine with AVX512 instructions cause all threads to be moved off E-cores. So first intel would have to introduce new CPUID semantics to indicate that e.g. AVX512 is not supported by default and then a separate flag indicating that it's specifically supported on this core and then userspace would have to pin the thread if it wants to use them or stick to the default set if it wants to be migratable.
      - 3y ago
- Zardoz84 3y ago
  > Because of the number of registers and 512 bits width, it takes a lot of space in silicon. This makes it costly, and so is reserved for more expensive CPUs. > Had it been limited to, or also offered in a 256 bits version, this instruction set would have most likely be included in many more CPUs. Making it much more useful. Someone it's blatantly ignoring AMD CPUs...
  - brrrrrm 3y ago
    AMD supports the 256 wide subset of the AVX-512 ISA?
    - MobiusHorizons 3y ago
      When amd launched AVX-512 support, it was present on all cores of that generation. I think it was zen 3, but I might be misremembering. If I recall correctly it was using 256 bit vectors internally, so full 512 bit operations were slower than intel.
    - KolenCh 3y ago
      It takes a double pumped approach to use 2 cycle with 256bit wide to finish a AVX 512 instruction. So it sort of does.
  - bombela 3y ago
    I merely summarized the article, don't shoot the messenger :)
- Aardwolf 3y ago
  Or use double pumping like AMD does, seems like the solution for "efficiency cores" to me
celeritascelery 3y ago
Fascinating. I wonder why intel never released “AVX256”. Was it to drive adoption of their new extra wide SIMD hardware? Do the extra instructions add a lot of complexity outside of just the increased register size? Either way I recently had to write a SIMD implementation in both SSE (Intel) and Neon (Arm). This was my first time writing SIMD. I found the neon instruction set much more intuitive and complete than SSE. There are all these weird limitations in SSE (such as trying to do a reduction sum across a vector or shifting across vectors) that made it feel incomplete. Never had a chance to try out AVX.
- jsheard 3y ago
  That was around the time when Intel was struggling to get 10nm out of the door, when they were designing AVX512 it might have seemed that the initial implementation on 14nm was just a stop-gap and it would be more practical to implement such wide units on the next process that's just around the corner, but little did they know they would end up running in circles rehashing Skylake/14nm for the next six years while they waited for 10nm to finally come online. If they had known things would go that way, perhaps they would have done "AVX256".
  - moonchild 3y ago
    Indeed, and to add on to this, I suspect that they had their own plans about where to take the client space in general, which involved more avx512 (which a couple of generations prior to alderlake supported!), but depended on their having a near-monopoly. The competitiveness of zen forced them to pivot, and avx512 fell to the wayside, largely because of its lack of users (which in turn is in large part because no one really cares about performance on clients).
  - dopa42365 3y ago
    >10nm aka 10nm Enhanced SuperFin aka Intel 7 (12000 and 13000 series) Which funnily enough don't support AVX512, unlike the previous 10000 (14nm++) and 11000 (14nm+++) series.
    - 3y ago
crest 3y ago
Didn't some of the DEC Alpha and IBM Power designs use a cached register-file for lack of a better terminology. Allowing them to support more architectural state than the wide multi-ported register file could fit? Would a small double/quad pumped AVX-512 implementation with a dual ported register file + caching/queuing the recently produced results at the functional units that produced them safe die size and still allow useful throughput?
dan-robertson 3y ago
I never thought that just the registers for AVX-512 were so large compared to L1. (Though some newer chips have 48kb instead of 32). But I think I’m mostly surprised because I hadn’t thought of the size of the register file (which is comparable) when considering renaming. Some interesting replies too, eg https://mastodon.gamedev.place/@TomF/110572967731705754 https://mastodon.gamedev.place/@TomF/110572967731705754 A story I would have believed was that the instruction set was designed with some useful seeming instructions and a hope that compilers would improve. But it sounds like it was designed much more closely with actual example programs and a compiler, just not the kind that attempts to vectorise scalar code.
inopinatus 3y ago
The masked variants of most operations are a killer AVX-512 feature for me. Vectorised conditional execution was/is the last piece of the puzzle. It baffles me that clang in particular disregards them. Clang’s intrinsics and builtins generally use the unmasked forms and fake it with subsequent combining operations. This always benchmarks slower in loops, and often demands an extra register. I haven’t delved deeply, but it feels like either the cost model is mispredicting potential k-register bottleneck, or it doesn’t know about masked AVX-512 instructions at all. In comparison, GCC does, but it falls down (on my code at least) in needing more explicit vectorisation than clang.
- brigade 3y ago
  > This always benchmarks slower in loops Really? It's a forced read-write dependency on the destination register. Which makes sense for cores with limited superscalar. But for ops with >1 cycle latency or >1/cycle throughput, chained masks are likely to inhibit ILP and be slower...
  - sparkie 3y ago
    A big benefit comes from having branchless code. For example, if you have an if/else statement where the consequent acts on some elements of the vector and the antecedent acts on the others, you can perform them all with no branch, by taking the mask resulting from the condition for the consequent instructions, then complementing the mask and applying the antecedent to the same registers. This can also have predictable performance, because all instructions from the consequent and antecedent are executed each time, and there are no branch prediction misses to worry about. It's very useful for timing sensitive code (cryptography), and situations where you want a measurable WCET.
    - brigade 3y ago
      Masked instructions vs subsequent merging are both branchless and have no implicit data-dependent timing relative to each other.
    - inopinatus 3y ago
bee_rider 3y ago
> This, combined with the 512b vectors that are the "-512" part, quadruples the amount of architectural FP/SIMD state, which is one of the main reasons we _don't_ get any AVX-512 in the "small" cores. That’s an interesting point. Does anyone know—the Knights Landing Phi had AVX-512 and was based on Atom cores. Did they bolt on all these extra registers?
- moonchild 3y ago
  Yes. Phi was a massive vector processor bolted onto a tiny scalar processor. This seems to have worked pretty well, for what it was. But it wouldn't be suitable for the applications where intel is using gracemont, because client workloads tend not to use avx512.
- Dylan16807 3y ago
  Note that quadrupling the architectural state doesn't mean quadrupling the actual state. In fact, it looks like Haswell and Skylake-X had the same number of physical registers, 168. So that's a straightforward doubling from 256x168 to 512x168. But further into the thread it looks like the first gen E cores had about 200 128-bit register lines, so trying to fit 512x32 would have been very tight. To put some of that a different way: The vector design headed for E cores was 128 bits stretching to 256 bits. If it had been 256 bits all the way through, it's likely they would have added AVX-512 support, even if they couldn't increase the size of the register file at all.
- namibj 3y ago
  They actually separated architectural registers from physical ones, due to being SMT4. There's a chips and cheese article on this.
- detaro 3y ago
  All variants of Phi were quite small/simple cores (I think the early ones were derivatives of early Pentium designs(!), just lots of them in more modern processes) with massive vector units strapped to the side. Which was fine for that purpose, since they really only were intended to feed the vector units and not expected to be any good at general-purpose computing tasks.
gnu8 3y ago
> * VPTERNLOGD (your swiss army railgun for bitwise logic, can often fuse 2 or even 3 ops into one) I wonder if this would be useful for implementing cryptographic algorithms.
- sparkie 3y ago
  I doubt it would be much use as cryptographic operations tend to mainly use xor on two inputs. VPTERNLOGD basically works by constructing a truth table for 3 inputs. | A | B | C | R | 0 | 0 | 0 | x | 0 | 0 | 1 | x | 0 | 1 | 0 | x | 0 | 1 | 1 | x | 1 | 0 | 0 | x | 1 | 0 | 1 | x | 1 | 1 | 0 | x | 1 | 1 | 1 | x You pick the values you want for R, then pass this 8-bit value as the operand to the instruction along with the 3 values. For example, A ∧ B ∧ C would be 0b10000000. A ∧ ¬B ∧ ¬C would be 0xb00010000 There are 256 such tables and many of them can be represented by multiple boolean expressions.
  - marssaxman 3y ago
    That is delightful. Thanks for explaining it so clearly.
  - addaon 3y ago
    This is basically the same way as the basic primitive of an FPGA, a LUT (look-up table) works. It's a small ROM or RAM of size 2^N x 1, that is looked up by the N-bit "address" of the inputs. Modern FPGAs tend to use N=4 to N=6, with some additional fanciness occasionally present to make the up-to-64-bits of ROM/RAM useful in other ways as well.
  - mmozeiko 3y ago
    Here's a fancy trick from LLVM source: https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp#L4397-L4401 https://github.com/llvm/llvm-project/blob/main/llvm/lib/Targ... #define A 0xf0 #define B 0xcc #define C 0xaa And then you can build immediate for VPTERNLOG operation by writing bitwise expression with A/B/C values in source code. For example, A^B^C=150. A^(~B&C)=210. And so on... Also mentioned by Fabian here: https://twitter.com/rygorous/status/1187032693944410114 https://twitter.com/rygorous/status/1187032693944410114
mattst88 3y ago
I am a happy owner of a Tigerlake (Intel 11th Gen) Framework laptop. I've considered upgrading to a 12th or 13th Gen motherboard, and while I have no doubt they'd be great for me as a Gentoo developer with the greatly increased core counts, my hesitation is that the new CPUs have AVX-512 disabled. Maybe this doesn't matter, almost certainly wouldn't for most people, but I'm compiling the whole system myself so the compiler at least has the freedom to use AVX-512 wherever it pleases. Does anyone know if AVX-512 actually makes a difference in workloads that aren't specifically tuned for it? My guess is that given news like https://www.phoronix.com/news/GCC-AVX-512-Fully-Masked-Vector https://www.phoronix.com/news/GCC-AVX-512-Fully-Masked-Vecto... that compilers basically don't do anything interesting with AVX-512 without hand-written code.
- mtklein 3y ago
  The promise of the AVX-512 instruction set really was that it would be much easier to (auto-)vectorize code that wasn’t written with vectorization in mind, with tools like masked execution and gather/scatter that either didn’t exist at all before (SSE) or were very minimal (AVX). The tools are there in the instruction set, but that still leaves the issues of time and effort to implement in compilers, and enough performance improvement on enough machines in some market (browsers, games, etc) capable of running it all before any of this possibility becomes real. The skylake-xeon/icelake false start here really can’t have helped. It’s still a much more pragmatic thing to target the haswell feature set that all the intel chips and most amd chips can run (and run well).
- causality0 3y ago
  Does anyone know if AVX-512 actually makes a difference in workloads that aren't specifically tuned for it? I know game console emulators use it to great effect with significant performance increases.
  - jsheard 3y ago
    Incidentally that's another case where the 512bit-ness is the least interesting part, the new instructions are useful for efficiently emulating ARM NEON (Switch) and Cell SPU (Playstation 3) code but those platforms are themselves only 128bits wide so I don't believe the emulators have any use for the 512bit (or even 256bit?) variants of the AVX512 instructions.
lifthrasiir 3y ago
AVX-512F also introduced an embedded rounding and exception control in the instruction itself, which was a great pain for everyone doing accurate maths (e.g. interval arithmetic). It's a great shame that Intel made all good bits of AVX-512 a hostage to the less important 512-bit vector width.
ridiculous_fish 3y ago
VPTERNLOGD is a mouthful but is fun to think about. Notice there are 4 functions mapping 1 bit -> 1 bit: const 0, const 1, copy, invert. There are 16 functions mapping 2 bits -> 1 bit. Think 4 possible inputs, 2 possible output values for each input (0 or 1), so there are 2^4 = 16 such functions. Likewise 256 functions mapping 3 bits -> 1 bit: 8 possible inputs, 2 possible output values for each input, so 2^8 = 256 such functions. This means that the set of functions 3 bits -> 1 bit may be indexed by a single byte! With this instruction, you specify the byte as an immediate, it is interpreted as an index into a function table, and so you get any 3-valued boolean function. I wonder if we'll see a similar instruction for 2 bits -> 2 bits? Could be useful!
- moonchild 3y ago
  We've seen (+, and) multiply (that's just standard multiplication) and (xor, and) multiply ('carryless multiply'); it would be interesting to see a carryless multiply where both combining functions are user-controllable. (See also: <https://twitter.com/moon_chilled/status/1639829366304821249 https://twitter.com/moon_chilled/status/1639829366304821249>; I forget why this works, though.)
  - RaisingSpear 3y ago
    You might find this post interesting: https://bitmath.blogspot.com/2023/05/grevmul.html https://bitmath.blogspot.com/2023/05/grevmul.html
- RaisingSpear 3y ago
  > I wonder if we'll see a similar instruction for 2 bits -> 2 bits? Could be useful! You don't even need to support all 16 functions, as a number are duplicates or could be implemented with others. The following list covers all possibilities: - AND, OR, XOR [supported basically everywhere] - AND-NOT (a&~b), OR-NOT (a|~b) [sometimes supported] - NOT-AND (~(a&b)), NOT-OR (~(a|b)), NOT-XOR (aka XNOR) So only a few basic instructions need to exist to support all combinations of 2-operand bitwise logic.
  - 3y ago
rwmj 3y ago
It's a little annoying that valgrind doesn't support AVX-512 still (since patches were first posted in 2017). We like to test our binaries using valgrind, so need to compile everything with -mno-avx512f. https://bugs.kde.org/show_bug.cgi?id=383010 https://bugs.kde.org/show_bug.cgi?id=383010
hayesti 3y ago
An often overlooked gem of AVX-512 is its (optional) conflict detection instructions. Using these enables vectorizing gather-modify-scatter operations that may alias at the element/lane level. Algorithms that have irregular data-level parallelism (DLP) as opposed to perfect or embarrassingly parallel DLP can now take advantage of the SIMD registers/FUs. https://upcommons.upc.edu/bitstream/handle/2117/77204/VSR%20sort%20_post.pdf?sequence=8 https://upcommons.upc.edu/bitstream/handle/2117/77204/VSR%20... shows how radix sort can be vectorized much more efficiently using these types of instructions.
ketaminisiert 3y ago
This is my first time opening a mastodon link and wow. It's not asking me to sign in, to download the app, just straight to the point. I like it!
Toaster-King 3y ago
See also: Why Is AVX 512 Useful for RPCS3?[1] Many of the instructions mentioned by Fabian are used to great effect when emulating the PS3's SPUs. [1]: https://whatcookie.github.io/posts/why-is-avx-512-useful-for-rpcs3/ https://whatcookie.github.io/posts/why-is-avx-512-useful-for...

192 points by luu 3y ago

Summary: AVX-512 adds many instructions that can replace what used to take 3 less efficient instructions. This instruction set also double the number of available SIMD (single instruction multiple data) registers. Those instructions are very useful on 128 bits vectors. And not a lot of people actually need 512 bit vectors. Because of the number of registers and 512 bits width, it takes a lot of space in silicon. This makes it costly, and so is reserved for more expensive CPUs. Had it been limited to, or also offered in a 256 bits version, this instruction set would have most likely be included in many more CPUs. Making it much more useful.

mtklein 3y ago
IIRC what you’re describing here is indeed what shipped. All the AVX-512 instructions are available for 128-bit, 256-bit and 512-bit registers (xmm, ymm, zmm). If not strictly all, essentially all?
- moonchild 3y ago
  Base avx512, on the first phis, did in fact only work for 512-bit registers (it also used a slightly different encoding). A later extension is avx512vl (included on all 'normal' cpus implementing avx512), which adds support for the instructions on smaller vector sizes. But there is no standard mode which allows the hardware to support the instructions only for 256- or 128-bit registers; they must be supported at least for 512-bit registers or else not at all.
  - mtklein 3y ago
    Oh! Yeah! I do forget about “base” avx-512 sometimes. No one but HPC folks really ever had any proximity to that. In practice if you’ve got a computer that supports avx-512, it supports VL.
  - namibj 3y ago
    You can microcode them on top of high density SRAM if need be.
  - thechao 3y ago
    The Phi’s inherited that from Larrabee: the processor simply didn’t have any 128/256 or even “real” x87 (fpu). It did have a lot more FMA variants, including an instruction that did “single cycle next step rasterization” (faddsets).
- saagarjha 3y ago
  Right, but the issue is you can't "just" offer AVX-512 with a 256-bit vector length. You have to also offer the 512-bit options too, which has costs that your processor vendor may not be willing to pay. So you end up only getting AVX2 support.
  - borissk 3y ago
    Hmmm, there are only two desktop processor vendors. Is AVX-512 at all available on mobile? From the two desktop vendors AMD has AVX-512 support on all their AM5 CPUs. Intel has support of AVX-512 on all 11th gen CPUs and on some 12th gen CPUs. The supports is there in the silicon on all P-cores in 12/13th gen CPUs, just disabled in microcode. So AMD and Intel have already paid the cost.
    - ben-schaaf 3y ago
      AMD - until AM5 - has not been willing to pay the cost. Intel was not willing to pay the cost for their E-cores. AVX-512 is almost 10 years old at this point and because of adoption issues still can't be relied upon.
    - adrian_b 3y ago
      Previously Intel had AVX-512 support in 3 generations of mobile CPUs, Cannon Lake U, Ice Lake U, Tiger Lake H/U, but only the last generation had widespread availability. Starting with Alder Lake, Intel has dropped the AVX-512 support in non-server CPUs. On the other hand, AMD has just launched their Phoenix mobile CPUs (Ryzen x 7x40 HS or U), which have excellent AVX-512 support.
reitzensteinm 3y ago
The physical register file is much larger than the logical register count. A budget option could simply reduce the amount of renaming done to save space. In Intel's case, Cannon Lake did have AVX 512, but was blocked from being a mainstream part due to 10nm yields. And then their rushed efficiency core strategy effectively disabled AVX-512 just as they were getting back on track. I don't think there's an intrinsic reason you couldn't have efficiency cores run AVX-512 albeit slowly and expect we'll see just that.
- phh 3y ago
  > And then their rushed efficiency core strategy effectively disabled AVX-512 just as they were getting back on track. I partly blame Linux for that. I remember asking at Kernel Recipes about supporting truly heterogenous multi - processor systems and got shrugged "don't buy broken hardware". Back then, it was for a Broadcom home gateway product, which has an asymmetrical dual core, one with a FPU, the other without. Since then we have seen many examples of such assymetry: most HMP smartphones have asymmetrical instruction set. Mono (and probably all JIT VMs) hit issues of varying cache length so the perfect abstraction is already gone. And now we have Intel E vs P. This is a rather hard problem, I won't pretend otherwise but the amount of dead silicon, and lost power efficiency accumulates significantly.
  - the8472 3y ago
    At least on x86 the CPUID instruction is part of the problem. Userspace can do feature-detection and then rely on that. But if it's inconsistent between cores then thread migration would cause illegal instruction faults. If the kernel tried to fix that by moving such faulting threads to P-cores that would lead to a memcpy routine with AVX512 instructions cause all threads to be moved off E-cores. So first intel would have to introduce new CPUID semantics to indicate that e.g. AVX512 is not supported by default and then a separate flag indicating that it's specifically supported on this core and then userspace would have to pin the thread if it wants to use them or stick to the default set if it wants to be migratable.
    - 3y ago
Zardoz84 3y ago
> Because of the number of registers and 512 bits width, it takes a lot of space in silicon. This makes it costly, and so is reserved for more expensive CPUs. > Had it been limited to, or also offered in a 256 bits version, this instruction set would have most likely be included in many more CPUs. Making it much more useful. Someone it's blatantly ignoring AMD CPUs...
- brrrrrm 3y ago
  AMD supports the 256 wide subset of the AVX-512 ISA?
  - MobiusHorizons 3y ago
    When amd launched AVX-512 support, it was present on all cores of that generation. I think it was zen 3, but I might be misremembering. If I recall correctly it was using 256 bit vectors internally, so full 512 bit operations were slower than intel.
  - KolenCh 3y ago
    It takes a double pumped approach to use 2 cycle with 256bit wide to finish a AVX 512 instruction. So it sort of does.
- bombela 3y ago
  I merely summarized the article, don't shoot the messenger :)
Aardwolf 3y ago
Or use double pumping like AMD does, seems like the solution for "efficiency cores" to me

celeritascelery 3y ago

Fascinating. I wonder why intel never released “AVX256”. Was it to drive adoption of their new extra wide SIMD hardware? Do the extra instructions add a lot of complexity outside of just the increased register size? Either way I recently had to write a SIMD implementation in both SSE (Intel) and Neon (Arm). This was my first time writing SIMD. I found the neon instruction set much more intuitive and complete than SSE. There are all these weird limitations in SSE (such as trying to do a reduction sum across a vector or shifting across vectors) that made it feel incomplete. Never had a chance to try out AVX.

jsheard 3y ago
That was around the time when Intel was struggling to get 10nm out of the door, when they were designing AVX512 it might have seemed that the initial implementation on 14nm was just a stop-gap and it would be more practical to implement such wide units on the next process that's just around the corner, but little did they know they would end up running in circles rehashing Skylake/14nm for the next six years while they waited for 10nm to finally come online. If they had known things would go that way, perhaps they would have done "AVX256".
- moonchild 3y ago
  Indeed, and to add on to this, I suspect that they had their own plans about where to take the client space in general, which involved more avx512 (which a couple of generations prior to alderlake supported!), but depended on their having a near-monopoly. The competitiveness of zen forced them to pivot, and avx512 fell to the wayside, largely because of its lack of users (which in turn is in large part because no one really cares about performance on clients).
- dopa42365 3y ago
  >10nm aka 10nm Enhanced SuperFin aka Intel 7 (12000 and 13000 series) Which funnily enough don't support AVX512, unlike the previous 10000 (14nm++) and 11000 (14nm+++) series.
  - 3y ago

crest 3y ago

Didn't some of the DEC Alpha and IBM Power designs use a cached register-file for lack of a better terminology. Allowing them to support more architectural state than the wide multi-ported register file could fit? Would a small double/quad pumped AVX-512 implementation with a dual ported register file + caching/queuing the recently produced results at the functional units that produced them safe die size and still allow useful throughput?

dan-robertson 3y ago

I never thought that just the registers for AVX-512 were so large compared to L1. (Though some newer chips have 48kb instead of 32). But I think I’m mostly surprised because I hadn’t thought of the size of the register file (which is comparable) when considering renaming. Some interesting replies too, eg https://mastodon.gamedev.place/@TomF/110572967731705754 https://mastodon.gamedev.place/@TomF/110572967731705754 A story I would have believed was that the instruction set was designed with some useful seeming instructions and a hope that compilers would improve. But it sounds like it was designed much more closely with actual example programs and a compiler, just not the kind that attempts to vectorise scalar code.

inopinatus 3y ago

The masked variants of most operations are a killer AVX-512 feature for me. Vectorised conditional execution was/is the last piece of the puzzle. It baffles me that clang in particular disregards them. Clang’s intrinsics and builtins generally use the unmasked forms and fake it with subsequent combining operations. This always benchmarks slower in loops, and often demands an extra register. I haven’t delved deeply, but it feels like either the cost model is mispredicting potential k-register bottleneck, or it doesn’t know about masked AVX-512 instructions at all. In comparison, GCC does, but it falls down (on my code at least) in needing more explicit vectorisation than clang.

brigade 3y ago
> This always benchmarks slower in loops Really? It's a forced read-write dependency on the destination register. Which makes sense for cores with limited superscalar. But for ops with >1 cycle latency or >1/cycle throughput, chained masks are likely to inhibit ILP and be slower...
- sparkie 3y ago
  A big benefit comes from having branchless code. For example, if you have an if/else statement where the consequent acts on some elements of the vector and the antecedent acts on the others, you can perform them all with no branch, by taking the mask resulting from the condition for the consequent instructions, then complementing the mask and applying the antecedent to the same registers. This can also have predictable performance, because all instructions from the consequent and antecedent are executed each time, and there are no branch prediction misses to worry about. It's very useful for timing sensitive code (cryptography), and situations where you want a measurable WCET.
  - brigade 3y ago
    Masked instructions vs subsequent merging are both branchless and have no implicit data-dependent timing relative to each other.
  - inopinatus 3y ago

bee_rider 3y ago

> This, combined with the 512b vectors that are the "-512" part, quadruples the amount of architectural FP/SIMD state, which is one of the main reasons we _don't_ get any AVX-512 in the "small" cores. That’s an interesting point. Does anyone know—the Knights Landing Phi had AVX-512 and was based on Atom cores. Did they bolt on all these extra registers?

moonchild 3y ago
Yes. Phi was a massive vector processor bolted onto a tiny scalar processor. This seems to have worked pretty well, for what it was. But it wouldn't be suitable for the applications where intel is using gracemont, because client workloads tend not to use avx512.
Dylan16807 3y ago
Note that quadrupling the architectural state doesn't mean quadrupling the actual state. In fact, it looks like Haswell and Skylake-X had the same number of physical registers, 168. So that's a straightforward doubling from 256x168 to 512x168. But further into the thread it looks like the first gen E cores had about 200 128-bit register lines, so trying to fit 512x32 would have been very tight. To put some of that a different way: The vector design headed for E cores was 128 bits stretching to 256 bits. If it had been 256 bits all the way through, it's likely they would have added AVX-512 support, even if they couldn't increase the size of the register file at all.
namibj 3y ago
They actually separated architectural registers from physical ones, due to being SMT4. There's a chips and cheese article on this.
detaro 3y ago
All variants of Phi were quite small/simple cores (I think the early ones were derivatives of early Pentium designs(!), just lots of them in more modern processes) with massive vector units strapped to the side. Which was fine for that purpose, since they really only were intended to feed the vector units and not expected to be any good at general-purpose computing tasks.

gnu8 3y ago

> * VPTERNLOGD (your swiss army railgun for bitwise logic, can often fuse 2 or even 3 ops into one) I wonder if this would be useful for implementing cryptographic algorithms.

sparkie 3y ago
I doubt it would be much use as cryptographic operations tend to mainly use xor on two inputs. VPTERNLOGD basically works by constructing a truth table for 3 inputs. | A | B | C | R | 0 | 0 | 0 | x | 0 | 0 | 1 | x | 0 | 1 | 0 | x | 0 | 1 | 1 | x | 1 | 0 | 0 | x | 1 | 0 | 1 | x | 1 | 1 | 0 | x | 1 | 1 | 1 | x You pick the values you want for R, then pass this 8-bit value as the operand to the instruction along with the 3 values. For example, A ∧ B ∧ C would be 0b10000000. A ∧ ¬B ∧ ¬C would be 0xb00010000 There are 256 such tables and many of them can be represented by multiple boolean expressions.
- marssaxman 3y ago
  That is delightful. Thanks for explaining it so clearly.
- addaon 3y ago
  This is basically the same way as the basic primitive of an FPGA, a LUT (look-up table) works. It's a small ROM or RAM of size 2^N x 1, that is looked up by the N-bit "address" of the inputs. Modern FPGAs tend to use N=4 to N=6, with some additional fanciness occasionally present to make the up-to-64-bits of ROM/RAM useful in other ways as well.
- mmozeiko 3y ago
  Here's a fancy trick from LLVM source: https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp#L4397-L4401 https://github.com/llvm/llvm-project/blob/main/llvm/lib/Targ... #define A 0xf0 #define B 0xcc #define C 0xaa And then you can build immediate for VPTERNLOG operation by writing bitwise expression with A/B/C values in source code. For example, A^B^C=150. A^(~B&C)=210. And so on... Also mentioned by Fabian here: https://twitter.com/rygorous/status/1187032693944410114 https://twitter.com/rygorous/status/1187032693944410114

mattst88 3y ago

I am a happy owner of a Tigerlake (Intel 11th Gen) Framework laptop. I've considered upgrading to a 12th or 13th Gen motherboard, and while I have no doubt they'd be great for me as a Gentoo developer with the greatly increased core counts, my hesitation is that the new CPUs have AVX-512 disabled. Maybe this doesn't matter, almost certainly wouldn't for most people, but I'm compiling the whole system myself so the compiler at least has the freedom to use AVX-512 wherever it pleases. Does anyone know if AVX-512 actually makes a difference in workloads that aren't specifically tuned for it? My guess is that given news like https://www.phoronix.com/news/GCC-AVX-512-Fully-Masked-Vector https://www.phoronix.com/news/GCC-AVX-512-Fully-Masked-Vecto... that compilers basically don't do anything interesting with AVX-512 without hand-written code.

mtklein 3y ago
The promise of the AVX-512 instruction set really was that it would be much easier to (auto-)vectorize code that wasn’t written with vectorization in mind, with tools like masked execution and gather/scatter that either didn’t exist at all before (SSE) or were very minimal (AVX). The tools are there in the instruction set, but that still leaves the issues of time and effort to implement in compilers, and enough performance improvement on enough machines in some market (browsers, games, etc) capable of running it all before any of this possibility becomes real. The skylake-xeon/icelake false start here really can’t have helped. It’s still a much more pragmatic thing to target the haswell feature set that all the intel chips and most amd chips can run (and run well).
causality0 3y ago
Does anyone know if AVX-512 actually makes a difference in workloads that aren't specifically tuned for it? I know game console emulators use it to great effect with significant performance increases.
- jsheard 3y ago
  Incidentally that's another case where the 512bit-ness is the least interesting part, the new instructions are useful for efficiently emulating ARM NEON (Switch) and Cell SPU (Playstation 3) code but those platforms are themselves only 128bits wide so I don't believe the emulators have any use for the 512bit (or even 256bit?) variants of the AVX512 instructions.

lifthrasiir 3y ago

AVX-512F also introduced an embedded rounding and exception control in the instruction itself, which was a great pain for everyone doing accurate maths (e.g. interval arithmetic). It's a great shame that Intel made all good bits of AVX-512 a hostage to the less important 512-bit vector width.

ridiculous_fish 3y ago

VPTERNLOGD is a mouthful but is fun to think about. Notice there are 4 functions mapping 1 bit -> 1 bit: const 0, const 1, copy, invert. There are 16 functions mapping 2 bits -> 1 bit. Think 4 possible inputs, 2 possible output values for each input (0 or 1), so there are 2^4 = 16 such functions. Likewise 256 functions mapping 3 bits -> 1 bit: 8 possible inputs, 2 possible output values for each input, so 2^8 = 256 such functions. This means that the set of functions 3 bits -> 1 bit may be indexed by a single byte! With this instruction, you specify the byte as an immediate, it is interpreted as an index into a function table, and so you get any 3-valued boolean function. I wonder if we'll see a similar instruction for 2 bits -> 2 bits? Could be useful!

moonchild 3y ago
We've seen (+, and) multiply (that's just standard multiplication) and (xor, and) multiply ('carryless multiply'); it would be interesting to see a carryless multiply where both combining functions are user-controllable. (See also: <https://twitter.com/moon_chilled/status/1639829366304821249 https://twitter.com/moon_chilled/status/1639829366304821249>; I forget why this works, though.)
- RaisingSpear 3y ago
  You might find this post interesting: https://bitmath.blogspot.com/2023/05/grevmul.html https://bitmath.blogspot.com/2023/05/grevmul.html
RaisingSpear 3y ago
> I wonder if we'll see a similar instruction for 2 bits -> 2 bits? Could be useful! You don't even need to support all 16 functions, as a number are duplicates or could be implemented with others. The following list covers all possibilities: - AND, OR, XOR [supported basically everywhere] - AND-NOT (a&~b), OR-NOT (a|~b) [sometimes supported] - NOT-AND (~(a&b)), NOT-OR (~(a|b)), NOT-XOR (aka XNOR) So only a few basic instructions need to exist to support all combinations of 2-operand bitwise logic.
- 3y ago

rwmj 3y ago

It's a little annoying that valgrind doesn't support AVX-512 still (since patches were first posted in 2017). We like to test our binaries using valgrind, so need to compile everything with -mno-avx512f. https://bugs.kde.org/show_bug.cgi?id=383010 https://bugs.kde.org/show_bug.cgi?id=383010

hayesti 3y ago

An often overlooked gem of AVX-512 is its (optional) conflict detection instructions. Using these enables vectorizing gather-modify-scatter operations that may alias at the element/lane level. Algorithms that have irregular data-level parallelism (DLP) as opposed to perfect or embarrassingly parallel DLP can now take advantage of the SIMD registers/FUs. https://upcommons.upc.edu/bitstream/handle/2117/77204/VSR%20sort%20_post.pdf?sequence=8 https://upcommons.upc.edu/bitstream/handle/2117/77204/VSR%20... shows how radix sort can be vectorized much more efficiently using these types of instructions.

ketaminisiert 3y ago

This is my first time opening a mastodon link and wow. It's not asking me to sign in, to download the app, just straight to the point. I like it!

Toaster-King 3y ago

See also: Why Is AVX 512 Useful for RPCS3?[1] Many of the instructions mentioned by Fabian are used to great effect when emulating the PS3's SPUs. [1]: https://whatcookie.github.io/posts/why-is-avx-512-useful-for-rpcs3/ https://whatcookie.github.io/posts/why-is-avx-512-useful-for...