4 ms·

Hopefully we'll see AVX-512 in Intel's little cores soon. Centaur's last CPU architecture proves that it is possible to implement the extension without a huge a

by skavi 4y ago

Hopefully we'll see AVX-512 in Intel's little cores soon. Centaur's last CPU architecture proves that it is possible to implement the extension without a huge amount of area [0]. Once that happens, I expect we'll finally consistently see AVX-512 on new Intel processors. The masks really are a huge improvement to the design. AMD should be implementing AVX-512 on their own cores soon as well. Once Armv9 (with SVE2) becomes dominant, we'll pretty much be in a golden age of SIMD. [0]: https://chipsandcheese.com/2022/04/30/examining-centaur-chas-die-and-implementation-goals/ https://chipsandcheese.com/2022/04/30/examining-centaur-chas...

dragontamer 4y ago
> we'll pretty much be in a golden age of SIMD. We already are in the golden age of SIMD. NVidia and AMD GPUs are easier and easier to program through standard interfaces. Intel / AMD are pushing SIMD on a CPU, which is useful for sure, but always is going to be smaller in scope than a dedicated SIMD-processor like A100, 3060, AMD Vega, AMD 6800 xt and the like. SIMD-on-a-CPU is useful because you can perform SIMD over the L1 cache as communication (rather than traversing L1 -> L2 -> L3 -> DDR4 / PCIe -> GPU VRAM -> GPU Registers -> SIMD, and back). But if you have a large-scale operation that can work SIMD, the GPU-traversal absolutely works and is commonly done.
- skavi 4y ago
  Good point. Should have clarified I was referring to CPU SIMD.
  - dragontamer 4y ago
    AVX2 is not as good as AVX512. But AVX2 still has vgather instructions, pshufb, and a few other useful tricks. AVX512 and ARM SVE2 bring the CPU up to parity with maybe 2010s-era GPUs or so (full gather/scatter, more permutation instructions, etc. etc.). But GPUs continued to evolve. Butterfly-shuffles are the generic any-to-any network building block, and are exposed in PTX (NVidia assembly) shfl.bfly, and AMD DPP (data-parallel primitives). Having a richer set of lane-to-lane shuffling (especially ready-to-use butterfly networks) would be best. It really is surprising how many problems require those rich-sets of data-movement instructions, or otherwise benefit from them. NEON and SVE had hard-coded data-movement for specific applications. The general-purpose instruction (pshufb) is kinda like permute/shfl from AMD/NVidia. A backwards-permute IIRC doesn't exist yet on CPU-side. And butterfly networks are the general-purpose solution, capable of implementing any arbitrary data-movement in just log(width) steps. (pshufb / permute instructions would be the full-sized butterfly network, but some cases might be "easier" and faster to execute with only a limited number of butterfly swaps, such as what inevitably comes up in sorting) -------- Still, all of these operations can be implemented in AVX2 (albeit slower / less efficiently). So its not like the "language" of AVX2 / AVX is incomplete... its just missing a few general-purpose instructions that could lead to better performance.
jeffbee 4y ago
Considering that the execution units, register file, etc that support AVX-512 are themselves nearly as large as the entire Gracemont core ... don't hold your breath.
- brigade 4y ago
  You don’t need larger than the 128-bit ALUs or the 207x128-bit register file Gracemont already has to implement AVX-512. It doesn’t make sense on its own with that backend, but for ISA compatibility with a big core it does.
  - Dylan16807 4y ago
    Can the shuffling instructions be reasonably efficient with a small ALU?
    - brigade 4y ago
      Depends on what you consider reasonable. Worst case is 512-bit vpermi2*, which could be implemented with 16x 128-bit vpermi2-like uops, if the needed masking was implicit. Which to me is reasonable for ISA compatibility. (Also considering that having to deal with ISA incompatibility across active cores is not reasonable at all.)
      - moonchild 4y ago
        > having to deal with ISA incompatibility across active cores is not reasonable at all If the OS would give me the tools to deal with it, I would find it completely reasonable to write (eg) both avx2 and avx512 kernels to be run concurrently on the same machine.
  - jeffbee 4y ago
    I'm not sure that users would accept that. You could have a situation where an ifunc is resolved on a fast core with a slightly superior AVX-512 definition, but then the thread migrates to an efficiency core and the AVX-512 definition is dramatically slower than what could have been achieved with AVX2 (e.g. if a microcoded permute was 16x slower).
torginus 4y ago
I'm kinda torn on AVX-512 (and SIMD in general). On one hand, AVX-512 finally introduced a sane programming model with mask registers for branching code, which makes the lives of compilers much easier. On the other hand, the tooling for turning high-level languages into SIMD code is not there yet, ISPC refuses to support ARM, and is still kind of a novelty tool. Additionally, 512-bit wide vectors are just too big - the resulting vector units take up too much die space even on big cores, and the power consumption causes issues causing said dies to downclock. Probably it won't be viable on small cores.
- dr_zoidberg 4y ago
  > Additionally, 512-bit wide vectors are just too big - the resulting vector units take up too much die space even on big cores, and the power consumption causes issues causing said dies to downclock. This is no longer true, citing [0]: > At least, it means we need to adjust our mental model of the frequency related cost of AVX-512 instructions. Rather than the prior-generation verdict of “AVX-512 generally causes significant downclocking”, on these Ice Lake and Rocket Lake client chips we can say that AVX-512 causes insignificant (usually, none at all) license-based downclocking and I expect this to be true on other ICL and RKL client chips as well. And we still have to see AMDs implementation of AVX512 on Zen4 to know what behavior and limits it may have (if any). [0] https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre...

by skavi 4y ago

dragontamer 4y ago

> we'll pretty much be in a golden age of SIMD. We already are in the golden age of SIMD. NVidia and AMD GPUs are easier and easier to program through standard interfaces. Intel / AMD are pushing SIMD on a CPU, which is useful for sure, but always is going to be smaller in scope than a dedicated SIMD-processor like A100, 3060, AMD Vega, AMD 6800 xt and the like. SIMD-on-a-CPU is useful because you can perform SIMD over the L1 cache as communication (rather than traversing L1 -> L2 -> L3 -> DDR4 / PCIe -> GPU VRAM -> GPU Registers -> SIMD, and back). But if you have a large-scale operation that can work SIMD, the GPU-traversal absolutely works and is commonly done.

skavi 4y ago
Good point. Should have clarified I was referring to CPU SIMD.
- dragontamer 4y ago
  AVX2 is not as good as AVX512. But AVX2 still has vgather instructions, pshufb, and a few other useful tricks. AVX512 and ARM SVE2 bring the CPU up to parity with maybe 2010s-era GPUs or so (full gather/scatter, more permutation instructions, etc. etc.). But GPUs continued to evolve. Butterfly-shuffles are the generic any-to-any network building block, and are exposed in PTX (NVidia assembly) shfl.bfly, and AMD DPP (data-parallel primitives). Having a richer set of lane-to-lane shuffling (especially ready-to-use butterfly networks) would be best. It really is surprising how many problems require those rich-sets of data-movement instructions, or otherwise benefit from them. NEON and SVE had hard-coded data-movement for specific applications. The general-purpose instruction (pshufb) is kinda like permute/shfl from AMD/NVidia. A backwards-permute IIRC doesn't exist yet on CPU-side. And butterfly networks are the general-purpose solution, capable of implementing any arbitrary data-movement in just log(width) steps. (pshufb / permute instructions would be the full-sized butterfly network, but some cases might be "easier" and faster to execute with only a limited number of butterfly swaps, such as what inevitably comes up in sorting) -------- Still, all of these operations can be implemented in AVX2 (albeit slower / less efficiently). So its not like the "language" of AVX2 / AVX is incomplete... its just missing a few general-purpose instructions that could lead to better performance.

jeffbee 4y ago

Considering that the execution units, register file, etc that support AVX-512 are themselves nearly as large as the entire Gracemont core ... don't hold your breath.

brigade 4y ago
You don’t need larger than the 128-bit ALUs or the 207x128-bit register file Gracemont already has to implement AVX-512. It doesn’t make sense on its own with that backend, but for ISA compatibility with a big core it does.
- Dylan16807 4y ago
  Can the shuffling instructions be reasonably efficient with a small ALU?
  - brigade 4y ago
    Depends on what you consider reasonable. Worst case is 512-bit vpermi2*, which could be implemented with 16x 128-bit vpermi2-like uops, if the needed masking was implicit. Which to me is reasonable for ISA compatibility. (Also considering that having to deal with ISA incompatibility across active cores is not reasonable at all.)
    - moonchild 4y ago
      > having to deal with ISA incompatibility across active cores is not reasonable at all If the OS would give me the tools to deal with it, I would find it completely reasonable to write (eg) both avx2 and avx512 kernels to be run concurrently on the same machine.
- jeffbee 4y ago
  I'm not sure that users would accept that. You could have a situation where an ifunc is resolved on a fast core with a slightly superior AVX-512 definition, but then the thread migrates to an efficiency core and the AVX-512 definition is dramatically slower than what could have been achieved with AVX2 (e.g. if a microcoded permute was 16x slower).

torginus 4y ago

I'm kinda torn on AVX-512 (and SIMD in general). On one hand, AVX-512 finally introduced a sane programming model with mask registers for branching code, which makes the lives of compilers much easier. On the other hand, the tooling for turning high-level languages into SIMD code is not there yet, ISPC refuses to support ARM, and is still kind of a novelty tool. Additionally, 512-bit wide vectors are just too big - the resulting vector units take up too much die space even on big cores, and the power consumption causes issues causing said dies to downclock. Probably it won't be viable on small cores.

dr_zoidberg 4y ago
> Additionally, 512-bit wide vectors are just too big - the resulting vector units take up too much die space even on big cores, and the power consumption causes issues causing said dies to downclock. This is no longer true, citing [0]: > At least, it means we need to adjust our mental model of the frequency related cost of AVX-512 instructions. Rather than the prior-generation verdict of “AVX-512 generally causes significant downclocking”, on these Ice Lake and Rocket Lake client chips we can say that AVX-512 causes insignificant (usually, none at all) license-based downclocking and I expect this to be true on other ICL and RKL client chips as well. And we still have to see AMDs implementation of AVX512 on Zen4 to know what behavior and limits it may have (if any). [0] https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre...