4 ms·

Worse still is Intels rollout of AVX512 specifically, which started nearly a decade ago but to this day it's still not available across their whole product stac

Worse still is Intels rollout of AVX512 specifically, which started nearly a decade ago but to this day it's still not available across their whole product stack, so the countdown to it becoming ubiquitous hasn't even started yet. They painted themselves into a corner by making 512bit vectors a mandatory feature, which they then decided isn't feasible to support in their small E-cores, so now they're walking it all back with a new "AVX10" spec which is just a redux of AVX512 except 512bit vectors are optional this time. Then we'll have to wait a another decade or so for AVX10 to become baseline, so AVX2 will probably be old enough to drink (in the US) before it's fully phased out.

Vecr 2y ago
I don't think you can phase out AVX2, it's the base of AVX512, because you can't always go 512 wide, and you'd have no backwards compatibility.
- jsheard 2y ago
  I know AVX2 will continue to exist in hardware forever for backwards compatibility, by "fully phased out" I mean the eventual point when software no longer has to maintain a dedicated path for hardware which supports AVX2 but doesn't support AVX10, because all relevent hardware supports AVX10.
- the8472 2y ago
  EVEX prefix can address XMM/YMM/ZMM registers. So you can apply the AVX512 instruction set to 128bit and 256bit registers too.
xeeeeeeeeeeenu 2y ago
While it does seem that AVX10 was mainly designed for consumer CPUs so they could use modern vector instructions without 512-bit vectors, the upcoming Arrow Lake will not have it.[1] I guess we will have to wait for at least one more generation. [1] - According to Intel® Architecture Instruction Set Extensions Programming Reference: https://cdrdv2-public.intel.com/826290/architecture-instruction-set-extensions-programming-reference.pdf https://cdrdv2-public.intel.com/826290/architecture-instruct...
- jsheard 2y ago
  Intel is making Xeons out of E-cores (up to 288 of them on one chip) so I assume those will also be motivating the rollout of AVX10, not just their consumer parts.
  - xeeeeeeeeeeenu 2y ago
    Well, they don't support it either. According to the document I linked, neither the just-released Sierra Forest, nor the planned Clearwater Forest support AVX10.
    - deleted 2y ago
      [deleted]
  - kllrnohj 2y ago
    But surely they could just double pump like AMD does on Zen 4(c) and also on (some?) Zen 5c. It's weird to see an Intel so... Broke? That they are seemingly forced to recycle old architectures endlessly
    - jsheard 2y ago
      I think Intels E-cores are quite a bit smaller than the Zen 4c/5c cores, maybe at that scale it's prohibitive to even double up the register file? That's required even if the logic is double-pumped. AIUI the small Zen cores are mostly the same design as the big ones, just with less cache, silicon layout retuned for density rather than speed, and the removal of the 3D Cache stacking vias, while Intels small cores are clean-sheet designs with next to nothing in common with their big cores so they have to opportunity to shrink them a lot more.
      - suresk 2y ago
        My non-expert brain immediately jumped to double-pumping + maybe working with their thread director to have tasks using a lot of AVX512 instructions prefer P cores more. It feels like such an obvious solution to a really dumb problem that I assumed there was something simple I was missing. The register file size makes sense, I didn't think they were that much of the die on those processors but I guess they had to be pretty aggressive to meet power goals?
janwas 2y ago
I do not understand why ubiquity or baselines should be required in order to use CPU features :) Since many years, performance-critical libraries have used runtime/dynamic dispatch. Our github.com/google/highway intrinsics even automate this. You can write your code once and it is compiled for each instruction set, and the best codepath is selected at runtime.
- Remnant44 2y ago
  Dynamic dispatch adds headaches to the build process; they are surmountable for sure, but in my experience the build wrangling to make it all happen is harder than the original work of rewriting your code with intrinsics! The other major problem I have with dynamic dispatch, at least for the SIMD code I've written, is that you have to do so at a fairly high level of granularity. Most optimized routines are doing as much fusion & cache tiling as possible and so the dispatch has to happen at the level of the higher-level function rather than the more array-op-like components within it. And mostly, that means you've written your (often quite complicated) procedure several times uniquely instead of simply accelerating components within it. I have not used Highway - if it dramatically simplifies the above, that's excellent!
  - janwas 2y ago
    :) Yes indeed, no changes to the build required. Example: https://gcc.godbolt.org/z/KM3ben7ET https://gcc.godbolt.org/z/KM3ben7ET I agree dispatch should be at a reasonably high level. This hasn't been a problem in my experience, we have been able to inline together most SIMD code and dispatch infrequently.

Vecr 2y ago

I don't think you can phase out AVX2, it's the base of AVX512, because you can't always go 512 wide, and you'd have no backwards compatibility.

jsheard 2y ago
I know AVX2 will continue to exist in hardware forever for backwards compatibility, by "fully phased out" I mean the eventual point when software no longer has to maintain a dedicated path for hardware which supports AVX2 but doesn't support AVX10, because all relevent hardware supports AVX10.
the8472 2y ago
EVEX prefix can address XMM/YMM/ZMM registers. So you can apply the AVX512 instruction set to 128bit and 256bit registers too.

xeeeeeeeeeeenu 2y ago

While it does seem that AVX10 was mainly designed for consumer CPUs so they could use modern vector instructions without 512-bit vectors, the upcoming Arrow Lake will not have it.[1] I guess we will have to wait for at least one more generation. [1] - According to Intel® Architecture Instruction Set Extensions Programming Reference: https://cdrdv2-public.intel.com/826290/architecture-instruction-set-extensions-programming-reference.pdf https://cdrdv2-public.intel.com/826290/architecture-instruct...

jsheard 2y ago
Intel is making Xeons out of E-cores (up to 288 of them on one chip) so I assume those will also be motivating the rollout of AVX10, not just their consumer parts.
- xeeeeeeeeeeenu 2y ago
  Well, they don't support it either. According to the document I linked, neither the just-released Sierra Forest, nor the planned Clearwater Forest support AVX10.
  - deleted 2y ago
    [deleted]
- kllrnohj 2y ago
  But surely they could just double pump like AMD does on Zen 4(c) and also on (some?) Zen 5c. It's weird to see an Intel so... Broke? That they are seemingly forced to recycle old architectures endlessly
  - jsheard 2y ago
    I think Intels E-cores are quite a bit smaller than the Zen 4c/5c cores, maybe at that scale it's prohibitive to even double up the register file? That's required even if the logic is double-pumped. AIUI the small Zen cores are mostly the same design as the big ones, just with less cache, silicon layout retuned for density rather than speed, and the removal of the 3D Cache stacking vias, while Intels small cores are clean-sheet designs with next to nothing in common with their big cores so they have to opportunity to shrink them a lot more.
    - suresk 2y ago
      My non-expert brain immediately jumped to double-pumping + maybe working with their thread director to have tasks using a lot of AVX512 instructions prefer P cores more. It feels like such an obvious solution to a really dumb problem that I assumed there was something simple I was missing. The register file size makes sense, I didn't think they were that much of the die on those processors but I guess they had to be pretty aggressive to meet power goals?

janwas 2y ago

I do not understand why ubiquity or baselines should be required in order to use CPU features :) Since many years, performance-critical libraries have used runtime/dynamic dispatch. Our github.com/google/highway intrinsics even automate this. You can write your code once and it is compiled for each instruction set, and the best codepath is selected at runtime.

Remnant44 2y ago
Dynamic dispatch adds headaches to the build process; they are surmountable for sure, but in my experience the build wrangling to make it all happen is harder than the original work of rewriting your code with intrinsics! The other major problem I have with dynamic dispatch, at least for the SIMD code I've written, is that you have to do so at a fairly high level of granularity. Most optimized routines are doing as much fusion & cache tiling as possible and so the dispatch has to happen at the level of the higher-level function rather than the more array-op-like components within it. And mostly, that means you've written your (often quite complicated) procedure several times uniquely instead of simply accelerating components within it. I have not used Highway - if it dramatically simplifies the above, that's excellent!
- janwas 2y ago
  :) Yes indeed, no changes to the build required. Example: https://gcc.godbolt.org/z/KM3ben7ET https://gcc.godbolt.org/z/KM3ben7ET I agree dispatch should be at a reasonably high level. This hasn't been a problem in my experience, we have been able to inline together most SIMD code and dispatch infrequently.