5 ms·

You are misunderstanding, avx10/256 adds nothing, all those instructions already exist under avx512. All avx10/256 does is create yet another target, would be

You are misunderstanding, avx10/256 adds nothing, all those instructions already exist under avx512. All avx10/256 does is create yet another target, would be better to unify under avx512(which already supports 256 & 128 registers as a requirement).

raphlinus 3y ago
I don't agree. A 256 bit profile of the new AVX stuff gives you lots of goodies over AVX2, including mask predication, the fancy ternary op, and so on. Additionally, because it consumes half as many register bits as the 512 bit profile, it could be a lot more economical (and power efficient) to implement on cores that are not the high end server parts. Keep in mind ARM (even the top of their line) is still stuck on 128. So I think the case for this is a good one.
- anonymoushn 3y ago
  To clarify for readers, since I'm sure raphlinus understands this, those 256-bit instructions are present in AVX-512. But it will be nice to have those 256-bit instructions (my favorites are vpcompressb and friends) on cpus that do not have 512-bit instructions.
jcranmer 3y ago
AVX10/256 allows you to use the EVEX encoding with 256-bit registers without assuming that you'll have 512-bit register support. And if I have to be honest, it's all the things that EVEX encoding adds (double the number of registers, per-instruction rounding modes, masked vector operations) that are interesting, not the fact that you have 512-bit registers.
- TinkersW 3y ago
  Those things are nice but in the end if I'm trying to get the most work done in the shortest amount of time, having wider registers help more than that stuff.
  - adgjlsfhk1 3y ago
    masked operations are really useful for dealing with the tail of loops. You can add a single masked instruction to deal with the last few elements rather than having to do a scalar loop at the end to deal with the bits that don't fit nicely into a register. Doubling the number of registers is also great. It's basically free (since CPUS have lots of virtual registers) and can lead to way fewer load and store instructions. Most importantly, these changes are basically free from an area and power perspective. A 2x wider register will give you ~1.8x better performance, but it typically comes with ~1.5x higher power and wasted silicon for non-vectorized code. EVEX gives you all the really nice to have features that might only give you 20% better performance, but do so while lowering power and not taking up extra die area.
  - jcranmer 3y ago
    Not necessarily. The last I checked, even Intel's compilers default to 256-bit vectorization when compiling against AVX-512 targets. And of course, AMD implemented AVX-512 via double-pumping of 256-bit SIMD operations (so AVX-512 isn't really faster than AVX-256 there). If memory bandwidth is a significant factor in your execution time, doubling the SIMD vector size is going to give you far less than the 100% theoretical speed boost (before taking into account thermal throttling, of course). Furthermore, the execution penalties due to things like remainder loops or vector misalignment (every misaligned 512-bit vector crosses cache lines) are comparatively more painful. The fancy new features ameliorate those downsides a lot and furthermore make vectorization profitable on a larger fraction of your code. And finally, if the limiting factor of your code's execution time is the number of FMA units you can ram data through, then at this point, you've already likely switched to using a GPU to do that instead of the CPU.

raphlinus 3y ago

I don't agree. A 256 bit profile of the new AVX stuff gives you lots of goodies over AVX2, including mask predication, the fancy ternary op, and so on. Additionally, because it consumes half as many register bits as the 512 bit profile, it could be a lot more economical (and power efficient) to implement on cores that are not the high end server parts. Keep in mind ARM (even the top of their line) is still stuck on 128. So I think the case for this is a good one.

anonymoushn 3y ago
To clarify for readers, since I'm sure raphlinus understands this, those 256-bit instructions are present in AVX-512. But it will be nice to have those 256-bit instructions (my favorites are vpcompressb and friends) on cpus that do not have 512-bit instructions.

jcranmer 3y ago

AVX10/256 allows you to use the EVEX encoding with 256-bit registers without assuming that you'll have 512-bit register support. And if I have to be honest, it's all the things that EVEX encoding adds (double the number of registers, per-instruction rounding modes, masked vector operations) that are interesting, not the fact that you have 512-bit registers.

TinkersW 3y ago
Those things are nice but in the end if I'm trying to get the most work done in the shortest amount of time, having wider registers help more than that stuff.
- adgjlsfhk1 3y ago
  masked operations are really useful for dealing with the tail of loops. You can add a single masked instruction to deal with the last few elements rather than having to do a scalar loop at the end to deal with the bits that don't fit nicely into a register. Doubling the number of registers is also great. It's basically free (since CPUS have lots of virtual registers) and can lead to way fewer load and store instructions. Most importantly, these changes are basically free from an area and power perspective. A 2x wider register will give you ~1.8x better performance, but it typically comes with ~1.5x higher power and wasted silicon for non-vectorized code. EVEX gives you all the really nice to have features that might only give you 20% better performance, but do so while lowering power and not taking up extra die area.
- jcranmer 3y ago
  Not necessarily. The last I checked, even Intel's compilers default to 256-bit vectorization when compiling against AVX-512 targets. And of course, AMD implemented AVX-512 via double-pumping of 256-bit SIMD operations (so AVX-512 isn't really faster than AVX-256 there). If memory bandwidth is a significant factor in your execution time, doubling the SIMD vector size is going to give you far less than the 100% theoretical speed boost (before taking into account thermal throttling, of course). Furthermore, the execution penalties due to things like remainder loops or vector misalignment (every misaligned 512-bit vector crosses cache lines) are comparatively more painful. The fancy new features ameliorate those downsides a lot and furthermore make vectorization profitable on a larger fraction of your code. And finally, if the limiting factor of your code's execution time is the number of FMA units you can ram data through, then at this point, you've already likely switched to using a GPU to do that instead of the CPU.