3 ms·

Those things are nice but in the end if I'm trying to get the most work done in the shortest amount of time, having wider registers help more than that stuff.

adgjlsfhk1 3y ago
masked operations are really useful for dealing with the tail of loops. You can add a single masked instruction to deal with the last few elements rather than having to do a scalar loop at the end to deal with the bits that don't fit nicely into a register. Doubling the number of registers is also great. It's basically free (since CPUS have lots of virtual registers) and can lead to way fewer load and store instructions. Most importantly, these changes are basically free from an area and power perspective. A 2x wider register will give you ~1.8x better performance, but it typically comes with ~1.5x higher power and wasted silicon for non-vectorized code. EVEX gives you all the really nice to have features that might only give you 20% better performance, but do so while lowering power and not taking up extra die area.
jcranmer 3y ago
Not necessarily. The last I checked, even Intel's compilers default to 256-bit vectorization when compiling against AVX-512 targets. And of course, AMD implemented AVX-512 via double-pumping of 256-bit SIMD operations (so AVX-512 isn't really faster than AVX-256 there). If memory bandwidth is a significant factor in your execution time, doubling the SIMD vector size is going to give you far less than the 100% theoretical speed boost (before taking into account thermal throttling, of course). Furthermore, the execution penalties due to things like remainder loops or vector misalignment (every misaligned 512-bit vector crosses cache lines) are comparatively more painful. The fancy new features ameliorate those downsides a lot and furthermore make vectorization profitable on a larger fraction of your code. And finally, if the limiting factor of your code's execution time is the number of FMA units you can ram data through, then at this point, you've already likely switched to using a GPU to do that instead of the CPU.

adgjlsfhk1 3y ago

masked operations are really useful for dealing with the tail of loops. You can add a single masked instruction to deal with the last few elements rather than having to do a scalar loop at the end to deal with the bits that don't fit nicely into a register. Doubling the number of registers is also great. It's basically free (since CPUS have lots of virtual registers) and can lead to way fewer load and store instructions. Most importantly, these changes are basically free from an area and power perspective. A 2x wider register will give you ~1.8x better performance, but it typically comes with ~1.5x higher power and wasted silicon for non-vectorized code. EVEX gives you all the really nice to have features that might only give you 20% better performance, but do so while lowering power and not taking up extra die area.

jcranmer 3y ago

Not necessarily. The last I checked, even Intel's compilers default to 256-bit vectorization when compiling against AVX-512 targets. And of course, AMD implemented AVX-512 via double-pumping of 256-bit SIMD operations (so AVX-512 isn't really faster than AVX-256 there). If memory bandwidth is a significant factor in your execution time, doubling the SIMD vector size is going to give you far less than the 100% theoretical speed boost (before taking into account thermal throttling, of course). Furthermore, the execution penalties due to things like remainder loops or vector misalignment (every misaligned 512-bit vector crosses cache lines) are comparatively more painful. The fancy new features ameliorate those downsides a lot and furthermore make vectorization profitable on a larger fraction of your code. And finally, if the limiting factor of your code's execution time is the number of FMA units you can ram data through, then at this point, you've already likely switched to using a GPU to do that instead of the CPU.