3 ms·

I was hoping for a "with hindsight" view from a solid CPU guy, but what you've said is interesting, but can you quantify 'superior' / 'flexible' and why you fee

I was hoping for a "with hindsight" view from a solid CPU guy, but what you've said is interesting, but can you quantify 'superior' / 'flexible' and why you feel it's a good target compared to an imagined deeply pipelined version of existing instructions? TIA

adrian_b 4y ago
The most obvious advantage of AVX-512 is that it has mask registers which select the parts of a register that are modified by an operation. This brings a huge simplification to any loop which must process arrays of variable sizes and alignments. With SSE or AVX you have to write very complex prologue and epilogue code for loops, to handle the sizes that are not a multiple of the register size and to cope with various alignments. A compiler may be not smart enough to add such code, so it may fail to vectorize a loop, while when the code is written by hand it is hard to avoid bugs. With AVX-512 handling any size or alignment is trivial and it can also be done automatically by compilers when vectorizing loops. The second feature is that in AVX-512 there are both gather load and scatter store instructions, which simplify the manual or automatic vectorizing of loops that must access some data structures that do not have the layout required by the SIMD instructions. Also AVX-512 has a more complete and flexible set of instructions to do various permutations between the parts of the SIMD registers, which also allow the vectorization of some loops that would have been difficult to convert to SIMD instructions when using SSE and AVX. The net effect of all these features is that many loops that were not worthwhile to convert to SIMD with SSE or AVX, because the resulting programs would have been very complex and would have contained a large number of instructions, can be converted to AVX-512 with little effort and the resulting program has much less instructions than in the SSE/AVX version. For the loops that are easy to convert to SIMD, e.g. they access only aligned arrays whose size is a multiple of the SIMD register size and there is no need to access other kinds of data structures, e.g. arrays of structures with heterogeneous members, AVX-512 may have no advantage over AVX, except if the target CPU happens to have double throughput with 512-bit registers (true for a part of the server/workstation Intel CPUs).
- adgjlsfhk1 4y ago
  AVX-512 has really high potential, but for some reason, LLVM (not sure about GCC) is really bad at effectively using it.

adrian_b 4y ago

The most obvious advantage of AVX-512 is that it has mask registers which select the parts of a register that are modified by an operation. This brings a huge simplification to any loop which must process arrays of variable sizes and alignments. With SSE or AVX you have to write very complex prologue and epilogue code for loops, to handle the sizes that are not a multiple of the register size and to cope with various alignments. A compiler may be not smart enough to add such code, so it may fail to vectorize a loop, while when the code is written by hand it is hard to avoid bugs. With AVX-512 handling any size or alignment is trivial and it can also be done automatically by compilers when vectorizing loops. The second feature is that in AVX-512 there are both gather load and scatter store instructions, which simplify the manual or automatic vectorizing of loops that must access some data structures that do not have the layout required by the SIMD instructions. Also AVX-512 has a more complete and flexible set of instructions to do various permutations between the parts of the SIMD registers, which also allow the vectorization of some loops that would have been difficult to convert to SIMD instructions when using SSE and AVX. The net effect of all these features is that many loops that were not worthwhile to convert to SIMD with SSE or AVX, because the resulting programs would have been very complex and would have contained a large number of instructions, can be converted to AVX-512 with little effort and the resulting program has much less instructions than in the SSE/AVX version. For the loops that are easy to convert to SIMD, e.g. they access only aligned arrays whose size is a multiple of the SIMD register size and there is no need to access other kinds of data structures, e.g. arrays of structures with heterogeneous members, AVX-512 may have no advantage over AVX, except if the target CPU happens to have double throughput with 512-bit registers (true for a part of the server/workstation Intel CPUs).

adgjlsfhk1 4y ago
AVX-512 has really high potential, but for some reason, LLVM (not sure about GCC) is really bad at effectively using it.