3 ms·

> we'll pretty much be in a golden age of SIMD. We already are in the golden age of SIMD. NVidia and AMD GPUs are easier and easier to program through standard

> we'll pretty much be in a golden age of SIMD. We already are in the golden age of SIMD. NVidia and AMD GPUs are easier and easier to program through standard interfaces. Intel / AMD are pushing SIMD on a CPU, which is useful for sure, but always is going to be smaller in scope than a dedicated SIMD-processor like A100, 3060, AMD Vega, AMD 6800 xt and the like. SIMD-on-a-CPU is useful because you can perform SIMD over the L1 cache as communication (rather than traversing L1 -> L2 -> L3 -> DDR4 / PCIe -> GPU VRAM -> GPU Registers -> SIMD, and back). But if you have a large-scale operation that can work SIMD, the GPU-traversal absolutely works and is commonly done.

skavi 4y ago
Good point. Should have clarified I was referring to CPU SIMD.
- dragontamer 4y ago
  AVX2 is not as good as AVX512. But AVX2 still has vgather instructions, pshufb, and a few other useful tricks. AVX512 and ARM SVE2 bring the CPU up to parity with maybe 2010s-era GPUs or so (full gather/scatter, more permutation instructions, etc. etc.). But GPUs continued to evolve. Butterfly-shuffles are the generic any-to-any network building block, and are exposed in PTX (NVidia assembly) shfl.bfly, and AMD DPP (data-parallel primitives). Having a richer set of lane-to-lane shuffling (especially ready-to-use butterfly networks) would be best. It really is surprising how many problems require those rich-sets of data-movement instructions, or otherwise benefit from them. NEON and SVE had hard-coded data-movement for specific applications. The general-purpose instruction (pshufb) is kinda like permute/shfl from AMD/NVidia. A backwards-permute IIRC doesn't exist yet on CPU-side. And butterfly networks are the general-purpose solution, capable of implementing any arbitrary data-movement in just log(width) steps. (pshufb / permute instructions would be the full-sized butterfly network, but some cases might be "easier" and faster to execute with only a limited number of butterfly swaps, such as what inevitably comes up in sorting) -------- Still, all of these operations can be implemented in AVX2 (albeit slower / less efficiently). So its not like the "language" of AVX2 / AVX is incomplete... its just missing a few general-purpose instructions that could lead to better performance.

skavi 4y ago

Good point. Should have clarified I was referring to CPU SIMD.

dragontamer 4y ago
AVX2 is not as good as AVX512. But AVX2 still has vgather instructions, pshufb, and a few other useful tricks. AVX512 and ARM SVE2 bring the CPU up to parity with maybe 2010s-era GPUs or so (full gather/scatter, more permutation instructions, etc. etc.). But GPUs continued to evolve. Butterfly-shuffles are the generic any-to-any network building block, and are exposed in PTX (NVidia assembly) shfl.bfly, and AMD DPP (data-parallel primitives). Having a richer set of lane-to-lane shuffling (especially ready-to-use butterfly networks) would be best. It really is surprising how many problems require those rich-sets of data-movement instructions, or otherwise benefit from them. NEON and SVE had hard-coded data-movement for specific applications. The general-purpose instruction (pshufb) is kinda like permute/shfl from AMD/NVidia. A backwards-permute IIRC doesn't exist yet on CPU-side. And butterfly networks are the general-purpose solution, capable of implementing any arbitrary data-movement in just log(width) steps. (pshufb / permute instructions would be the full-sized butterfly network, but some cases might be "easier" and faster to execute with only a limited number of butterfly swaps, such as what inevitably comes up in sorting) -------- Still, all of these operations can be implemented in AVX2 (albeit slower / less efficiently). So its not like the "language" of AVX2 / AVX is incomplete... its just missing a few general-purpose instructions that could lead to better performance.