3 ms·

To me, the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger, and it really doesn't look good at all

To me, the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger, and it really doesn't look good at all for big-little designs which seem to be the future. Most compilers still aren't good at generating AVX-512 code, and very few applications use it because it requires optimization for a very small portion of the market. With a variable length vector, everyone gets good code. Also, I'm not clear why you think riscv style vector instructions will perform worse on 2-4 length vectors.

astrange 4y ago
> Also, I'm not clear why you think riscv style vector instructions will perform worse on 2-4 length vectors. Assumes there’s no switching cost to changing it and there won’t be restrictions on the values it can be set to. Weird restrictions on memory loads are pretty traditional for SIMD instructions after all. The switching cost is the biggest problem; this is exactly the kind of thing that x86 has specified, then found they can’t do without microcoding it or otherwise making it cripplingly inefficient on different HW generations.
janwas 4y ago
> the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger This can be fixed with an abstraction layer such as github.com/google/highway (disclosure: I am the main author). The programming model is variable-length vectors and we can map that both to SSE4/AVX2/AVX-512 and SVE/RVV. It seems to me that the actual pain (of large numbers of unique instructions in the ISA) is pushed to the CPU decoder, binutils, and abstraction layer. > doesn't look good at all for big-little designs which seem to be the future Why not? It's not clear to me why OSes shouldn't provide a "don't migrate me while I'm checking CPU capabilities and then running this SIMD/vector kernel". > very few applications use it [AVX-512] because it requires optimization for a very small portion of the market. Maybe so (though increasing), but the cost is also low - we literally just pay with binary size and compile time (for the extra codepath).
fulafel 4y ago
Compilers and languages have had a very long time to work on autovectorizing existing code for SIMD (54 years since ILLIAC, 25+ since SPARC VIS and MMX from Intel in 1994/1997). It mostly doesn't work for C/C++. The language has to support it, other optimization opportunities will be constantly blocked by lack information and/or restrictive specified semantics of operations. The potential of transparent SIMD use by compilers has apparently stayed marginal enough that no mainstream application languages in all that time have updated towards facilitating it.

astrange 4y ago

> Also, I'm not clear why you think riscv style vector instructions will perform worse on 2-4 length vectors. Assumes there’s no switching cost to changing it and there won’t be restrictions on the values it can be set to. Weird restrictions on memory loads are pretty traditional for SIMD instructions after all. The switching cost is the biggest problem; this is exactly the kind of thing that x86 has specified, then found they can’t do without microcoding it or otherwise making it cripplingly inefficient on different HW generations.

janwas 4y ago

> the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger This can be fixed with an abstraction layer such as github.com/google/highway (disclosure: I am the main author). The programming model is variable-length vectors and we can map that both to SSE4/AVX2/AVX-512 and SVE/RVV. It seems to me that the actual pain (of large numbers of unique instructions in the ISA) is pushed to the CPU decoder, binutils, and abstraction layer. > doesn't look good at all for big-little designs which seem to be the future Why not? It's not clear to me why OSes shouldn't provide a "don't migrate me while I'm checking CPU capabilities and then running this SIMD/vector kernel". > very few applications use it [AVX-512] because it requires optimization for a very small portion of the market. Maybe so (though increasing), but the cost is also low - we literally just pay with binary size and compile time (for the extra codepath).

fulafel 4y ago

Compilers and languages have had a very long time to work on autovectorizing existing code for SIMD (54 years since ILLIAC, 25+ since SPARC VIS and MMX from Intel in 1994/1997). It mostly doesn't work for C/C++. The language has to support it, other optimization opportunities will be constantly blocked by lack information and/or restrictive specified semantics of operations. The potential of transparent SIMD use by compilers has apparently stayed marginal enough that no mainstream application languages in all that time have updated towards facilitating it.