4 ms·

> They specifies a constant SIMD width so it's non-portable. This is incorrect, you can use vectors wider than native SIMD width and the compiler will break th

by exDM69 2mo ago

> They specifies a constant SIMD width so it's non-portable. This is incorrect, you can use vectors wider than native SIMD width and the compiler will break them down to register size of the target cpu. In fact it's sometimes better to used wider than native width, in some applications I see 20% better throughput with f32x16 (512 bits) on an AVX2 CPU (256 bits). It is kinda like loop unrolling it.

camel-cdr 2mo ago
Except you can't use this in actual code, because either, as is the case in this example with f32x32, you run out of registers and spill all over the place. Or you aren't using your full vector register or could've gotten better performance by "unrolling" more often for the larger vectors. If you use f32x16 (the avx-512 wisth), SSE now effectively has 4 registers to work with and will spill when doing anything beyond the most simple stuff. The default should imo be relative to the native register width, so you can do 1x, 2x or sometimes 4x the native width, depensing on your register preasure.
- exDM69 2mo ago
  I can and I do use this is "actual code" and I've got benchmarks to prove that it's got better throughput (for the particular use case, don't extrapolate from there) and the same applies to AVX2 and AVX512: twice the native vector width has ~20% better throughput (ie. using `f32x32` on AVX-512). I pass in the vector width as a generic parameter like this: fn do_simd_stuff<const N: usize>(x: Simd<f32, N>) { x.mul_add(x+x, x*x); } With this I can easily benchmark the same code for any vector width. I can also do some compile time heuristics to choose the vector width based on what's available on the compile target CPU. > you run out of registers and spill all over the place As usual when optimizing SIMD code, you should keep an eye on the generated disassembly and the benchmark results and watch for register pressure and the other usual things. I'm definitely NOT saying that you always get the best perf by using 2x SIMD width, but in this particular case it was so. This is much much easier to do with portable_simd than if you'd write the same with intrinsics, you can change the SIMD width without having to rewrite all your code (e.g. changing from SSE `_mm_add_ps` to AVX `_mm256_add_ps` etc). It's still a partial solution, you still need to drop down to intrinsics for some special instructions every now and then (which is easy), but in my projects this accounts for much less than 1% of the lines of code. Not applicable everywhere of course.
  - camel-cdr 2mo ago
    > twice the native vector width has ~20% better throughput Yes, this is what I was saying, but twice the vector width of AVX-512 will perform horrible in SSE, which is why portable SIMD abstractions should make writing code relative to the native vector width simple. > I pass in the vector width as a generic parameter like this: > fn do_simd_stuff<const N: usize>(x: Simd<f32, N>) { ... } My problem is that no portable_simd example code I've seen does this, which causes people to choose one specific N and run with that. The second part of the problem is how you find the native vector length, so you can instantiate the generic function. IIRC this isn't even exposed in portable_simd and you have to use a seperate crate to get it.
    - exDM69 2mo ago
      > The second part of the problem is how you find the native vector length, so you can instantiate the generic function. IIRC this isn't even exposed in portable_simd and you have to use a seperate crate to get it. This is trivial (but not pretty!) to do with something like `#[cfg(target_feature = "avx2")] const SIMD_WIDTH: usize = 8`. You need a few lines of ugly cfg logic to configure this. A somewhat orthogonal and much more difficult problem is how to select it at runtime. You would either need to have different binaries built with different compiler options, link object files built with different compiler options to same binary, or dynamically link the correct code at runtime. This is actually one of the (IMO only) cases where intrinsics are more practical: you can use `_mm256_add_ps` from AVX2 intrinsics regardless of whether you've configured your compiler to support AVX2 or not. As long as you check at runtime before calling the code so you don't get illegal instruction exceptions.
- imtringued 2mo ago
  Your complaint is silly. Worst case the compiler copy pastes the code multiple times or adds a constant size for loop meaning the cost is negligible. In the better cases it figures out how to reorder the instructions and gains performance over the naive implementation.

by exDM69 2mo ago

camel-cdr 2mo ago

Except you can't use this in actual code, because either, as is the case in this example with f32x32, you run out of registers and spill all over the place. Or you aren't using your full vector register or could've gotten better performance by "unrolling" more often for the larger vectors. If you use f32x16 (the avx-512 wisth), SSE now effectively has 4 registers to work with and will spill when doing anything beyond the most simple stuff. The default should imo be relative to the native register width, so you can do 1x, 2x or sometimes 4x the native width, depensing on your register preasure.

exDM69 2mo ago
I can and I do use this is "actual code" and I've got benchmarks to prove that it's got better throughput (for the particular use case, don't extrapolate from there) and the same applies to AVX2 and AVX512: twice the native vector width has ~20% better throughput (ie. using `f32x32` on AVX-512). I pass in the vector width as a generic parameter like this: fn do_simd_stuff<const N: usize>(x: Simd<f32, N>) { x.mul_add(x+x, x*x); } With this I can easily benchmark the same code for any vector width. I can also do some compile time heuristics to choose the vector width based on what's available on the compile target CPU. > you run out of registers and spill all over the place As usual when optimizing SIMD code, you should keep an eye on the generated disassembly and the benchmark results and watch for register pressure and the other usual things. I'm definitely NOT saying that you always get the best perf by using 2x SIMD width, but in this particular case it was so. This is much much easier to do with portable_simd than if you'd write the same with intrinsics, you can change the SIMD width without having to rewrite all your code (e.g. changing from SSE `_mm_add_ps` to AVX `_mm256_add_ps` etc). It's still a partial solution, you still need to drop down to intrinsics for some special instructions every now and then (which is easy), but in my projects this accounts for much less than 1% of the lines of code. Not applicable everywhere of course.
- camel-cdr 2mo ago
  > twice the native vector width has ~20% better throughput Yes, this is what I was saying, but twice the vector width of AVX-512 will perform horrible in SSE, which is why portable SIMD abstractions should make writing code relative to the native vector width simple. > I pass in the vector width as a generic parameter like this: > fn do_simd_stuff<const N: usize>(x: Simd<f32, N>) { ... } My problem is that no portable_simd example code I've seen does this, which causes people to choose one specific N and run with that. The second part of the problem is how you find the native vector length, so you can instantiate the generic function. IIRC this isn't even exposed in portable_simd and you have to use a seperate crate to get it.
  - exDM69 2mo ago
    > The second part of the problem is how you find the native vector length, so you can instantiate the generic function. IIRC this isn't even exposed in portable_simd and you have to use a seperate crate to get it. This is trivial (but not pretty!) to do with something like `#[cfg(target_feature = "avx2")] const SIMD_WIDTH: usize = 8`. You need a few lines of ugly cfg logic to configure this. A somewhat orthogonal and much more difficult problem is how to select it at runtime. You would either need to have different binaries built with different compiler options, link object files built with different compiler options to same binary, or dynamically link the correct code at runtime. This is actually one of the (IMO only) cases where intrinsics are more practical: you can use `_mm256_add_ps` from AVX2 intrinsics regardless of whether you've configured your compiler to support AVX2 or not. As long as you check at runtime before calling the code so you don't get illegal instruction exceptions.
imtringued 2mo ago
Your complaint is silly. Worst case the compiler copy pastes the code multiple times or adds a constant size for loop meaning the cost is negligible. In the better cases it figures out how to reorder the instructions and gains performance over the naive implementation.