4 ms·

What is the real cost to just have those few methods be compiled in and then a branch? You don't need to ship a separate binary for each target, you can have de

What is the real cost to just have those few methods be compiled in and then a branch? You don't need to ship a separate binary for each target, you can have dead code in it. I mean fat binaries take this idea to the extreme to support multiple architectures. https://en.wikipedia.org/wiki/Fat_binary https://en.wikipedia.org/wiki/Fat_binary

dzaima 3y ago
That's 5 copies for SVE (had an error in first message - SVE allows up to 2048-bit vectors, not 1024), and 10 copies for RVV if you wanted to target all widths (though you'd probably be fine for a decade or a couple by targeting just 128 & 256-bit, and maybe 512-bit). Plus one more for a scalar fallback. And yes, it's not particularly large of a cost, other than it being an extremely pointless waste of space given that it is possible to have just one variant that covers them all. Though, it would become significantly more problematic if you wanted to target different extension groups too (which you would quite likely want to some extent) as those'd multiply with all the length targets - SVE vs SVE2 vs more future extensions, and on RVV there's just a lot (Zvfh & Zvfhmin for FP16, Zvbb for extra bitmanip stuff, many more here[1]; and potentially at some point there could be an extension that uses a wider encoding scheme to inline vsetvl fields & allow masking by registers other than v0, which could benefit everything) [1]: https://github.com/riscv/riscv-crypto/blob/c8ddeb7e64a3444dda0438316af1238aeed72041/doc/vector/riscv-crypto-spec-vector.adoc https://github.com/riscv/riscv-crypto/blob/c8ddeb7e64a3444dd...
- anonymoushn 3y ago
  Prior to SVE2 string processing algorithms already had to ship a bunch of copies for SVE because it lacked a "shuffle the 16-byte lanes of A according to the shuffle in the 16-byte lanes of B" instruction :(
elabajaba 3y ago
Not being able to inline and having to branch on every call to a simd function can sometimes make it slower than the basic scalar version.
- Pet_Ant 3y ago
  Just a thought, but would it be possible to hot patch at the time of loading the binary? I realise it might require updates to the binary format, but it might be very well justified.
  - anonymoushn 3y ago
    This sounds similar to shipping all these routines in dynamic libraries and loading the right one at runtime. I'm not sure if the memory cost of having all these duplicate simd kernels is a big deal though.
- dzaima 3y ago
  You sould branch at the level where inlining doesn't make sense, which would usually be some function wrapping the big loop, which should be rather free. Which is the same situation as on x86-64 if you want to target pre-AVX2/post-AVX2/AVX-512.

dzaima 3y ago

That's 5 copies for SVE (had an error in first message - SVE allows up to 2048-bit vectors, not 1024), and 10 copies for RVV if you wanted to target all widths (though you'd probably be fine for a decade or a couple by targeting just 128 & 256-bit, and maybe 512-bit). Plus one more for a scalar fallback. And yes, it's not particularly large of a cost, other than it being an extremely pointless waste of space given that it is possible to have just one variant that covers them all. Though, it would become significantly more problematic if you wanted to target different extension groups too (which you would quite likely want to some extent) as those'd multiply with all the length targets - SVE vs SVE2 vs more future extensions, and on RVV there's just a lot (Zvfh & Zvfhmin for FP16, Zvbb for extra bitmanip stuff, many more here[1]; and potentially at some point there could be an extension that uses a wider encoding scheme to inline vsetvl fields & allow masking by registers other than v0, which could benefit everything) [1]: https://github.com/riscv/riscv-crypto/blob/c8ddeb7e64a3444dda0438316af1238aeed72041/doc/vector/riscv-crypto-spec-vector.adoc https://github.com/riscv/riscv-crypto/blob/c8ddeb7e64a3444dd...

anonymoushn 3y ago
Prior to SVE2 string processing algorithms already had to ship a bunch of copies for SVE because it lacked a "shuffle the 16-byte lanes of A according to the shuffle in the 16-byte lanes of B" instruction :(

elabajaba 3y ago

Not being able to inline and having to branch on every call to a simd function can sometimes make it slower than the basic scalar version.

Pet_Ant 3y ago
Just a thought, but would it be possible to hot patch at the time of loading the binary? I realise it might require updates to the binary format, but it might be very well justified.
- anonymoushn 3y ago
  This sounds similar to shipping all these routines in dynamic libraries and loading the right one at runtime. I'm not sure if the memory cost of having all these duplicate simd kernels is a big deal though.
dzaima 3y ago
You sould branch at the level where inlining doesn't make sense, which would usually be some function wrapping the big loop, which should be rather free. Which is the same situation as on x86-64 if you want to target pre-AVX2/post-AVX2/AVX-512.