5 ms·

what's wrong with the riscv approach?

astrange 4y ago
They want to go back to the 70s (not meant as an insult) and use Cray-style vector instructions instead of SIMD. Vector instructions are kind of like loops in one instruction; you set a vector length register and then all the vector instructions change to run on that much data. That’s ok for big vectors you’d find in like scientific computations, but I believe it’s bad for anything I’ve ever written in SIMD. Games and multimedia usually use short vectors (say 4 ints) or mixed lengths (2 and 4 at once) and are pretty comfortable with how x86/ARM/PPC work. Not saying it couldn’t be better, but the RISCV designers wrote an article about how their approach would be better basically entirely because they thought SIMD adds too many new instructions and isn’t aesthetically pretty enough. Which doesn’t matter. Also I remember them calling SIMD something weird and politically incorrect in the article but can’t remember what it was…
- adgjlsfhk1 4y ago
  To me, the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger, and it really doesn't look good at all for big-little designs which seem to be the future. Most compilers still aren't good at generating AVX-512 code, and very few applications use it because it requires optimization for a very small portion of the market. With a variable length vector, everyone gets good code. Also, I'm not clear why you think riscv style vector instructions will perform worse on 2-4 length vectors.
  - astrange 4y ago
    > Also, I'm not clear why you think riscv style vector instructions will perform worse on 2-4 length vectors. Assumes there’s no switching cost to changing it and there won’t be restrictions on the values it can be set to. Weird restrictions on memory loads are pretty traditional for SIMD instructions after all. The switching cost is the biggest problem; this is exactly the kind of thing that x86 has specified, then found they can’t do without microcoding it or otherwise making it cripplingly inefficient on different HW generations.
  - janwas 4y ago
    > the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger This can be fixed with an abstraction layer such as github.com/google/highway (disclosure: I am the main author). The programming model is variable-length vectors and we can map that both to SSE4/AVX2/AVX-512 and SVE/RVV. It seems to me that the actual pain (of large numbers of unique instructions in the ISA) is pushed to the CPU decoder, binutils, and abstraction layer. > doesn't look good at all for big-little designs which seem to be the future Why not? It's not clear to me why OSes shouldn't provide a "don't migrate me while I'm checking CPU capabilities and then running this SIMD/vector kernel". > very few applications use it [AVX-512] because it requires optimization for a very small portion of the market. Maybe so (though increasing), but the cost is also low - we literally just pay with binary size and compile time (for the extra codepath).
  - fulafel 4y ago
    Compilers and languages have had a very long time to work on autovectorizing existing code for SIMD (54 years since ILLIAC, 25+ since SPARC VIS and MMX from Intel in 1994/1997). It mostly doesn't work for C/C++. The language has to support it, other optimization opportunities will be constantly blocked by lack information and/or restrictive specified semantics of operations. The potential of transparent SIMD use by compilers has apparently stayed marginal enough that no mainstream application languages in all that time have updated towards facilitating it.
- atq2119 4y ago
  Isn't there a sort of obvious "best of both worlds" by having a vector instruction ISA with a vector length register, plus the promise that a length >= 4 is always supported and good support for intra-group-of-4 shuffles? Then you can do whatever you did for games and multimedia in the past, except that you can process N samples/pixels/vectors/whatever at once, where N = vector length / 4, and your code can automatically make use of chips that allow longer vectors without requiring a recompile. Mind you, I don't know if that's the direction that the RISC-V people are taking. But it seems like a pretty obvious thing to do.
  - codedokode 4y ago
    In my opinion "the best" would be to support only one, fixed, largest vector size and emulate legacy instructions for smaller sizes using microcode (this is slow, but requires minimum transistor count). There is no need to optimize for legacy software; instead it should be recompiled and compiler should generate versions of code for all existing generations of CPUs. This way the CPU design can be simplified and all the complexity moved into compilers.
    - adgjlsfhk1 4y ago
      That would be awful for power consumption and only works well if everyone compiles everything from source. Otherwise every binary is awful performance for almost everyone.
      - jrockway 4y ago
        > everyone compiles everything from source One thing I learned from the M1 transition is that people will do it if someone tells them to. I bought a Mac this weekend to do exactly that; lots of users complaining about lack of M1 support. Time to add it (in a way that I can test). I have no choice.
        astrange 4y ago
        M1 has a very good x86 emulator for the moment. The users probably underestimate how good it is. But GPU programs get compiled from source at runtime all the time, and sort of are all about vectors. (M1’s GPU doesn’t actually have vectors.)
- devit 4y ago
  The RISC-V design is the sensible one since it does not hardcode the vector register size; the other designs are idiotic since a new instruction set needs to be created every time the vector register size is increased (MMX->SSE2->AVX2->AVX-512) and you can't use shorter vector registers on lower-power CPUs without reducing application compatibility. And it doesn't "loop" in one instruction, the vsetvl instruction will set the vector length to the _minimum_ of the vector register size and the amount of data left to process.
  - codedokode 4y ago
    I have read about ideas to make forward-compatible vector instructions, but personally I don't see much value in such features. If there appears a new CPU with different vector size then you can just recompile the program and let compiler use new instructions. Therefore it doesn't make sense to sacrifice chip area or performance for this feature. Regarding backward compatibility (running old software on a next generation CPU), I think that newer CPUs do not need to support obsolete instructions from previous generations. Instead they could just include a decoder that would (using microcode and therefore slowly) emulate old instructions. This allows user run their package manager or recompile their software and switch to new instructions. It is totally fine to completely redesign the ISA provided that there is a minimal converter for old instructions or at least support for software-based translator like qemu.
  - brigade 4y ago
    How do you handle LMUL>1 without "looping" across the multiple destination registers?
    - brucehoult 4y ago
      Well, for example the Alibaba T-Head Xuantie C910 has 128 bit vector registers but a 256 bit vector ALU, so both LMUL=1 and LMUL=2 instructions execute in a single clock cycle. LMUL=8 instructions take 4 cycles to execute, but that's a 1024 bit vector right there. Also, the person who said graphics uses 3 or 4 element vectors so you only need short vector registers didn't understand how a vector processor such as RISC-V will be programmed for that take. Usually you have thousands of XYZ or XYZW points that you want to transfor my the same matrix or normalize or whatever. You don't load one point into one vector register. You load all the X coordinates from 4 or 8 or 64 or 256 points into one vector register, all the Y coordinates into another vector register, all the Z coordinates into another one. And you put the 9 or 12 elements of the matrices you want to transform them by into other vector registers (or scalar registers if you're transforming everything by the same matrix) and you transform a whole bunch of points in parallel.

They want to go back to the 70s (not meant as an insult) and use Cray-style vector instructions instead of SIMD. Vector instructions are kind of like loops in one instruction; you set a vector length register and then all the vector instructions change to run on that much data. That’s ok for big vectors you’d find in like scientific computations, but I believe it’s bad for anything I’ve ever written in SIMD. Games and multimedia usually use short vectors (say 4 ints) or mixed lengths (2 and 4 at once) and are pretty comfortable with how x86/ARM/PPC work. Not saying it couldn’t be better, but the RISCV designers wrote an article about how their approach would be better basically entirely because they thought SIMD adds too many new instructions and isn’t aesthetically pretty enough. Which doesn’t matter. Also I remember them calling SIMD something weird and politically incorrect in the article but can’t remember what it was…

adgjlsfhk1 4y ago
To me, the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger, and it really doesn't look good at all for big-little designs which seem to be the future. Most compilers still aren't good at generating AVX-512 code, and very few applications use it because it requires optimization for a very small portion of the market. With a variable length vector, everyone gets good code. Also, I'm not clear why you think riscv style vector instructions will perform worse on 2-4 length vectors.
- astrange 4y ago
  > Also, I'm not clear why you think riscv style vector instructions will perform worse on 2-4 length vectors. Assumes there’s no switching cost to changing it and there won’t be restrictions on the values it can be set to. Weird restrictions on memory loads are pretty traditional for SIMD instructions after all. The switching cost is the biggest problem; this is exactly the kind of thing that x86 has specified, then found they can’t do without microcoding it or otherwise making it cripplingly inefficient on different HW generations.
- janwas 4y ago
  > the much bigger problem with SIMD is that it is really hard to program for since vector sizes keep getting bigger This can be fixed with an abstraction layer such as github.com/google/highway (disclosure: I am the main author). The programming model is variable-length vectors and we can map that both to SSE4/AVX2/AVX-512 and SVE/RVV. It seems to me that the actual pain (of large numbers of unique instructions in the ISA) is pushed to the CPU decoder, binutils, and abstraction layer. > doesn't look good at all for big-little designs which seem to be the future Why not? It's not clear to me why OSes shouldn't provide a "don't migrate me while I'm checking CPU capabilities and then running this SIMD/vector kernel". > very few applications use it [AVX-512] because it requires optimization for a very small portion of the market. Maybe so (though increasing), but the cost is also low - we literally just pay with binary size and compile time (for the extra codepath).
- fulafel 4y ago
  Compilers and languages have had a very long time to work on autovectorizing existing code for SIMD (54 years since ILLIAC, 25+ since SPARC VIS and MMX from Intel in 1994/1997). It mostly doesn't work for C/C++. The language has to support it, other optimization opportunities will be constantly blocked by lack information and/or restrictive specified semantics of operations. The potential of transparent SIMD use by compilers has apparently stayed marginal enough that no mainstream application languages in all that time have updated towards facilitating it.
atq2119 4y ago
Isn't there a sort of obvious "best of both worlds" by having a vector instruction ISA with a vector length register, plus the promise that a length >= 4 is always supported and good support for intra-group-of-4 shuffles? Then you can do whatever you did for games and multimedia in the past, except that you can process N samples/pixels/vectors/whatever at once, where N = vector length / 4, and your code can automatically make use of chips that allow longer vectors without requiring a recompile. Mind you, I don't know if that's the direction that the RISC-V people are taking. But it seems like a pretty obvious thing to do.
- codedokode 4y ago
  In my opinion "the best" would be to support only one, fixed, largest vector size and emulate legacy instructions for smaller sizes using microcode (this is slow, but requires minimum transistor count). There is no need to optimize for legacy software; instead it should be recompiled and compiler should generate versions of code for all existing generations of CPUs. This way the CPU design can be simplified and all the complexity moved into compilers.
  - adgjlsfhk1 4y ago
    That would be awful for power consumption and only works well if everyone compiles everything from source. Otherwise every binary is awful performance for almost everyone.
    - jrockway 4y ago
      > everyone compiles everything from source One thing I learned from the M1 transition is that people will do it if someone tells them to. I bought a Mac this weekend to do exactly that; lots of users complaining about lack of M1 support. Time to add it (in a way that I can test). I have no choice.
      - astrange 4y ago
        M1 has a very good x86 emulator for the moment. The users probably underestimate how good it is. But GPU programs get compiled from source at runtime all the time, and sort of are all about vectors. (M1’s GPU doesn’t actually have vectors.)
devit 4y ago
The RISC-V design is the sensible one since it does not hardcode the vector register size; the other designs are idiotic since a new instruction set needs to be created every time the vector register size is increased (MMX->SSE2->AVX2->AVX-512) and you can't use shorter vector registers on lower-power CPUs without reducing application compatibility. And it doesn't "loop" in one instruction, the vsetvl instruction will set the vector length to the _minimum_ of the vector register size and the amount of data left to process.
- codedokode 4y ago
  I have read about ideas to make forward-compatible vector instructions, but personally I don't see much value in such features. If there appears a new CPU with different vector size then you can just recompile the program and let compiler use new instructions. Therefore it doesn't make sense to sacrifice chip area or performance for this feature. Regarding backward compatibility (running old software on a next generation CPU), I think that newer CPUs do not need to support obsolete instructions from previous generations. Instead they could just include a decoder that would (using microcode and therefore slowly) emulate old instructions. This allows user run their package manager or recompile their software and switch to new instructions. It is totally fine to completely redesign the ISA provided that there is a minimal converter for old instructions or at least support for software-based translator like qemu.
- brigade 4y ago
  How do you handle LMUL>1 without "looping" across the multiple destination registers?
  - brucehoult 4y ago
    Well, for example the Alibaba T-Head Xuantie C910 has 128 bit vector registers but a 256 bit vector ALU, so both LMUL=1 and LMUL=2 instructions execute in a single clock cycle. LMUL=8 instructions take 4 cycles to execute, but that's a 1024 bit vector right there. Also, the person who said graphics uses 3 or 4 element vectors so you only need short vector registers didn't understand how a vector processor such as RISC-V will be programmed for that take. Usually you have thousands of XYZ or XYZW points that you want to transfor my the same matrix or normalize or whatever. You don't load one point into one vector register. You load all the X coordinates from 4 or 8 or 64 or 256 points into one vector register, all the Y coordinates into another vector register, all the Z coordinates into another one. And you put the 9 or 12 elements of the matrices you want to transform them by into other vector registers (or scalar registers if you're transforming everything by the same matrix) and you transform a whole bunch of points in parallel.