6 ms·

SVE was designed mindful of how CPUs currently operate, whereas RISC-V vector extensions were designed with fondness for how CPUs operated decades ago. Well th

SVE was designed mindful of how CPUs currently operate, whereas RISC-V vector extensions were designed with fondness for how CPUs operated decades ago. Well that's somewhat of an exaggeration, but XT-910 speculatively executes vector instructions based on a prediction of how vsetvl(i) modifies register configurations in order to achieve good performance, so changing this register configuration causes speculation failures as though it were a mispredicted branch. Which you need to do if you're doing any mixed precision operations, or mixed integer/floating point, and discourages small SIMD functions. Quote from their white paper: "this is not friendly to deeply pipelined processor architecture." Fundamentally, I dislike how completely the meaning of RISC-V vector instructions depends on what instructions were executed an arbitrarily long time beforehand. Also he's really complaining about register indexing in load/stores?

_chris_ 5y ago
> Quote from their white paper: "this is not friendly to deeply pipelined processor architecture." There is zero reason you can't rename the VL register. "Speculating" that it doesn't change is only one design point.
- brigade 5y ago
  Well... when the operation of register renaming itself depends on the vtype register, that isn't a complete solution.
SuchAnonMuchWow 5y ago
Also, the RISC-5 vector extension is at the opposite of the philosophy of RISC vs CISC
- brucehoult 5y ago
  In what way? - load/store with arithmetic register-to-register only. Check. - single instruction length. Check. - fixed instruction execution time. Check. - large set of registers. Check.
  - SuchAnonMuchWow 5y ago
    The vector extension has many good properties, like the ability to adapt to various size/complexity of processors: it can be implemented slowly on a small processor, and quickly on fast processor. This is because the instructions are very abstract and most of the control is done by the core instead of the compiler, like splitting the vectors into chunks that can be computed per cycle by the core. But then you loose the fixed instruction execution time: if you configure your registers to be larger than the compute units of the core, the instruction will take multiple cycles. Citing wikipedia on RISC archs: "The main distinguishing feature of RISC architecture is that the instruction set is optimized with a large number of registers and a highly regular instruction pipeline, allowing a low number of clock cycles per instruction (CPI)" - a highly regular instruction pipeline: Nope - a low number of clock cycles per instruction (CPI): Nope In the 80'-90', it became apparent that RISC vs CISC was a moot point, and both types of ISA could be equally efficient. However, I think if you want a universal ISA, CISC is better because it is easier to decompose complex instructions into smaller ones on small processor, rather than to detect patterns of simple instructions and optimize them on a large processor. This is the direction RISC-V took with the vector extension. But I find it a stretch to call it a RISC ISA then.
    - brucehoult 5y ago
      Thanks for the reply. >But then you loose the fixed instruction execution time: if you configure your registers to be larger than the compute units of the core, the instruction will take multiple cycles. "multiple" doesn't imply "not fixed". Floating Point on pretty much any modern CPU, CISC or RISC, takes 3 or 4 clock cycles for add/subtract/multiply/FMA. It's not 1 cycle, but it's always the same. This means the compiler can (if it's told what CPU the code will be running on with -mtune or whatever) accurately schedule the code for maximum efficiency. And usually the FPU will be pipelined, so you can start a new add or multiply every clock cycle, as long as it doesn't depend on the result of a very recent one. Recently I've been using the Allwinner D1, which implements draft version 0.7.1 of the RISC-V Vector ISA with 128 bit registers. Like an FPU, most instructions take 3 clock cycles with LMUL=1. If you increase LMUL then, yes, the instructions take longer, but it is deterministic and you can do your instruction scheduling around it. If you look at the result of a vectorised memcpy() I did on the D1, you can see that the execution time is absolutely identical at 30.9 ns (31 clock cycles at 1.008 GHz) from sz=0 to sz=64 http://hoult.org/d1_memcpy.txt http://hoult.org/d1_memcpy.txt At sz=0 this is 60% of the time for the standard scalar memcpy() in glibc!! By sz=64 it is 3.6 times faster than the standard memcpy(). I'm using LMUL=4 there. LMUL=2 provided slightly lower latency for sz<=32 but couldn't achieve as much speed in cache. LMUL=8 didn't provide any more speed in cache but had latency closer to the standard scalar code (but still less than it) for sz=0.
brucehoult 5y ago
vsetvl{i} should not cause speculation failures. The VTYPE part of vsetvl (which sets the VTYPE CSR) is conceptually part of dynamically following instruction opcodes. There simply isn't room for it in the current fixed-size 32 bit opcodes, but in future 64 bit instruction encodings [1] the VTYPE will be explicit in eery instruction. This is no different to every IEEE-compliant Floating Point implementation (which ARM and x86 of course are) including a rounding mode field in every instruction which often (in fact usually) contains the value "dynamic" meaning that the rounding mode is taken from a CSR. In both cases the instruction fetch stage can simply jam the current value from the CSR on to the end of each instruction and have it pass through the pipeline with it as if it had been in the instruction opcode in the first place. VTYPE has 8 active bits in the current v1.0 draft, and the XT-910 and XT-906 implement a different 6 bit VTYPE of which 2 bits must always be zero (so 4 bits really). VSETVL with a register argument could cause a mis-speculation, but it will normally only be used for saving and restoring VTYPE on task switches. Normal code will usually be using the immediate form, which the instruction decoder can pick off and save locally. As for the VL CSR part, there is a bit in VTYPE that specifies whether a) you don't care about elements past VL -- they can be computed the same as if they were within the vector length or not, or b) elements past VL remain undisturbed. Designers/users of OoO machines may be happier when the software specifies a) and designers/users of microcontrollers may be happier when the software specifies b). So when "tail agnostic" is set, very wide hardware might calculate all elements anyway, while hardware with N ALUs might calculate whatever portion of the tail falls in the block of N elements containing the end of the vector, but not later blocks. The XT-910 designers of course didn't have this option as their CPU implements a two year old draft specification that no one was ever supposed to implement in mass-production hardware -- or at least if they do they have to know it has no guarantee to be compatible with RVV v1.0 The XT-906 uses the same vector processor (or at least the same ISA). I've been using one via ssh recently and hope to have my own board with the Allwinner D1 chip that uses the XT-906 core later this month. I've got some simple vector test results at: http://hoult.org/d1_memcpy.txt http://hoult.org/d1_memcpy.txt http://hoult.org/d1_strcpy.txt http://hoult.org/d1_strcpy.txt [1] 64 bit RVV opcodes will also likely have other enhancements such as being able to use any V register for masking, other masking options (e.g. invert the mask), ability to address more V registers, four operand versions of multiply-accumulate, and I don't know what else. No real effort has been expended on actually designing that in detail yet.

_chris_ 5y ago

> Quote from their white paper: "this is not friendly to deeply pipelined processor architecture." There is zero reason you can't rename the VL register. "Speculating" that it doesn't change is only one design point.

brigade 5y ago
Well... when the operation of register renaming itself depends on the vtype register, that isn't a complete solution.

SuchAnonMuchWow 5y ago

Also, the RISC-5 vector extension is at the opposite of the philosophy of RISC vs CISC

brucehoult 5y ago
In what way? - load/store with arithmetic register-to-register only. Check. - single instruction length. Check. - fixed instruction execution time. Check. - large set of registers. Check.
- SuchAnonMuchWow 5y ago
  The vector extension has many good properties, like the ability to adapt to various size/complexity of processors: it can be implemented slowly on a small processor, and quickly on fast processor. This is because the instructions are very abstract and most of the control is done by the core instead of the compiler, like splitting the vectors into chunks that can be computed per cycle by the core. But then you loose the fixed instruction execution time: if you configure your registers to be larger than the compute units of the core, the instruction will take multiple cycles. Citing wikipedia on RISC archs: "The main distinguishing feature of RISC architecture is that the instruction set is optimized with a large number of registers and a highly regular instruction pipeline, allowing a low number of clock cycles per instruction (CPI)" - a highly regular instruction pipeline: Nope - a low number of clock cycles per instruction (CPI): Nope In the 80'-90', it became apparent that RISC vs CISC was a moot point, and both types of ISA could be equally efficient. However, I think if you want a universal ISA, CISC is better because it is easier to decompose complex instructions into smaller ones on small processor, rather than to detect patterns of simple instructions and optimize them on a large processor. This is the direction RISC-V took with the vector extension. But I find it a stretch to call it a RISC ISA then.
  - brucehoult 5y ago
    Thanks for the reply. >But then you loose the fixed instruction execution time: if you configure your registers to be larger than the compute units of the core, the instruction will take multiple cycles. "multiple" doesn't imply "not fixed". Floating Point on pretty much any modern CPU, CISC or RISC, takes 3 or 4 clock cycles for add/subtract/multiply/FMA. It's not 1 cycle, but it's always the same. This means the compiler can (if it's told what CPU the code will be running on with -mtune or whatever) accurately schedule the code for maximum efficiency. And usually the FPU will be pipelined, so you can start a new add or multiply every clock cycle, as long as it doesn't depend on the result of a very recent one. Recently I've been using the Allwinner D1, which implements draft version 0.7.1 of the RISC-V Vector ISA with 128 bit registers. Like an FPU, most instructions take 3 clock cycles with LMUL=1. If you increase LMUL then, yes, the instructions take longer, but it is deterministic and you can do your instruction scheduling around it. If you look at the result of a vectorised memcpy() I did on the D1, you can see that the execution time is absolutely identical at 30.9 ns (31 clock cycles at 1.008 GHz) from sz=0 to sz=64 http://hoult.org/d1_memcpy.txt http://hoult.org/d1_memcpy.txt At sz=0 this is 60% of the time for the standard scalar memcpy() in glibc!! By sz=64 it is 3.6 times faster than the standard memcpy(). I'm using LMUL=4 there. LMUL=2 provided slightly lower latency for sz<=32 but couldn't achieve as much speed in cache. LMUL=8 didn't provide any more speed in cache but had latency closer to the standard scalar code (but still less than it) for sz=0.

brucehoult 5y ago

vsetvl{i} should not cause speculation failures. The VTYPE part of vsetvl (which sets the VTYPE CSR) is conceptually part of dynamically following instruction opcodes. There simply isn't room for it in the current fixed-size 32 bit opcodes, but in future 64 bit instruction encodings [1] the VTYPE will be explicit in eery instruction. This is no different to every IEEE-compliant Floating Point implementation (which ARM and x86 of course are) including a rounding mode field in every instruction which often (in fact usually) contains the value "dynamic" meaning that the rounding mode is taken from a CSR. In both cases the instruction fetch stage can simply jam the current value from the CSR on to the end of each instruction and have it pass through the pipeline with it as if it had been in the instruction opcode in the first place. VTYPE has 8 active bits in the current v1.0 draft, and the XT-910 and XT-906 implement a different 6 bit VTYPE of which 2 bits must always be zero (so 4 bits really). VSETVL with a register argument could cause a mis-speculation, but it will normally only be used for saving and restoring VTYPE on task switches. Normal code will usually be using the immediate form, which the instruction decoder can pick off and save locally. As for the VL CSR part, there is a bit in VTYPE that specifies whether a) you don't care about elements past VL -- they can be computed the same as if they were within the vector length or not, or b) elements past VL remain undisturbed. Designers/users of OoO machines may be happier when the software specifies a) and designers/users of microcontrollers may be happier when the software specifies b). So when "tail agnostic" is set, very wide hardware might calculate all elements anyway, while hardware with N ALUs might calculate whatever portion of the tail falls in the block of N elements containing the end of the vector, but not later blocks. The XT-910 designers of course didn't have this option as their CPU implements a two year old draft specification that no one was ever supposed to implement in mass-production hardware -- or at least if they do they have to know it has no guarantee to be compatible with RVV v1.0 The XT-906 uses the same vector processor (or at least the same ISA). I've been using one via ssh recently and hope to have my own board with the Allwinner D1 chip that uses the XT-906 core later this month. I've got some simple vector test results at: http://hoult.org/d1_memcpy.txt http://hoult.org/d1_memcpy.txt http://hoult.org/d1_strcpy.txt http://hoult.org/d1_strcpy.txt [1] 64 bit RVV opcodes will also likely have other enhancements such as being able to use any V register for masking, other masking options (e.g. invert the mask), ability to address more V registers, four operand versions of multiply-accumulate, and I don't know what else. No real effort has been expended on actually designing that in detail yet.