5 ms·

You're arguing that a world in which some companies make Ferraris and some companies make Bambinos is a bad world and "someone" should make sure only things bet

You're arguing that a world in which some companies make Ferraris and some companies make Bambinos is a bad world and "someone" should make sure only things between a Corolla and a Camry exist. Which is, basically, what Intel or Arm do, at any given time. I prefer the possibility that some people make amazing products, and some people make underperforming (but possibly much chaper) products, and the market decides which ones to buy. The most important thing is that they can all use the same roads (software), even if they should stick to different lanes. > SG2380 / X280 are already known to have a very horrible vrgather, which is gonna be extremely awful for fixed-width stuff, and even some scalable stuff too. I don't believe that to be the case. IIRC X280 does vrgather in ~1 cycle for vector lengths up to the datapath width (256 bits, 32 bytes), so anything corresponding to your NEON or AVX2 cases are going to be just fine. As I'm sure you know the computational complexity of vrgather is proportional to vl^2 so no one is going to do single-cycle vrgather at LMUL=8 => 4096 bit sizes, or even probably at 512 bit. You could use a 256 bit datapath to implement 512 bit vrgather in 4 cycles, and LMUL=8 (4096 bit) in 256 cycles but that's not the intended use for the X280 so they didn't spend those transistors. At longer sizes X280 does 1 element per cycle, which is still a significant speedup over what you could do on the dual-issue in-order scalar CPU. > And SG2380 is rather explicitly for general-purpose computing. The *P670* cores on the SG2380 is for general-purpose computing. The X280 cores are for media processing and similar tasks that don't typically use large gather operations.

dzaima 2y ago
I'm arguing that it should be possible for city planners to be able to, while designing their road networks, assume all cars will be capable of driving through their roads reasonably near the speed limit where possible (with exceptions alike hardware bugs patched in microcode, but those are to be hated and not the expectation). (I suppose there are specialty vehicles not meant for general travel, but I'd assume there are actual restrictions around such where reasonable (I know nothing cars), whereas, with hardware, if it exists, people will use it for whatever they want regardless of its intent) > As I'm sure you know the computational complexity of vrgather is proportional to vl^2 But that sucks for anything wanting just a fixed 16-byte table (which is a pretty frequent thing in fixed-width SIMD), and low VL will not help with that as the table argument of vrgather is always VLMAX; you necessarily need low LMUL (so a more correct average complexity is LMUL*VL even for hardware dynamically scaling resource usage based on VL). But, if I wanted a 32-byte table, I'd have to jump to LMUL=2 for portable code, at which point VLEN=512 hardware would be forced to consider each result element potentially selecting from 128 bytes of data. Granted, here at least it can be reasonably expected that general-purpose hardware would make LMUL=1 not horribly slow for typical use-cases (be it via spamming silicon at it, not having high VLEN, or specializing for small local variance of indices at runtime (which, while neat, is silicon that could've been spent on actually meaningful things instead of reconstructing what the developer already knew; and even then it probably would result in higher latency)).
- brucehoult 2y ago
  If you have code that is tuned so critically on LMUL then you can always determine the correct LMUL value at program startup, for the hardware you find yourself running on, whether by looking at the CPU model or running a few test cases or whatever. Unlike with choosing between SSE, AVX, AVX512 etc, with RVV you don't have to duplicate your code to do so.
  - dzaima 2y ago
    But for SSE/AVX/AVX512, I don't have to duplicate code to get reasonable performance - code written for SSE2 will still run very reasonably on AVX-512 hardware. And if all I need is primarily 16-byte, there may not be any reason to even bother with AVX2/AVX-512 (biggest benefit might very well be just the extra registers & separate destination, which don't require source-level changes anyway). Whereas with RVV you might have to dynamically select LMUL to not get unreasonable perf (a very low bar!). And.. a 16- or 32-byte shuffle should not be considered as some "critically tuned" thing, this is basic fixed-width SIMD stuff.
    - brucehoult 2y ago
      And we're back to you wanting all RVV implementors to make the same performance vs size/cost trade-offs when the whole point of it is that different implementors have the freedom to make the trade-offs that make sense for their target markets -- and you have the freedom to buy them or not buy them, depending on whether they fit your needs. There is not a single vendor who has to address all markets and use-cases, there will be dozens of them, all using a common ISA and all (sensibly witten) software will run everywhere, at varying levels of performance and cost.
      - dzaima 2y ago
        It's not so much that I want that, more that it's a basic and requirement for fixed-width usages, and is easily attained by aarch64 & x86. Whereas RVV's potential of tradeoffs clearly interferes with that. And if hardware making bad tradeoffs for fixed-width usages ever makes it to general-purpose usage to any significance, RVV as a whole would end up being required to be considered as not fit for fixed-width stuff (outside of extremely important & well-funded things where people can afford to add manual tuning for each separate funky CPU, but it's a pretty safe bet that most developers won't have every single piece of RISC-V hardware.. especially that which they would hate). Without reasonable performance guarantees, "RVV can imitate NEON" is as useful of a statement as "base RV64G without any extensions at all can imitate NEON". It's just stating that it's a turing-complete system.
camel-cdr 2y ago
My understanding is that the X280 performs at one cycle per element, and that there is supposed to be a faster gather for LMUL=1/2. As you mentioned the data-path width. If that is true, then their higher LMUL implementations are basically a design mistake. They could've used the LMUL=1/2 vrgather to implement LMUL=1 and above by calling it repeatedly (LMUL*2)^2 times, that should add almost no extra die area. This is what the C908 and X60 seem to do.
- dzaima 2y ago
  To be fair, a 1 cycle/element isn't that bad at, say, e64,m8 - 64 cycles - whereas the extending mf2 to m8 would be 16^2=256 shuffles (and there's still some hardware cost involved from having to merge together the shuffles based on the index bits, and of course scheduling it all). But yeah it's pretty bad for LMUL=1, being slower at just vl>4, and still has worse worst-case timings (e8,m8 still being 256 shuffles, where VLMAX=512).
  - camel-cdr 2y ago
    Yeah, C908 and X60 have a 256 cycle LMUL=8 vrgather for any SEW. I've written what I expect sane implementations to do here: https://gitlab.com/riseproject/riscv-optimization-guide/-/issues/1#note_1977583125 https://gitlab.com/riseproject/riscv-optimization-guide/-/is... This didn't take into account higher SEW, but I could imagine an implementation that as a LMUL2 native implementation for e32/e64. The number of connections for a specific SEW and VLEN should be (VLEN/SEW)(SEW/8), however the regularity and distance probably also impacts final implementation cost.
    - dzaima 2y ago
      There is the better, but much more complex, option of a variable-latency vrgather, where, for each DLEN-sized output segment, it counts the number of distinct DLEN-sized chunks of the table the current indices require, and spends, say, ceil(that_count/2) uops (each uop being able to take two DLEN-sized table chunks, the indices, and outputting DLEN data; could also have only one DLEN-sized table chunk input at the obvious cost; also at some point there needs to be the necessary blending, idk where that fits in; probably could just be a temporary directly in the shuffle silicon). This would result in k*VL/DLEN uops for simple index patters such as zip/unzip/reverse/broadcast with k=1 or k=2, and worst-case (e.g. transpose) is as bad as your option 2 with N=1 or N=2, and requires just DLEN^2 connections in the main shuffle, as much of the heavy lifting is done by the already-necessary silicon for getting DLEN-sized chunks of operands (maybe there might be complications with having those not being requested sequentially though; I'm not a hardware person).

dzaima 2y ago

I'm arguing that it should be possible for city planners to be able to, while designing their road networks, assume all cars will be capable of driving through their roads reasonably near the speed limit where possible (with exceptions alike hardware bugs patched in microcode, but those are to be hated and not the expectation). (I suppose there are specialty vehicles not meant for general travel, but I'd assume there are actual restrictions around such where reasonable (I know nothing cars), whereas, with hardware, if it exists, people will use it for whatever they want regardless of its intent) > As I'm sure you know the computational complexity of vrgather is proportional to vl^2 But that sucks for anything wanting just a fixed 16-byte table (which is a pretty frequent thing in fixed-width SIMD), and low VL will not help with that as the table argument of vrgather is always VLMAX; you necessarily need low LMUL (so a more correct average complexity is LMUL*VL even for hardware dynamically scaling resource usage based on VL). But, if I wanted a 32-byte table, I'd have to jump to LMUL=2 for portable code, at which point VLEN=512 hardware would be forced to consider each result element potentially selecting from 128 bytes of data. Granted, here at least it can be reasonably expected that general-purpose hardware would make LMUL=1 not horribly slow for typical use-cases (be it via spamming silicon at it, not having high VLEN, or specializing for small local variance of indices at runtime (which, while neat, is silicon that could've been spent on actually meaningful things instead of reconstructing what the developer already knew; and even then it probably would result in higher latency)).

brucehoult 2y ago
If you have code that is tuned so critically on LMUL then you can always determine the correct LMUL value at program startup, for the hardware you find yourself running on, whether by looking at the CPU model or running a few test cases or whatever. Unlike with choosing between SSE, AVX, AVX512 etc, with RVV you don't have to duplicate your code to do so.
- dzaima 2y ago
  But for SSE/AVX/AVX512, I don't have to duplicate code to get reasonable performance - code written for SSE2 will still run very reasonably on AVX-512 hardware. And if all I need is primarily 16-byte, there may not be any reason to even bother with AVX2/AVX-512 (biggest benefit might very well be just the extra registers & separate destination, which don't require source-level changes anyway). Whereas with RVV you might have to dynamically select LMUL to not get unreasonable perf (a very low bar!). And.. a 16- or 32-byte shuffle should not be considered as some "critically tuned" thing, this is basic fixed-width SIMD stuff.
  - brucehoult 2y ago
    And we're back to you wanting all RVV implementors to make the same performance vs size/cost trade-offs when the whole point of it is that different implementors have the freedom to make the trade-offs that make sense for their target markets -- and you have the freedom to buy them or not buy them, depending on whether they fit your needs. There is not a single vendor who has to address all markets and use-cases, there will be dozens of them, all using a common ISA and all (sensibly witten) software will run everywhere, at varying levels of performance and cost.
    - dzaima 2y ago
      It's not so much that I want that, more that it's a basic and requirement for fixed-width usages, and is easily attained by aarch64 & x86. Whereas RVV's potential of tradeoffs clearly interferes with that. And if hardware making bad tradeoffs for fixed-width usages ever makes it to general-purpose usage to any significance, RVV as a whole would end up being required to be considered as not fit for fixed-width stuff (outside of extremely important & well-funded things where people can afford to add manual tuning for each separate funky CPU, but it's a pretty safe bet that most developers won't have every single piece of RISC-V hardware.. especially that which they would hate). Without reasonable performance guarantees, "RVV can imitate NEON" is as useful of a statement as "base RV64G without any extensions at all can imitate NEON". It's just stating that it's a turing-complete system.

camel-cdr 2y ago

My understanding is that the X280 performs at one cycle per element, and that there is supposed to be a faster gather for LMUL=1/2. As you mentioned the data-path width. If that is true, then their higher LMUL implementations are basically a design mistake. They could've used the LMUL=1/2 vrgather to implement LMUL=1 and above by calling it repeatedly (LMUL*2)^2 times, that should add almost no extra die area. This is what the C908 and X60 seem to do.

dzaima 2y ago
To be fair, a 1 cycle/element isn't that bad at, say, e64,m8 - 64 cycles - whereas the extending mf2 to m8 would be 16^2=256 shuffles (and there's still some hardware cost involved from having to merge together the shuffles based on the index bits, and of course scheduling it all). But yeah it's pretty bad for LMUL=1, being slower at just vl>4, and still has worse worst-case timings (e8,m8 still being 256 shuffles, where VLMAX=512).
- camel-cdr 2y ago
  Yeah, C908 and X60 have a 256 cycle LMUL=8 vrgather for any SEW. I've written what I expect sane implementations to do here: https://gitlab.com/riseproject/riscv-optimization-guide/-/issues/1#note_1977583125 https://gitlab.com/riseproject/riscv-optimization-guide/-/is... This didn't take into account higher SEW, but I could imagine an implementation that as a LMUL2 native implementation for e32/e64. The number of connections for a specific SEW and VLEN should be (VLEN/SEW)(SEW/8), however the regularity and distance probably also impacts final implementation cost.
  - dzaima 2y ago
    There is the better, but much more complex, option of a variable-latency vrgather, where, for each DLEN-sized output segment, it counts the number of distinct DLEN-sized chunks of the table the current indices require, and spends, say, ceil(that_count/2) uops (each uop being able to take two DLEN-sized table chunks, the indices, and outputting DLEN data; could also have only one DLEN-sized table chunk input at the obvious cost; also at some point there needs to be the necessary blending, idk where that fits in; probably could just be a temporary directly in the shuffle silicon). This would result in k*VL/DLEN uops for simple index patters such as zip/unzip/reverse/broadcast with k=1 or k=2, and worst-case (e.g. transpose) is as bad as your option 2 with N=1 or N=2, and requires just DLEN^2 connections in the main shuffle, as much of the heavy lifting is done by the already-necessary silicon for getting DLEN-sized chunks of operands (maybe there might be complications with having those not being requested sequentially though; I'm not a hardware person).