2 ms·

I'm arguing that it should be possible for city planners to be able to, while designing their road networks, assume all cars will be capable of driving through

by dzaima 2y ago

I'm arguing that it should be possible for city planners to be able to, while designing their road networks, assume all cars will be capable of driving through their roads reasonably near the speed limit where possible (with exceptions alike hardware bugs patched in microcode, but those are to be hated and not the expectation). (I suppose there are specialty vehicles not meant for general travel, but I'd assume there are actual restrictions around such where reasonable (I know nothing cars), whereas, with hardware, if it exists, people will use it for whatever they want regardless of its intent) > As I'm sure you know the computational complexity of vrgather is proportional to vl^2 But that sucks for anything wanting just a fixed 16-byte table (which is a pretty frequent thing in fixed-width SIMD), and low VL will not help with that as the table argument of vrgather is always VLMAX; you necessarily need low LMUL (so a more correct average complexity is LMUL*VL even for hardware dynamically scaling resource usage based on VL). But, if I wanted a 32-byte table, I'd have to jump to LMUL=2 for portable code, at which point VLEN=512 hardware would be forced to consider each result element potentially selecting from 128 bytes of data. Granted, here at least it can be reasonably expected that general-purpose hardware would make LMUL=1 not horribly slow for typical use-cases (be it via spamming silicon at it, not having high VLEN, or specializing for small local variance of indices at runtime (which, while neat, is silicon that could've been spent on actually meaningful things instead of reconstructing what the developer already knew; and even then it probably would result in higher latency)).

brucehoult 2y ago
If you have code that is tuned so critically on LMUL then you can always determine the correct LMUL value at program startup, for the hardware you find yourself running on, whether by looking at the CPU model or running a few test cases or whatever. Unlike with choosing between SSE, AVX, AVX512 etc, with RVV you don't have to duplicate your code to do so.
- dzaima 2y ago
  But for SSE/AVX/AVX512, I don't have to duplicate code to get reasonable performance - code written for SSE2 will still run very reasonably on AVX-512 hardware. And if all I need is primarily 16-byte, there may not be any reason to even bother with AVX2/AVX-512 (biggest benefit might very well be just the extra registers & separate destination, which don't require source-level changes anyway). Whereas with RVV you might have to dynamically select LMUL to not get unreasonable perf (a very low bar!). And.. a 16- or 32-byte shuffle should not be considered as some "critically tuned" thing, this is basic fixed-width SIMD stuff.
  - brucehoult 2y ago
    And we're back to you wanting all RVV implementors to make the same performance vs size/cost trade-offs when the whole point of it is that different implementors have the freedom to make the trade-offs that make sense for their target markets -- and you have the freedom to buy them or not buy them, depending on whether they fit your needs. There is not a single vendor who has to address all markets and use-cases, there will be dozens of them, all using a common ISA and all (sensibly witten) software will run everywhere, at varying levels of performance and cost.
    - dzaima 2y ago
      It's not so much that I want that, more that it's a basic and requirement for fixed-width usages, and is easily attained by aarch64 & x86. Whereas RVV's potential of tradeoffs clearly interferes with that. And if hardware making bad tradeoffs for fixed-width usages ever makes it to general-purpose usage to any significance, RVV as a whole would end up being required to be considered as not fit for fixed-width stuff (outside of extremely important & well-funded things where people can afford to add manual tuning for each separate funky CPU, but it's a pretty safe bet that most developers won't have every single piece of RISC-V hardware.. especially that which they would hate). Without reasonable performance guarantees, "RVV can imitate NEON" is as useful of a statement as "base RV64G without any extensions at all can imitate NEON". It's just stating that it's a turing-complete system.
      - brucehoult 2y ago
        And, once again, I point out that X280 is not intended for general-purpose usage. It's for special purpose stuff, and I'm sure fits what NASA wants in their space PIC64 chip very well. In the specific case of the SG2380, the 16 P670 cores are there for general-purpose use, with the 4 X280s for specialised uses. And, again, even at 1 element per cycle, using vrgather is significantly faster than not using it and using the dual-issue scalar side instead. AND at short NEON or AVX vector lengths the X280 is single-cycle for vrgather *anyway*.
        2y ago
camel-cdr 2y ago
Fortunately 16-byte tables are the good case that always fits into a LMUL=1 register under the standard V extension. I do agree however, this would result in bad performance for VLEN>128 and VLEN<DLEN implementations. Adding a gather instructions limited to 16 elements might be a solution, however it could also result in vendors feeling more free to have a slower LMUL=1 vrgahter, that still needs to perform reasonably. The LMUL=1 and 16 element vrgahters are the most common cases. IMO the software ecosystem needs to steer the hardware here, in place of ARM for neon, and Intel/AMD for AVX, stopping people from implementations with disproportionately slow permutes. People can't expect to put a ML core, like the X280, as the main CPU on a regular desktop class processor and expect general purpose software to be optimized on it. I suppose this could be added to the RVA profile as well, but they don't want to specify micro architectural details, so what does it mean for an instruction to be fast? Maybe there could be a non-normative clause that software is expected to use LMUL=1 vrgather for LUTs and other shuffles inside of hot loops. I suppose creating an issue regarding this can't hurt, so I'll look into it.

by dzaima 2y ago

brucehoult 2y ago

If you have code that is tuned so critically on LMUL then you can always determine the correct LMUL value at program startup, for the hardware you find yourself running on, whether by looking at the CPU model or running a few test cases or whatever. Unlike with choosing between SSE, AVX, AVX512 etc, with RVV you don't have to duplicate your code to do so.

dzaima 2y ago
But for SSE/AVX/AVX512, I don't have to duplicate code to get reasonable performance - code written for SSE2 will still run very reasonably on AVX-512 hardware. And if all I need is primarily 16-byte, there may not be any reason to even bother with AVX2/AVX-512 (biggest benefit might very well be just the extra registers & separate destination, which don't require source-level changes anyway). Whereas with RVV you might have to dynamically select LMUL to not get unreasonable perf (a very low bar!). And.. a 16- or 32-byte shuffle should not be considered as some "critically tuned" thing, this is basic fixed-width SIMD stuff.
- brucehoult 2y ago
  And we're back to you wanting all RVV implementors to make the same performance vs size/cost trade-offs when the whole point of it is that different implementors have the freedom to make the trade-offs that make sense for their target markets -- and you have the freedom to buy them or not buy them, depending on whether they fit your needs. There is not a single vendor who has to address all markets and use-cases, there will be dozens of them, all using a common ISA and all (sensibly witten) software will run everywhere, at varying levels of performance and cost.
  - dzaima 2y ago
    It's not so much that I want that, more that it's a basic and requirement for fixed-width usages, and is easily attained by aarch64 & x86. Whereas RVV's potential of tradeoffs clearly interferes with that. And if hardware making bad tradeoffs for fixed-width usages ever makes it to general-purpose usage to any significance, RVV as a whole would end up being required to be considered as not fit for fixed-width stuff (outside of extremely important & well-funded things where people can afford to add manual tuning for each separate funky CPU, but it's a pretty safe bet that most developers won't have every single piece of RISC-V hardware.. especially that which they would hate). Without reasonable performance guarantees, "RVV can imitate NEON" is as useful of a statement as "base RV64G without any extensions at all can imitate NEON". It's just stating that it's a turing-complete system.
    - brucehoult 2y ago
      And, once again, I point out that X280 is not intended for general-purpose usage. It's for special purpose stuff, and I'm sure fits what NASA wants in their space PIC64 chip very well. In the specific case of the SG2380, the 16 P670 cores are there for general-purpose use, with the 4 X280s for specialised uses. And, again, even at 1 element per cycle, using vrgather is significantly faster than not using it and using the dual-issue scalar side instead. AND at short NEON or AVX vector lengths the X280 is single-cycle for vrgather *anyway*.
      - 2y ago

camel-cdr 2y ago

Fortunately 16-byte tables are the good case that always fits into a LMUL=1 register under the standard V extension. I do agree however, this would result in bad performance for VLEN>128 and VLEN<DLEN implementations. Adding a gather instructions limited to 16 elements might be a solution, however it could also result in vendors feeling more free to have a slower LMUL=1 vrgahter, that still needs to perform reasonably. The LMUL=1 and 16 element vrgahters are the most common cases. IMO the software ecosystem needs to steer the hardware here, in place of ARM for neon, and Intel/AMD for AVX, stopping people from implementations with disproportionately slow permutes. People can't expect to put a ML core, like the X280, as the main CPU on a regular desktop class processor and expect general purpose software to be optimized on it. I suppose this could be added to the RVA profile as well, but they don't want to specify micro architectural details, so what does it mean for an instruction to be fast? Maybe there could be a non-normative clause that software is expected to use LMUL=1 vrgather for LUTs and other shuffles inside of hot loops. I suppose creating an issue regarding this can't hurt, so I'll look into it.