4 ms·

The main abstraction of intrinsics is register allocation, right? Is there anything else that can be gained by handwritten asm?

camel-cdr 3y ago
For rvv specifically there are a few things that aren't possible using the intrinsics abstraction. E.g. in asm you can run the same instruction sequence with different vtype (element width and LMUL).
- janwas 3y ago
  We are able to do the same with Highway's RVV :)
  - dzaima 3y ago
    I believe what camel-cdr is saying is being able to run the same code without duplication (say, a loop) which has no vsetvl-s inside, by conditionally choosing either an initial "vsetvli x0,x0,e32,m1" or "vsetvl x0,x0,e16,m1" or "vsetvl x0,x0,e32,m2" etc, which is just unachievable with the intrinsics as they hard-code vtype in each intrinsic. It's an extremely fun idea (primarily just for code size though), but thankfully (?) its usability is restricted by load/store instrs hard-coding the element type, so the main use of this would end up for switching LMUL, which has very limited usefulness. What Highway can support is generating multiple loops of different vtype from the same code, which effectively achieves the same thing, at the cost of machine code duplication.
    - camel-cdr 3y ago
      > for switching LMUL, which has very limited usefulness I currently have a quite usefull use case for it, I'm concerting utf8 to utf32 and if I've got an average utf8 character size of above 2 I could reduce the LMUL for that loop iteration. This shouldn't actually improve performance that much in good rvv implementations, since you can use vl and not LMUL to schedule your execution units. Sadly this is currently not the standard, and ara is the only implementation, that does this I know of. I think this wouldn't even be about code size reduction, consider an input, where there is basically a 50/50 probability LMUL can be reduced, that would be horrible for the branch predictor, but with only a branch over vsetvl, this could behave as a conditional vsetvl via instruction fusion. We'll have to see if such optimization become relevant once there is more hardware out there.
      - dzaima 3y ago
        That's an interesting use-case, though I wouldn't be surprised if some impls really wouldn't like LMUL dynamically switching at runtime a lot (i.e. something like LMUL being forwarded at decode-time, so it couldn't decode after an unknown-LMUL vsetvl, ruining perf)
    - janwas 3y ago
      Oh, I see, thanks for clarifying. Yes, I was referring only to "same source code" and agree our approach would generate multiple copies of the instructions.
anonymoushn 3y ago
I think so. For the most part I'm happy to have the compiler rearrange my intrinsics and decide where the spills should go (inevitably you have spills if you unroll the loop enough times to avoid stalls, because these instructions tend to be like "you can have 6 in flight at a time but that would take 18 registers lol". The main problem I've met here is that the compiler will emit useless instructions to narrow or widen integers in general-purpose registers (e.g. the result of movemask) but this can be solved by looking at the generated assembly and fixing the high-level code.
nkurz 3y ago
If you are trying to maximize port utilization, sometimes the exact instruction ordering can make a big difference. Compilers often want to "hoist" loads to the top, which can sometimes reduce performance. And sometimes you need a particular addressing mode to avoid a bottleneck. In general, kierank is right: if you want to full optimize something, and you know what you want the actual code to look like, just write it in assembly. Nothing else gives you full control over loads and stores, and anything else leaves you at the mercy of some future compiler "optimization" stepping in to defeat you.
- dist1ll 3y ago
  > If you are trying to maximize port utilization, sometimes the exact instruction ordering can make a big difference. Interesting. I thought you'd be at the mercy of the instruction scheduler for aggressive OoO cores.
brigade 3y ago
Custom function ABIs. Though the maintenance overhead really isn't worth the reduced cache footprint, especially since asm writers are allergic to leaving behind any comments. Also instruction scheduling. Low-end Cortex will probably be in-order till the end of time...

camel-cdr 3y ago

For rvv specifically there are a few things that aren't possible using the intrinsics abstraction. E.g. in asm you can run the same instruction sequence with different vtype (element width and LMUL).

janwas 3y ago
We are able to do the same with Highway's RVV :)
- dzaima 3y ago
  I believe what camel-cdr is saying is being able to run the same code without duplication (say, a loop) which has no vsetvl-s inside, by conditionally choosing either an initial "vsetvli x0,x0,e32,m1" or "vsetvl x0,x0,e16,m1" or "vsetvl x0,x0,e32,m2" etc, which is just unachievable with the intrinsics as they hard-code vtype in each intrinsic. It's an extremely fun idea (primarily just for code size though), but thankfully (?) its usability is restricted by load/store instrs hard-coding the element type, so the main use of this would end up for switching LMUL, which has very limited usefulness. What Highway can support is generating multiple loops of different vtype from the same code, which effectively achieves the same thing, at the cost of machine code duplication.
  - camel-cdr 3y ago
    > for switching LMUL, which has very limited usefulness I currently have a quite usefull use case for it, I'm concerting utf8 to utf32 and if I've got an average utf8 character size of above 2 I could reduce the LMUL for that loop iteration. This shouldn't actually improve performance that much in good rvv implementations, since you can use vl and not LMUL to schedule your execution units. Sadly this is currently not the standard, and ara is the only implementation, that does this I know of. I think this wouldn't even be about code size reduction, consider an input, where there is basically a 50/50 probability LMUL can be reduced, that would be horrible for the branch predictor, but with only a branch over vsetvl, this could behave as a conditional vsetvl via instruction fusion. We'll have to see if such optimization become relevant once there is more hardware out there.
    - dzaima 3y ago
      That's an interesting use-case, though I wouldn't be surprised if some impls really wouldn't like LMUL dynamically switching at runtime a lot (i.e. something like LMUL being forwarded at decode-time, so it couldn't decode after an unknown-LMUL vsetvl, ruining perf)
  - janwas 3y ago
    Oh, I see, thanks for clarifying. Yes, I was referring only to "same source code" and agree our approach would generate multiple copies of the instructions.

anonymoushn 3y ago

I think so. For the most part I'm happy to have the compiler rearrange my intrinsics and decide where the spills should go (inevitably you have spills if you unroll the loop enough times to avoid stalls, because these instructions tend to be like "you can have 6 in flight at a time but that would take 18 registers lol". The main problem I've met here is that the compiler will emit useless instructions to narrow or widen integers in general-purpose registers (e.g. the result of movemask) but this can be solved by looking at the generated assembly and fixing the high-level code.

nkurz 3y ago

If you are trying to maximize port utilization, sometimes the exact instruction ordering can make a big difference. Compilers often want to "hoist" loads to the top, which can sometimes reduce performance. And sometimes you need a particular addressing mode to avoid a bottleneck. In general, kierank is right: if you want to full optimize something, and you know what you want the actual code to look like, just write it in assembly. Nothing else gives you full control over loads and stores, and anything else leaves you at the mercy of some future compiler "optimization" stepping in to defeat you.

dist1ll 3y ago
> If you are trying to maximize port utilization, sometimes the exact instruction ordering can make a big difference. Interesting. I thought you'd be at the mercy of the instruction scheduler for aggressive OoO cores.

brigade 3y ago

Custom function ABIs. Though the maintenance overhead really isn't worth the reduced cache footprint, especially since asm writers are allergic to leaving behind any comments. Also instruction scheduling. Low-end Cortex will probably be in-order till the end of time...