6 ms·

That's a really good trick. New to me. Anyone know a reasonable way to do the query equivalent? Not allocate more, ask how many are currently available.

djmips 2y ago
On AMD ,not available to users yet AFAIK. Only available to the driver and currently targeted at ray tracing? On Nvidia you have to be writing in the PTX language to have access to something similar.
dragontamer 2y ago
Why bother? If the next assembly line says to use register#200 (but you failed because you only have 160 registers), what's the processor supposed to do? The only hope is to call for more registers and lock until those registers are available. There's no way to progress otherwise.
- im3w1l 2y ago
  Dynamically switch algorithms between one that is faster but has more register pressure and a slower one with less pressure. Lowering numerical precision might be on the table too.
  - wtallis 2y ago
    And what, refill the instruction cache in less time than it takes to wait for more registers to become available?
- JonChesterfield 2y ago
  Knowing how many registers are allocated is roughly how many are live, which is how many to write to the stack during a coroutine switch. I'm having trouble with the tradeoff between compiler complexity and perceived value of userspace scheduling on gpus so a means of moving work from compiler into runtime helps lower the barrier to landing the implementation. Oh, or with the work hat on, function pointers are a nuisance with kernel allocated resources, and a jump table keyed off the result of a register file size query would be a partial solution to that.
  - kimixa 2y ago
    At any time a coroutine switch could happen, doesn't the compiler statically know how many registers would be allocated by this anyway? Register numbers aren't dynamically indexed, they're baked into the instruction stream, so there's not really any new information you can't get from the instruction pointer and a static (compiler-generated) lookup table?
    - JonChesterfield 2y ago
      Yep, the compiler can do it all. However getting proof of concepts together to justify shipping the compiler complexity can go better with more work done by the runtime code.
  - phire 2y ago
    Yes... I could see that approach being desirable on a CPU, it would make all context switches cheaper. Are we re-inventing register windows? Of course, we are on a GPU here. Skimming through the RDNA 4 ISA document [1], looks like they expect waves using the dynamic VGPRs features to keep track of how many VGPRs they have requested in a Scalar register (SGPR). Each wave always has 106 SGPRs allocated (32-bit each), so the overhead of a single SGPR isn't an issue. You wouldn't even need the jump table, as AMD allows you to index into the VGPR register file. A small loop would allow you to push all active VGPRs onto the stack. Shame they don't also allow you to index into the SGPR registers too. But GPUs aren't really designed with coroutines or context switching in mind. Modern GPUs can do it, but GPUs have deep historical assumptions that every single wave on every single core will be running the exact same code, and that 99.99% of waves will run to completion without needing a context switch. When preemptive context switches are needed (no more than once every few ms, and only with long-running compute shaders), they can probably get away with simply DMAing the whole register file to memory. Though, part of the reason this feature was implemented is that the historic assumptions are no-longer true. DXR allows each material to register a unique hit shader, so Modern GPUs need to dynamic dispatch based on the result of ray hits. [1] https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna4-instruction-set-architecture.pdf https://www.amd.com/content/dam/amd/en/documents/radeon-tech...
Someone 2y ago
I don’t know whether it’s reasonable, but > Allocation requests don’t always succeed. s_alloc_vgpr sets the Scalar Condition Code (SCC) to indicate success, or clears it on failure. So, you can do a binary search: - set result to zero. - try to allocate 512. If that succeeds, add 512 to result - try to allocate 256. If that succeeds, add 256 to result - try to allocate 128. If that succeeds, add 128 to result ... - try to allocate 1. If that succeeds, add 1 to result (you’ll want to stop at the granularity at which registers are allocated) - free ’result’ registers (This isn’t race-free)
- Tuna-Fish 2y ago
  Note that: 1. Registers are not allocated individually, but in clusters of 32, and... 2. The max registers a single thread can access is 8 clusters (256 registers). The minimum is 1 cluster. You don't really need much of a binary search for this.
- dragontamer 2y ago
  The issue is there's plan b: spill your data to L1 cache. Whatever plan you have needs to be far faster than L1 cache to even make sense. I can imagine some very heavy register-pressure situations where registers are much better than cache accesses.
dagmx 2y ago
Given the speed of occupancy changes, you’d almost always fall afoul of TOCTOU on this.

djmips 2y ago

On AMD ,not available to users yet AFAIK. Only available to the driver and currently targeted at ray tracing? On Nvidia you have to be writing in the PTX language to have access to something similar.

dragontamer 2y ago

Why bother? If the next assembly line says to use register#200 (but you failed because you only have 160 registers), what's the processor supposed to do? The only hope is to call for more registers and lock until those registers are available. There's no way to progress otherwise.

im3w1l 2y ago
Dynamically switch algorithms between one that is faster but has more register pressure and a slower one with less pressure. Lowering numerical precision might be on the table too.
- wtallis 2y ago
  And what, refill the instruction cache in less time than it takes to wait for more registers to become available?
JonChesterfield 2y ago
Knowing how many registers are allocated is roughly how many are live, which is how many to write to the stack during a coroutine switch. I'm having trouble with the tradeoff between compiler complexity and perceived value of userspace scheduling on gpus so a means of moving work from compiler into runtime helps lower the barrier to landing the implementation. Oh, or with the work hat on, function pointers are a nuisance with kernel allocated resources, and a jump table keyed off the result of a register file size query would be a partial solution to that.
- kimixa 2y ago
  At any time a coroutine switch could happen, doesn't the compiler statically know how many registers would be allocated by this anyway? Register numbers aren't dynamically indexed, they're baked into the instruction stream, so there's not really any new information you can't get from the instruction pointer and a static (compiler-generated) lookup table?
  - JonChesterfield 2y ago
    Yep, the compiler can do it all. However getting proof of concepts together to justify shipping the compiler complexity can go better with more work done by the runtime code.
- phire 2y ago
  Yes... I could see that approach being desirable on a CPU, it would make all context switches cheaper. Are we re-inventing register windows? Of course, we are on a GPU here. Skimming through the RDNA 4 ISA document [1], looks like they expect waves using the dynamic VGPRs features to keep track of how many VGPRs they have requested in a Scalar register (SGPR). Each wave always has 106 SGPRs allocated (32-bit each), so the overhead of a single SGPR isn't an issue. You wouldn't even need the jump table, as AMD allows you to index into the VGPR register file. A small loop would allow you to push all active VGPRs onto the stack. Shame they don't also allow you to index into the SGPR registers too. But GPUs aren't really designed with coroutines or context switching in mind. Modern GPUs can do it, but GPUs have deep historical assumptions that every single wave on every single core will be running the exact same code, and that 99.99% of waves will run to completion without needing a context switch. When preemptive context switches are needed (no more than once every few ms, and only with long-running compute shaders), they can probably get away with simply DMAing the whole register file to memory. Though, part of the reason this feature was implemented is that the historic assumptions are no-longer true. DXR allows each material to register a unique hit shader, so Modern GPUs need to dynamic dispatch based on the result of ray hits. [1] https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna4-instruction-set-architecture.pdf https://www.amd.com/content/dam/amd/en/documents/radeon-tech...

Someone 2y ago

I don’t know whether it’s reasonable, but > Allocation requests don’t always succeed. s_alloc_vgpr sets the Scalar Condition Code (SCC) to indicate success, or clears it on failure. So, you can do a binary search: - set result to zero. - try to allocate 512. If that succeeds, add 512 to result - try to allocate 256. If that succeeds, add 256 to result - try to allocate 128. If that succeeds, add 128 to result ... - try to allocate 1. If that succeeds, add 1 to result (you’ll want to stop at the granularity at which registers are allocated) - free ’result’ registers (This isn’t race-free)

Tuna-Fish 2y ago
Note that: 1. Registers are not allocated individually, but in clusters of 32, and... 2. The max registers a single thread can access is 8 clusters (256 registers). The minimum is 1 cluster. You don't really need much of a binary search for this.
dragontamer 2y ago
The issue is there's plan b: spill your data to L1 cache. Whatever plan you have needs to be far faster than L1 cache to even make sense. I can imagine some very heavy register-pressure situations where registers are much better than cache accesses.

dagmx 2y ago

Given the speed of occupancy changes, you’d almost always fall afoul of TOCTOU on this.