3 ms·

> GPU can just raise IRQ and have a queue for the page faults. Handling GPU IRQ's is possible but has unnecessarily high latency and pipeline stalling problems

by exDM69 10y ago

> GPU can just raise IRQ and have a queue for the page faults. Handling GPU IRQ's is possible but has unnecessarily high latency and pipeline stalling problems. However, the kernel mode drivers do a bit of this behind the scenes (mainly when switching from app to app). But using sparse textures (ie. GL_EXT_sparse_texture2), the GPU can detect if a fault would occur and react to it. Instead of IRQ'ing for every missed page, a list of all missed pages can be extracted at the end of a frame. > Perhaps disk I/O DMA can even be connected to GPU DMA, bypassing system RAM entirely. Afaik Nvidia's NVLink does this but it's aimed at super computers, not graphics.

vardump 10y ago
> Handling GPU IRQ's is possible but has unnecessarily high latency and pipeline stalling problems. However, the kernel mode drivers do a bit of this behind the scenes (mainly when switching from app to app). Time it takes for GPU to access data present in its local internal GDDR RAM: about 1 microsecond. Time it takes from GPU asserting (pulls up) [1] IRQ until CPU handles IRQ: 5-50 microseconds. Time it takes for CPU to insert I/O request into OS internal queue: unknown, but it's almost certainly a small fraction of a microsecond. Hundreds of faults can be handled with one IRQ request. So there's not really that much more latency than in a normal CPU page fault. Actually probably less on average, because these faults can be grouped. GPU memory access latency is very slow and they are already built to handle high latency with a massive number of hardware threads. IRQ can for example be edge triggered by the first GPU page fault. Each GPU page fault doesn't need to trigger CPU side interrupt. That'd be pointless, it'd just cause high CPU load for no benefit. I think some sort of page fault FIFO is simpler hardware wise than to wait for something arbitrary event like "end of a frame". GPU can just keep pushing new faults in a CPU-visible FIFO. That way it's possible to amortize latency and to avoid IRQ storm (=excessive number of IRQ requests). CPU side can just pull faults from the FIFO at whatever rate I/O system can support. Fault data from GPU could also include priority, so that more visually important data can be fetched first even if it wasn't encountered first. For example geometry or textures covering major parts of screen could have high priority. (I've co-designed a HW mechanism for avoiding an excessive number of IRQs without sacrificing latency and designed and implemented a kernel mode driver to support it. Of course GPUs are quite a bit more complicated than that simple case.) [1]: Well, technically it's sending an MSI(-X) message. Just using traditional terminology for clarity. https://en.wikipedia.org/wiki/Message_Signaled_Interrupts#MSI-X https://en.wikipedia.org/wiki/Message_Signaled_Interrupts#MS...

by exDM69 10y ago

vardump 10y ago

> Handling GPU IRQ's is possible but has unnecessarily high latency and pipeline stalling problems. However, the kernel mode drivers do a bit of this behind the scenes (mainly when switching from app to app). Time it takes for GPU to access data present in its local internal GDDR RAM: about 1 microsecond. Time it takes from GPU asserting (pulls up) [1] IRQ until CPU handles IRQ: 5-50 microseconds. Time it takes for CPU to insert I/O request into OS internal queue: unknown, but it's almost certainly a small fraction of a microsecond. Hundreds of faults can be handled with one IRQ request. So there's not really that much more latency than in a normal CPU page fault. Actually probably less on average, because these faults can be grouped. GPU memory access latency is very slow and they are already built to handle high latency with a massive number of hardware threads. IRQ can for example be edge triggered by the first GPU page fault. Each GPU page fault doesn't need to trigger CPU side interrupt. That'd be pointless, it'd just cause high CPU load for no benefit. I think some sort of page fault FIFO is simpler hardware wise than to wait for something arbitrary event like "end of a frame". GPU can just keep pushing new faults in a CPU-visible FIFO. That way it's possible to amortize latency and to avoid IRQ storm (=excessive number of IRQ requests). CPU side can just pull faults from the FIFO at whatever rate I/O system can support. Fault data from GPU could also include priority, so that more visually important data can be fetched first even if it wasn't encountered first. For example geometry or textures covering major parts of screen could have high priority. (I've co-designed a HW mechanism for avoiding an excessive number of IRQs without sacrificing latency and designed and implemented a kernel mode driver to support it. Of course GPUs are quite a bit more complicated than that simple case.) [1]: Well, technically it's sending an MSI(-X) message. Just using traditional terminology for clarity. https://en.wikipedia.org/wiki/Message_Signaled_Interrupts#MSI-X https://en.wikipedia.org/wiki/Message_Signaled_Interrupts#MS...