3 ms·

I read that document quickly. I think it can only work through pinning, because it seems to rely on GPU bus mastering for accessing pages that are dedicated fo

by vardump 10y ago

I read that document quickly. I think it can only work through pinning, because it seems to rely on GPU bus mastering for accessing pages that are dedicated for graphics. So you have 32 GB RAM and say 8 GB GPU RAM. Imagine you have a program that has 100 GB of graphics assets without built in mechanism to guess which subset of assets might be required for current scene. The program needs about 50 MB in any given frame -- in other words it has 50 MB working set. As an operating system, which assets are you going to keep in memory? Now imagine you have multiple programs running concurrently, each having 100 GB of assets. Each app has that same 50 MB working set. How can the system handle this situation efficiently (or at all!) if the pages need to be pinned to RAM? If you can have true on demand GPU paging, all of these apps need to only swap their current working set of data. User would not perceive any delay when switching from app to app. She could even display all of them in the same time without any issues. If not, the system would grind to halt.

dogma1138 10y ago
It kinda varies if you go the GPUMMU or the IOMMU paths, but under WDDM it seems that assets can end up being on disk. https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/the-role-of-the-windows-display-driver-model-in-the-dwm/ https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/t... "In the event that video memory allocation is required, and both video memory and system memory are full, the WDDM and the overall virtual memory system will then turn to disk for video memory surfaces. This is an extremely unusual case, and the performance would suffer dearly in that case, but the point is that the system is sufficiently robust to allow this to occur and for the application to reliably continue." The only issue here is that as far as i can understand you can't really choose how this is done very specifically. When you allocate memory the driver pretty much takes over, if you want granular over memory allocation you pretty much have to go through the GPUMMU path which means each process has a separate GPU and CPU address space so while you can control what you store in GPU memory and what you store in System memory I still don't see a way to control mapping an asset to disk specifically other than it being an edge case of you running out of system memory which results in the page file being used.
Unklejoe 10y ago
I guess it depends on if the IOMMU has a fault interrupt that can be acted upon. If it does, then it can probably be done. However, I'm not sure if the OS will handle this or not. When the OS runs out of RAM, it can swap pages to disk. If this page also has a mapping in an IOMMU, it can invalidate the mapping there as well. Then, when the device attempts to touch the page, the IOMMU faults and the CPU would swap the page back in. I'm not sure if this is possible, but this is the route I'd expect something like this to follow.
- vardump 10y ago
  > I guess it depends on if the IOMMU has a fault interrupt that can be acted upon. If it does, then it can probably be done. However, I'm not sure if the OS will handle this or not. Interesting idea. I'd also like to know if IOMMU faults can be acted upon. It might require protocol support between the bus and hardware device (GPU) [1], to tell it the page is not currently present. And a way to tell GPU once the page is available. As far as I can see this kind of mechanism would require one CPU interrupt per GPU fault. That might be too inefficient. [1]: Edit: It's indeed possible to handle, if the device supports "PCI-SIG PCIe Address Translation Services (ATS) Page Request Interface (PRI) extension".

by vardump 10y ago

dogma1138 10y ago

It kinda varies if you go the GPUMMU or the IOMMU paths, but under WDDM it seems that assets can end up being on disk. https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/the-role-of-the-windows-display-driver-model-in-the-dwm/ https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/t... "In the event that video memory allocation is required, and both video memory and system memory are full, the WDDM and the overall virtual memory system will then turn to disk for video memory surfaces. This is an extremely unusual case, and the performance would suffer dearly in that case, but the point is that the system is sufficiently robust to allow this to occur and for the application to reliably continue." The only issue here is that as far as i can understand you can't really choose how this is done very specifically. When you allocate memory the driver pretty much takes over, if you want granular over memory allocation you pretty much have to go through the GPUMMU path which means each process has a separate GPU and CPU address space so while you can control what you store in GPU memory and what you store in System memory I still don't see a way to control mapping an asset to disk specifically other than it being an edge case of you running out of system memory which results in the page file being used.

Unklejoe 10y ago

I guess it depends on if the IOMMU has a fault interrupt that can be acted upon. If it does, then it can probably be done. However, I'm not sure if the OS will handle this or not. When the OS runs out of RAM, it can swap pages to disk. If this page also has a mapping in an IOMMU, it can invalidate the mapping there as well. Then, when the device attempts to touch the page, the IOMMU faults and the CPU would swap the page back in. I'm not sure if this is possible, but this is the route I'd expect something like this to follow.

vardump 10y ago
> I guess it depends on if the IOMMU has a fault interrupt that can be acted upon. If it does, then it can probably be done. However, I'm not sure if the OS will handle this or not. Interesting idea. I'd also like to know if IOMMU faults can be acted upon. It might require protocol support between the bus and hardware device (GPU) [1], to tell it the page is not currently present. And a way to tell GPU once the page is available. As far as I can see this kind of mechanism would require one CPU interrupt per GPU fault. That might be too inefficient. [1]: Edit: It's indeed possible to handle, if the device supports "PCI-SIG PCIe Address Translation Services (ATS) Page Request Interface (PRI) extension".