3 ms·

> You can pull assets from RAM somewhat transparently already (if you want to do it well you need to optimize it further, but to some extent NVIDIA and AMD do q

by vardump 10y ago

> You can pull assets from RAM somewhat transparently already (if you want to do it well you need to optimize it further, but to some extent NVIDIA and AMD do quite a bit of optimization in the driver too), shared page table between the CPU and GPU is also already in place under WDDM. Can GPU page table entry point to non-present page(s)? Or does it only work for "pinned pages" [1], that cannot paged out of RAM? If it doesn't require pinning, then what prevents from mmapping assets on the disk even today? If, however, it does require pinning, then John Carmack has a damn good point. [1]: https://en.wikipedia.org/wiki/Virtual_memory#Pinned_pages https://en.wikipedia.org/wiki/Virtual_memory#Pinned_pages

dogma1138 10y ago
IIRC the driver handles the pinning, there are basically 3 types of video memory under WDDM, Dedicated Video Memory (on-card) Dedicated System Memory (memory that is assigned to the GPU only usually via BIOS configuration and is off limit to the OS) and Shared System Memory (A part of the virtual system memory allocated for the GPU), so in theory if you only use Dedicated Video Memory and Shared System Memory you could be pulling off data from the on-disk page file, but it's not like you can map a specific asset (say a texture file) on disk to your video memory directly. WDDM also limited the volume and commit sizes based on some "arbitrary" limits that MSFT set out (IIRC it's something like system memory / 2 or something silly like that), there's a bit more silliness that for example if the limit of the max memory available for graphics is 2GB you can't commit 3GB but you can do 3x1GB just fine. I'm pretty sure atm WDDM/Vendor Display Driver pin all pages to RAM only so it won't end up in the page file, TBH I haven't had a page file on my system for a long long time windows doesn't use SSD's for paging unless it really has too and considering I haven't been using a system with less than 32GB of RAM for the past 5 years I never had issues with it. Here is a somewhat decent MSFT doc about memory management under WDDM (this is not for WDDM2.0 so it's not that upto date I'm guessing) http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4bae-9fde-d599bac8184a/GraphicsMemory.doc http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4... P.S. Apologies if I used any terminology incorrectly this is stretching both my knowledge and recollection regarding this subject, it's also more or less limited to how GPU/Graphics are handled within Windows.
- vardump 10y ago
  I read that document quickly. I think it can only work through pinning, because it seems to rely on GPU bus mastering for accessing pages that are dedicated for graphics. So you have 32 GB RAM and say 8 GB GPU RAM. Imagine you have a program that has 100 GB of graphics assets without built in mechanism to guess which subset of assets might be required for current scene. The program needs about 50 MB in any given frame -- in other words it has 50 MB working set. As an operating system, which assets are you going to keep in memory? Now imagine you have multiple programs running concurrently, each having 100 GB of assets. Each app has that same 50 MB working set. How can the system handle this situation efficiently (or at all!) if the pages need to be pinned to RAM? If you can have true on demand GPU paging, all of these apps need to only swap their current working set of data. User would not perceive any delay when switching from app to app. She could even display all of them in the same time without any issues. If not, the system would grind to halt.
  - dogma1138 10y ago
    It kinda varies if you go the GPUMMU or the IOMMU paths, but under WDDM it seems that assets can end up being on disk. https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/the-role-of-the-windows-display-driver-model-in-the-dwm/ https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/t... "In the event that video memory allocation is required, and both video memory and system memory are full, the WDDM and the overall virtual memory system will then turn to disk for video memory surfaces. This is an extremely unusual case, and the performance would suffer dearly in that case, but the point is that the system is sufficiently robust to allow this to occur and for the application to reliably continue." The only issue here is that as far as i can understand you can't really choose how this is done very specifically. When you allocate memory the driver pretty much takes over, if you want granular over memory allocation you pretty much have to go through the GPUMMU path which means each process has a separate GPU and CPU address space so while you can control what you store in GPU memory and what you store in System memory I still don't see a way to control mapping an asset to disk specifically other than it being an edge case of you running out of system memory which results in the page file being used.
  - 10y ago
exDM69 10y ago
> Can GPU page table entry point to non-present page(s)? Yes, new GPUs allow you to do this. This feature is called sparse textures / buffers in OpenGL (GL_ARB_sparse_texture and GL_EXT_sparse_texture2) and Vulkan (optional feature in Vulkan 1.0 core) or tiled resources in D3D. This allows you to leave textures (or buffers) partially non-resident (accesses to which are safe but results undefined) and allow you to detect when accessing a non-resident region (EXT_sparse_texture2) so that you can write a fallback path in the shader (lookup lower mip level and/or somehow tell the CPU that the page will be required for the next frame). The OpenGL extensions are a bit restrictive, but Vulkan/D3D12 allow much greater control (such as sharing pages between textures or repeating the same page inside a texture). Hardware support for this is not ubiquitous at the moment, but should improve as time goes on. This feature is somewhat orthogonal to pinned pages and WDDM residency magic (which is afaik more oriented to switching between processes), hopefully it will get more unified in the future.

by vardump 10y ago

dogma1138 10y ago

IIRC the driver handles the pinning, there are basically 3 types of video memory under WDDM, Dedicated Video Memory (on-card) Dedicated System Memory (memory that is assigned to the GPU only usually via BIOS configuration and is off limit to the OS) and Shared System Memory (A part of the virtual system memory allocated for the GPU), so in theory if you only use Dedicated Video Memory and Shared System Memory you could be pulling off data from the on-disk page file, but it's not like you can map a specific asset (say a texture file) on disk to your video memory directly. WDDM also limited the volume and commit sizes based on some "arbitrary" limits that MSFT set out (IIRC it's something like system memory / 2 or something silly like that), there's a bit more silliness that for example if the limit of the max memory available for graphics is 2GB you can't commit 3GB but you can do 3x1GB just fine. I'm pretty sure atm WDDM/Vendor Display Driver pin all pages to RAM only so it won't end up in the page file, TBH I haven't had a page file on my system for a long long time windows doesn't use SSD's for paging unless it really has too and considering I haven't been using a system with less than 32GB of RAM for the past 5 years I never had issues with it. Here is a somewhat decent MSFT doc about memory management under WDDM (this is not for WDDM2.0 so it's not that upto date I'm guessing) http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4bae-9fde-d599bac8184a/GraphicsMemory.doc http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4... P.S. Apologies if I used any terminology incorrectly this is stretching both my knowledge and recollection regarding this subject, it's also more or less limited to how GPU/Graphics are handled within Windows.

vardump 10y ago
I read that document quickly. I think it can only work through pinning, because it seems to rely on GPU bus mastering for accessing pages that are dedicated for graphics. So you have 32 GB RAM and say 8 GB GPU RAM. Imagine you have a program that has 100 GB of graphics assets without built in mechanism to guess which subset of assets might be required for current scene. The program needs about 50 MB in any given frame -- in other words it has 50 MB working set. As an operating system, which assets are you going to keep in memory? Now imagine you have multiple programs running concurrently, each having 100 GB of assets. Each app has that same 50 MB working set. How can the system handle this situation efficiently (or at all!) if the pages need to be pinned to RAM? If you can have true on demand GPU paging, all of these apps need to only swap their current working set of data. User would not perceive any delay when switching from app to app. She could even display all of them in the same time without any issues. If not, the system would grind to halt.
- dogma1138 10y ago
  It kinda varies if you go the GPUMMU or the IOMMU paths, but under WDDM it seems that assets can end up being on disk. https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/the-role-of-the-windows-display-driver-model-in-the-dwm/ https://blogs.msdn.microsoft.com/greg_schechter/2006/04/02/t... "In the event that video memory allocation is required, and both video memory and system memory are full, the WDDM and the overall virtual memory system will then turn to disk for video memory surfaces. This is an extremely unusual case, and the performance would suffer dearly in that case, but the point is that the system is sufficiently robust to allow this to occur and for the application to reliably continue." The only issue here is that as far as i can understand you can't really choose how this is done very specifically. When you allocate memory the driver pretty much takes over, if you want granular over memory allocation you pretty much have to go through the GPUMMU path which means each process has a separate GPU and CPU address space so while you can control what you store in GPU memory and what you store in System memory I still don't see a way to control mapping an asset to disk specifically other than it being an edge case of you running out of system memory which results in the page file being used.
- 10y ago

exDM69 10y ago

> Can GPU page table entry point to non-present page(s)? Yes, new GPUs allow you to do this. This feature is called sparse textures / buffers in OpenGL (GL_ARB_sparse_texture and GL_EXT_sparse_texture2) and Vulkan (optional feature in Vulkan 1.0 core) or tiled resources in D3D. This allows you to leave textures (or buffers) partially non-resident (accesses to which are safe but results undefined) and allow you to detect when accessing a non-resident region (EXT_sparse_texture2) so that you can write a fallback path in the shader (lookup lower mip level and/or somehow tell the CPU that the page will be required for the next frame). The OpenGL extensions are a bit restrictive, but Vulkan/D3D12 allow much greater control (such as sharing pages between textures or repeating the same page inside a texture). Hardware support for this is not ubiquitous at the moment, but should improve as time goes on. This feature is somewhat orthogonal to pinned pages and WDDM residency magic (which is afaik more oriented to switching between processes), hopefully it will get more unified in the future.