4 ms·

[The GPU doesn't have DMA to the disk or RAM it has to go through the CPU to access it which causes quite a delay.] How come? If it's on the PCI bus it should

[The GPU doesn't have DMA to the disk or RAM it has to go through the CPU to access it which causes quite a delay.] How come? If it's on the PCI bus it should be able to bus master its way into the system RAM, no? Or was this capability disabled to prevent bad graphics code from being able to exploit the system or something?

vardump 10y ago
How would GPU or GPU drivers know how to issue DMA transfer and know when DMA is ready on another piece of hardware? Those things are specific to a piece of hardware. Besides there's a small detail called IOMMU that prevents PCI(-e) devices from writing or reading wherever they please.
- dogma1138 10y ago
  On windows GPU memory is already virtualized. You can pull assets from RAM somewhat transparently already (if you want to do it well you need to optimize it further, but to some extent NVIDIA and AMD do quite a bit of optimization in the driver too), shared page table between the CPU and GPU is also already in place under WDDM. So adding a disk based page file to this won't be so hard, it could more or less work the same way as the page file currently works. I'm still not sure if this is actually needed, GPU's already come with stupid amount of RAM today, i have 24GB of GDDR5 in my system, and even mid range cards today will not come with less than 6GB of RAM.
  - vardump 10y ago
    > You can pull assets from RAM somewhat transparently already (if you want to do it well you need to optimize it further, but to some extent NVIDIA and AMD do quite a bit of optimization in the driver too), shared page table between the CPU and GPU is also already in place under WDDM. Can GPU page table entry point to non-present page(s)? Or does it only work for "pinned pages" [1], that cannot paged out of RAM? If it doesn't require pinning, then what prevents from mmapping assets on the disk even today? If, however, it does require pinning, then John Carmack has a damn good point. [1]: https://en.wikipedia.org/wiki/Virtual_memory#Pinned_pages https://en.wikipedia.org/wiki/Virtual_memory#Pinned_pages
    - dogma1138 10y ago
      IIRC the driver handles the pinning, there are basically 3 types of video memory under WDDM, Dedicated Video Memory (on-card) Dedicated System Memory (memory that is assigned to the GPU only usually via BIOS configuration and is off limit to the OS) and Shared System Memory (A part of the virtual system memory allocated for the GPU), so in theory if you only use Dedicated Video Memory and Shared System Memory you could be pulling off data from the on-disk page file, but it's not like you can map a specific asset (say a texture file) on disk to your video memory directly. WDDM also limited the volume and commit sizes based on some "arbitrary" limits that MSFT set out (IIRC it's something like system memory / 2 or something silly like that), there's a bit more silliness that for example if the limit of the max memory available for graphics is 2GB you can't commit 3GB but you can do 3x1GB just fine. I'm pretty sure atm WDDM/Vendor Display Driver pin all pages to RAM only so it won't end up in the page file, TBH I haven't had a page file on my system for a long long time windows doesn't use SSD's for paging unless it really has too and considering I haven't been using a system with less than 32GB of RAM for the past 5 years I never had issues with it. Here is a somewhat decent MSFT doc about memory management under WDDM (this is not for WDDM2.0 so it's not that upto date I'm guessing) http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4bae-9fde-d599bac8184a/GraphicsMemory.doc http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4... P.S. Apologies if I used any terminology incorrectly this is stretching both my knowledge and recollection regarding this subject, it's also more or less limited to how GPU/Graphics are handled within Windows.
      - vardump 10y ago
        I read that document quickly. I think it can only work through pinning, because it seems to rely on GPU bus mastering for accessing pages that are dedicated for graphics. So you have 32 GB RAM and say 8 GB GPU RAM. Imagine you have a program that has 100 GB of graphics assets without built in mechanism to guess which subset of assets might be required for current scene. The program needs about 50 MB in any given frame -- in other words it has 50 MB working set. As an operating system, which assets are you going to keep in memory? Now imagine you have multiple programs running concurrently, each having 100 GB of assets. Each app has that same 50 MB working set. How can the system handle this situation efficiently (or at all!) if the pages need to be pinned to RAM? If you can have true on demand GPU paging, all of these apps need to only swap their current working set of data. User would not perceive any delay when switching from app to app. She could even display all of them in the same time without any issues. If not, the system would grind to halt.
- Unklejoe 10y ago
  Of course, something like this would need driver support. What I was getting at was the fact that there should be nothing physically preventing them from implementing DMA support, so I was wondering why they didn't already support it. I’m not familiar with GPUs, so I assumed that this is how all large transfers between system RAM and the GPU worked. I can only speak for the Linux context, but the IOMMU isn’t an issue. You can just allocate memory with the DMA API (dma_alloc_coherent) which will automatically populate the IOMMU tables (if required), pin the pages, and return you a PCI bus address as well as a kernel virtual address which both correspond to the same chunk of physical memory. Or, you can map an existing buffer in page by page using the dma_map routines (I forget the names). Now, you have a shared pool of memory which can be accessed by both devices at the same time. The coherency fabric (if one exists) will handle all synchronization automatically, though this can be a bottleneck sometimes. If the CPU isn’t cache coherent, then the pages get marked as no-cache in the kernel PTEs so that any read from the CPU side pulls straight from memory. Passing “messages” can be accomplished by an external notification like an interrupt or something. You can then even map this buffer into a user space program. I’m sure there are some security concerns with this approach though. More complicated things like device to device transfers (GPU to and from disk) would have to be arbitrated by the CPU, but I see no reason that the CPU would actually have to do the copy itself. Why couldn’t the CPU just provide the GPU with the PCI bus address of the disk controller which should be written to? If the GPU wanted to write to the disk, the CPU would initiate a transfer to disk, but before writing the actual data, you pass the destination PCI address to the GPU and let it write the data. Then, the CPU can resume doing whatever it has to do while this happens in the background. Just brainstorming here.

vardump 10y ago

How would GPU or GPU drivers know how to issue DMA transfer and know when DMA is ready on another piece of hardware? Those things are specific to a piece of hardware. Besides there's a small detail called IOMMU that prevents PCI(-e) devices from writing or reading wherever they please.

dogma1138 10y ago
On windows GPU memory is already virtualized. You can pull assets from RAM somewhat transparently already (if you want to do it well you need to optimize it further, but to some extent NVIDIA and AMD do quite a bit of optimization in the driver too), shared page table between the CPU and GPU is also already in place under WDDM. So adding a disk based page file to this won't be so hard, it could more or less work the same way as the page file currently works. I'm still not sure if this is actually needed, GPU's already come with stupid amount of RAM today, i have 24GB of GDDR5 in my system, and even mid range cards today will not come with less than 6GB of RAM.
- vardump 10y ago
  > You can pull assets from RAM somewhat transparently already (if you want to do it well you need to optimize it further, but to some extent NVIDIA and AMD do quite a bit of optimization in the driver too), shared page table between the CPU and GPU is also already in place under WDDM. Can GPU page table entry point to non-present page(s)? Or does it only work for "pinned pages" [1], that cannot paged out of RAM? If it doesn't require pinning, then what prevents from mmapping assets on the disk even today? If, however, it does require pinning, then John Carmack has a damn good point. [1]: https://en.wikipedia.org/wiki/Virtual_memory#Pinned_pages https://en.wikipedia.org/wiki/Virtual_memory#Pinned_pages
  - dogma1138 10y ago
    IIRC the driver handles the pinning, there are basically 3 types of video memory under WDDM, Dedicated Video Memory (on-card) Dedicated System Memory (memory that is assigned to the GPU only usually via BIOS configuration and is off limit to the OS) and Shared System Memory (A part of the virtual system memory allocated for the GPU), so in theory if you only use Dedicated Video Memory and Shared System Memory you could be pulling off data from the on-disk page file, but it's not like you can map a specific asset (say a texture file) on disk to your video memory directly. WDDM also limited the volume and commit sizes based on some "arbitrary" limits that MSFT set out (IIRC it's something like system memory / 2 or something silly like that), there's a bit more silliness that for example if the limit of the max memory available for graphics is 2GB you can't commit 3GB but you can do 3x1GB just fine. I'm pretty sure atm WDDM/Vendor Display Driver pin all pages to RAM only so it won't end up in the page file, TBH I haven't had a page file on my system for a long long time windows doesn't use SSD's for paging unless it really has too and considering I haven't been using a system with less than 32GB of RAM for the past 5 years I never had issues with it. Here is a somewhat decent MSFT doc about memory management under WDDM (this is not for WDDM2.0 so it's not that upto date I'm guessing) http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4bae-9fde-d599bac8184a/GraphicsMemory.doc http://download.microsoft.com/download/9/c/5/9c5b2167-8017-4... P.S. Apologies if I used any terminology incorrectly this is stretching both my knowledge and recollection regarding this subject, it's also more or less limited to how GPU/Graphics are handled within Windows.
    - vardump 10y ago
      I read that document quickly. I think it can only work through pinning, because it seems to rely on GPU bus mastering for accessing pages that are dedicated for graphics. So you have 32 GB RAM and say 8 GB GPU RAM. Imagine you have a program that has 100 GB of graphics assets without built in mechanism to guess which subset of assets might be required for current scene. The program needs about 50 MB in any given frame -- in other words it has 50 MB working set. As an operating system, which assets are you going to keep in memory? Now imagine you have multiple programs running concurrently, each having 100 GB of assets. Each app has that same 50 MB working set. How can the system handle this situation efficiently (or at all!) if the pages need to be pinned to RAM? If you can have true on demand GPU paging, all of these apps need to only swap their current working set of data. User would not perceive any delay when switching from app to app. She could even display all of them in the same time without any issues. If not, the system would grind to halt.
Unklejoe 10y ago
Of course, something like this would need driver support. What I was getting at was the fact that there should be nothing physically preventing them from implementing DMA support, so I was wondering why they didn't already support it. I’m not familiar with GPUs, so I assumed that this is how all large transfers between system RAM and the GPU worked. I can only speak for the Linux context, but the IOMMU isn’t an issue. You can just allocate memory with the DMA API (dma_alloc_coherent) which will automatically populate the IOMMU tables (if required), pin the pages, and return you a PCI bus address as well as a kernel virtual address which both correspond to the same chunk of physical memory. Or, you can map an existing buffer in page by page using the dma_map routines (I forget the names). Now, you have a shared pool of memory which can be accessed by both devices at the same time. The coherency fabric (if one exists) will handle all synchronization automatically, though this can be a bottleneck sometimes. If the CPU isn’t cache coherent, then the pages get marked as no-cache in the kernel PTEs so that any read from the CPU side pulls straight from memory. Passing “messages” can be accomplished by an external notification like an interrupt or something. You can then even map this buffer into a user space program. I’m sure there are some security concerns with this approach though. More complicated things like device to device transfers (GPU to and from disk) would have to be arbitrated by the CPU, but I see no reason that the CPU would actually have to do the copy itself. Why couldn’t the CPU just provide the GPU with the PCI bus address of the disk controller which should be written to? If the GPU wanted to write to the disk, the CPU would initiate a transfer to disk, but before writing the actual data, you pass the destination PCI address to the GPU and let it write the data. Then, the CPU can resume doing whatever it has to do while this happens in the background. Just brainstorming here.