3 ms·

> Or is it at the OS layer instead of the assembly layer, with an OS syscall that defers to the GPU? Yes. Modern OSes don’t allow userspace code to directly ac

> Or is it at the OS layer instead of the assembly layer, with an OS syscall that defers to the GPU? Yes. Modern OSes don’t allow userspace code to directly access peripheral devices, GPUs included. Due to the enormous complexity of modern GPUs (GPUs typically have more transistors, consume more electricity, and have comparable amount of memory), the API surface is huge. On Windows, the native API surface is Direct3D. It has two large pieces, user-mode and kernel mode. The vendor-provided GPU driver is similarly split in two. User-mode parts implement HLSL compiler (Microsoft), another downstream compiler to compile DXBC into proprietary byte code GPUs actually run (vendors), implement other higher-level stuff like mipmap generation (vendors, at least for nVidia), and expose user-facing APIs, i.e. multiple versions of D3D (Microsoft). Kernel mode driver interfaces with actual hardware, the Microsoft’s vendor-agnostic part of that is in dxgkrnl.sys. Various GPU drivers often expose extra user-facing APIs in addition to Direct3D (CUDA, Vulkan, OpenGL, OpenCL), but all of them are optional.

nomel 6y ago
> Modern OSes don’t allow userspace code to directly access peripheral devices, GPUs included I was under the impression that user-space drivers (like Linux UIO) were given an address that they can mmap for direct reads/writes from/to the peripherals address space. Is this not "direct"?
- Const-me 6y ago
  > were given an address that they can mmap for direct reads/writes from/to the peripherals address space Kernel-mode drivers indeed do that under the hood, but the majority of GPU I/O bandwidth doesn’t go that way. With DMA, GPUs have full access to system memory. They have specialized piece of hardware (exposed to programmers as copy command queue in D3D12, or transfer queue in Vulcan) to move large blocks of data in either direction between system RAM and VRAM.
rrss 6y ago
> Modern OSes don’t allow userspace code to directly access peripheral devices, GPUs included This depends a lot on the OS and on the peripheral, but this is certainly not true across the board. RDMA verbs, for example, definitely involve userspace directly communicating with the NIC, with no kernel involvement. At least some of the AMD GPU graphics stacks support userspace submission to the GPU also, see http://www.hsafoundation.com/html/Content/Runtime/Topics/02_Core/queues.htm http://www.hsafoundation.com/html/Content/Runtime/Topics/02_.... This requires hardware support, like an MMU and ensuring bad submissions from userspace can't do bad things to other processes, but lots of hardware supports these things. The windows graphics stack does require going into the kernel to submit work to the GPU, but that isn't the case for all operating systems and all peripherals. somewhat related: GPUs started getting support for virtual memory in ~2006 or so [1] but the windows graphics stack was improved to actually use that capability in Windows 10, ~10 years later [2]. [1] https://www.anandtech.com/show/2116/2 https://www.anandtech.com/show/2116/2 [2] https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#WDDM_2.0 https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#W...
- Const-me 6y ago
  > this is certainly not true across the board. Interesting. > but the windows graphics stack was improved to actually use that capability in Windows 10 No, it happened much earlier, in WDDM 1.0 i.e. Windows Vista: https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#Virtualized_video_memory https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#V...
  - rrss 6y ago
    > No, it happened much earlier, in WDDM 1.0 WDDM 1.0 supported virtualized video memory, but it did it in software by patching command buffers to reference the new physical memory addresses. Not until WDDM 2.0 did WDDM actually support using the hardware virtual memory capabilities of GPUs. > A new memory model is implemented that gives each GPU a per-process virtual address space. Direct addressing of video memory is still supported by WDDMv2 for graphics hardware that requires it, but that is considered a legacy case. IHVs are expected to develop new hardware that supports virtual addressing. i.e. prior to WDDM 2.0, each process did not have a separate virtual address space on the GPU, and all hardware was assumed to use "direct" (i.e. physical) addressing of memory. https://docs.microsoft.com/en-us/windows-hardware/drivers/display/gpu-virtual-memory-in-wddm-2-0 https://docs.microsoft.com/en-us/windows-hardware/drivers/di... > Under Windows Display Driver Model (WDDM) v1.x, the device driver interface (DDI) is built such that graphics processing unit (GPU) engines are expected to reference memory through segment physical addresses. As segments are shared across applications and over committed, resources gets relocated through their lifetime and their assigned physical addresses change. This leads to the need to track memory references inside command buffers through allocation and patch location lists, and to patch those buffers with the correct physical memory reference before submission to a GPU engine > it is necessary to eliminate the need for the video memory manager to inspect and patch every command buffer before submission to a GPU engine. > To achieve this, WDDM v2 supports GPU virtual addressing

nomel 6y ago

> Modern OSes don’t allow userspace code to directly access peripheral devices, GPUs included I was under the impression that user-space drivers (like Linux UIO) were given an address that they can mmap for direct reads/writes from/to the peripherals address space. Is this not "direct"?

Const-me 6y ago
> were given an address that they can mmap for direct reads/writes from/to the peripherals address space Kernel-mode drivers indeed do that under the hood, but the majority of GPU I/O bandwidth doesn’t go that way. With DMA, GPUs have full access to system memory. They have specialized piece of hardware (exposed to programmers as copy command queue in D3D12, or transfer queue in Vulcan) to move large blocks of data in either direction between system RAM and VRAM.

rrss 6y ago

> Modern OSes don’t allow userspace code to directly access peripheral devices, GPUs included This depends a lot on the OS and on the peripheral, but this is certainly not true across the board. RDMA verbs, for example, definitely involve userspace directly communicating with the NIC, with no kernel involvement. At least some of the AMD GPU graphics stacks support userspace submission to the GPU also, see http://www.hsafoundation.com/html/Content/Runtime/Topics/02_Core/queues.htm http://www.hsafoundation.com/html/Content/Runtime/Topics/02_.... This requires hardware support, like an MMU and ensuring bad submissions from userspace can't do bad things to other processes, but lots of hardware supports these things. The windows graphics stack does require going into the kernel to submit work to the GPU, but that isn't the case for all operating systems and all peripherals. somewhat related: GPUs started getting support for virtual memory in ~2006 or so [1] but the windows graphics stack was improved to actually use that capability in Windows 10, ~10 years later [2]. [1] https://www.anandtech.com/show/2116/2 https://www.anandtech.com/show/2116/2 [2] https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#WDDM_2.0 https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#W...

Const-me 6y ago
> this is certainly not true across the board. Interesting. > but the windows graphics stack was improved to actually use that capability in Windows 10 No, it happened much earlier, in WDDM 1.0 i.e. Windows Vista: https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#Virtualized_video_memory https://en.wikipedia.org/wiki/Windows_Display_Driver_Model#V...
- rrss 6y ago
  > No, it happened much earlier, in WDDM 1.0 WDDM 1.0 supported virtualized video memory, but it did it in software by patching command buffers to reference the new physical memory addresses. Not until WDDM 2.0 did WDDM actually support using the hardware virtual memory capabilities of GPUs. > A new memory model is implemented that gives each GPU a per-process virtual address space. Direct addressing of video memory is still supported by WDDMv2 for graphics hardware that requires it, but that is considered a legacy case. IHVs are expected to develop new hardware that supports virtual addressing. i.e. prior to WDDM 2.0, each process did not have a separate virtual address space on the GPU, and all hardware was assumed to use "direct" (i.e. physical) addressing of memory. https://docs.microsoft.com/en-us/windows-hardware/drivers/display/gpu-virtual-memory-in-wddm-2-0 https://docs.microsoft.com/en-us/windows-hardware/drivers/di... > Under Windows Display Driver Model (WDDM) v1.x, the device driver interface (DDI) is built such that graphics processing unit (GPU) engines are expected to reference memory through segment physical addresses. As segments are shared across applications and over committed, resources gets relocated through their lifetime and their assigned physical addresses change. This leads to the need to track memory references inside command buffers through allocation and patch location lists, and to patch those buffers with the correct physical memory reference before submission to a GPU engine > it is necessary to eliminate the need for the video memory manager to inspect and patch every command buffer before submission to a GPU engine. > To achieve this, WDDM v2 supports GPU virtual addressing