4 ms·

But this feature is also why malloc on Windows can take a long time to return - seconds or more if you malloc too much. I recently bumped into this problem on t

by dahart 5y ago

But this feature is also why malloc on Windows can take a long time to return - seconds or more if you malloc too much. I recently bumped into this problem on the GPU. Since GPU memory in the Windows Display Driver Model requires it to be backed by the host machine’s virtual memory, a malloc on the GPU when running low on space can end up searching & even defragging virtual memory to satisfy the request. Crazy! Just asking for RAM can cause you to land in heavy disk swap. I’ve seen cases of cudaMalloc taking minutes because of how Windows handles allocation, and the same is true of CPU mallocs.

ayende 5y ago
What would you rather have, in this case? Assuming that the work needs to be done? You can either do that on allocation (which you can predict, batch, etc) or you can do that when you need the next page to be used. Which is far harder to predict and work with.
bscphil 5y ago
> Since GPU memory in the Windows Display Driver Model requires it to be backed by the host machine’s virtual memory Wow, this seems really unintuitive to me, especially on a OS that doesn't overcommit. Is it as unintuitive as it sounds, or is there a good reason for this?
- dahart 5y ago
  There are good reasons, not all of which I know or understand, but my basic mental model is that the backing is required for various scenarios where virtual memory might need to get relocated, either in response to a malloc or to something else. I’m told it’s complicated by having the GPU drive the display, and by scenarios involving multiple GPUs in a system. I’m guessing the description in the WDDM docs about DDI might be relevant to what’s happening under the covers in WDDM2? I’m not sure… :) https://docs.microsoft.com/en-us/windows-hardware/drivers/display/gpu-virtual-memory-in-wddm-2-0 https://docs.microsoft.com/en-us/windows-hardware/drivers/di...
- Const-me 5y ago
  > is there a good reason for this? Multitasking. User can easily alt+tab into another program which also uses GPU. When many processes are using GPU concurrently, it's possible their combined VRAM use exceeds the amount of physical memory available on the GPU.
foxfluff 5y ago
That sounds like poor implementation rather than something inherent.
varispeed 5y ago
I have a faint memory of writing custom memory manager and allocating memory in bulk upon program start. But it was still a mess. Eventually had to periodically pause and defragment the memory allocations, rewriting pointers and so on. Well, at least it was fast.

by dahart 5y ago

ayende 5y ago

What would you rather have, in this case? Assuming that the work needs to be done? You can either do that on allocation (which you can predict, batch, etc) or you can do that when you need the next page to be used. Which is far harder to predict and work with.

bscphil 5y ago

> Since GPU memory in the Windows Display Driver Model requires it to be backed by the host machine’s virtual memory Wow, this seems really unintuitive to me, especially on a OS that doesn't overcommit. Is it as unintuitive as it sounds, or is there a good reason for this?

dahart 5y ago
There are good reasons, not all of which I know or understand, but my basic mental model is that the backing is required for various scenarios where virtual memory might need to get relocated, either in response to a malloc or to something else. I’m told it’s complicated by having the GPU drive the display, and by scenarios involving multiple GPUs in a system. I’m guessing the description in the WDDM docs about DDI might be relevant to what’s happening under the covers in WDDM2? I’m not sure… :) https://docs.microsoft.com/en-us/windows-hardware/drivers/display/gpu-virtual-memory-in-wddm-2-0 https://docs.microsoft.com/en-us/windows-hardware/drivers/di...
Const-me 5y ago
> is there a good reason for this? Multitasking. User can easily alt+tab into another program which also uses GPU. When many processes are using GPU concurrently, it's possible their combined VRAM use exceeds the amount of physical memory available on the GPU.

foxfluff 5y ago

That sounds like poor implementation rather than something inherent.

varispeed 5y ago

I have a faint memory of writing custom memory manager and allocating memory in bulk upon program start. But it was still a mess. Eventually had to periodically pause and defragment the memory allocations, rewriting pointers and so on. Well, at least it was fast.