The driver handles time slicing between processes, mapping virtual memory to real memory, etc. You're right that this is an actual consideration for programs.

The driver handles time slicing between processes, mapping virtual memory to real memory, etc. You're right that this is an actual consideration for programs. I don't know about CUDA/GPUGPU stuff, but for games, you need to manage residency of your resources, essentially telling the driver which resources (buffers/textures) are critical and can't be unloaded to make space for other apps, and which aren't critical.