I've found quite readable and relatively high level description of memory management and command submission in Linux drivers for Intel GPUs. http://blog.ffwll.

I've found quite readable and relatively high level description of memory management and command submission in Linux drivers for Intel GPUs. http://blog.ffwll.ch/2013/01/i915gem-crashcourse-overview.html http://blog.ffwll.ch/2013/01/i915gem-crashcourse-overview.ht... This (plus some low-level register poking) is the stuff you would need to implement in your GPU daemon. Actual generation of drawing commands, compilation of shaders etc is performed by applications. The overhead should be in the order of few context switches to the GPU server for each frame rendered by any application (in Linux it's few syscalls instead of context switches). Probably not terrible.