4 ms·

Could compositors be optimised to eliminate the extra frame of lag in the case where every window on-screen is being displayed “directly” by invisibly switching

Could compositors be optimised to eliminate the extra frame of lag in the case where every window on-screen is being displayed “directly” by invisibly switching to a mode that maps each scanline and pixel column to a window’s framebuffer - and non-client areas to the window-manager’s UI buffer which is directly read by the monitor signal generator. While this would mean transparency effects wouldn’t work, it could supported with some special-casing. Basically a framebuffer-less hardware compositor. I think rendering windows to 3D deformable meshes [makes for cool demos](https://youtu.be/USedxVrU2Ko https://youtu.be/USedxVrU2Ko) but in practice we just don’t use it for anything besides window open/close animations. I had to use a monitor running at 30Hz for a while (4K over HDMI 1.4) and while that was bad enough, the compositor’s lag meant all window contents had an extra (unnecessary IMO) delay of 33ms. Add on to that normal monitor input lag. We’ll probably all shift to 120Hz w/ variable-rate refreshing as a new baseline standard over the next 10 years as Apple seems to be heading in that direction - at 120Hz the lag of the compositor would be acceptable - but I’m worried lazy that graphics devs are going to use that as an excuse to add another frame of latency...

modeless 7y ago
> Could compositors be optimised to eliminate the extra frame of lag in the case where every window on-screen is being displayed “directly” by invisibly switching to a mode that maps each scanline and pixel column to a window’s framebuffer Yes. This concept is called hardware overlays and there are varying levels of support for it in different GPUs and compositors. There are tradeoffs. Using multiple hardware overlays may cost extra power and/or memory bandwidth, the number of supported overlays may be very limited, alpha blending may not be supported, and the transforms that can be applied to overlays may be very limited. The extremely hardware specific nature of the restrictions and the lack of good APIs exposing overlays means they get much less use than they should.
- heavenlyblue 7y ago
  Technically if your pixel operation is commutative and reversible (as could be with alpha blending), you could store a buffer of pixels at the current position, then undo the previous pixel by current window at that buffer and then re-apply the operation with the new pixel value, and then directly send this to the display? Am I missing something apart from the fact that alpha blending is not actually commutative and/or reversible; and the fact that nobody implemented this yet?

modeless 7y ago

> Could compositors be optimised to eliminate the extra frame of lag in the case where every window on-screen is being displayed “directly” by invisibly switching to a mode that maps each scanline and pixel column to a window’s framebuffer Yes. This concept is called hardware overlays and there are varying levels of support for it in different GPUs and compositors. There are tradeoffs. Using multiple hardware overlays may cost extra power and/or memory bandwidth, the number of supported overlays may be very limited, alpha blending may not be supported, and the transforms that can be applied to overlays may be very limited. The extremely hardware specific nature of the restrictions and the lack of good APIs exposing overlays means they get much less use than they should.

heavenlyblue 7y ago
Technically if your pixel operation is commutative and reversible (as could be with alpha blending), you could store a buffer of pixels at the current position, then undo the previous pixel by current window at that buffer and then re-apply the operation with the new pixel value, and then directly send this to the display? Am I missing something apart from the fact that alpha blending is not actually commutative and/or reversible; and the fact that nobody implemented this yet?