4 ms·

There isn't much of a difference between a GUI toolkit you'd find in a desktop application and the GUI framework you'd see in a game - the most likely differenc

There isn't much of a difference between a GUI toolkit you'd find in a desktop application and the GUI framework you'd see in a game - the most likely difference will be that the game GUI will be redrawn every frame whereas the desktop GUI wont (and there are game GUI frameworks that cache their output to avoid redrawing the entire widget tree every frame). The difference when it comes to why games can be snappier is that games are "allowed" to bypass most of the layers and cruft that exists between the user and the hardware, including the compositor that the linked article is talking about (in Windows at least). Fortunately in Linux with Xorg you can get stuff on screen as fast as the code can ask for it, as long as you are not using a compositor (so you can even play games in a window with no additional lag!). Hopefully the Wayland mania wont get to kill Xorg since yet another issue Wayland has and X11 doesn't is that with Wayland you are forced in a composition model.

_pmf_ 9y ago
> the most likely difference will be that the game GUI will be redrawn every frame whereas the desktop GUI wont (and there are game GUI frameworks that cache their output to avoid redrawing the entire widget tree every frame). Modern UI toolkits (WPF, QML, JavaFX) operate on a scene graph, so they work exactly the same. Android is slowly catching up; it's a disgusting mix of the worst of both worlds.
Jasper_ 9y ago
The reason that apps on Windows can bypass the compositor is that you can use them as the scanout buffer directly on fullscreen. On Linux (both Xorg and Wayland), this same exact behavior is supported with a compositor. For strange legacy reasons, it's known as "fullscreen unredirection". If you're running windowed on all three OSes, you see the same compositor latency.
- badsectoracula 9y ago
  Note that on Linux you are not forced to use a compositor, personally i do not use one and so i do not have any such penalty when running games in windowed mode.
AnIdiotOnTheNet 9y ago
I'll take a frame of latency and Wayland's not-completely-insane model over X any day.
- cat199 9y ago
  yet strangely, wayland removes the frames-of-latency known as 'remote windowing' in it's quest for 'improvement'..
  - AnIdiotOnTheNet 9y ago
    Because it vastly simplified many things, and oh look, compositors have managed to add remoting back in somehow without it! I mean christ, windows has no support for remote windowing in it's API and still has RDP, even for single windows. Linux people just love to complain about any change that actually makes the system better because then they can't feel quite as elitist for using it.
nhaehnle 9y ago
The funny thing is that there is no technical reason at all for compositing to have worse latency, even for games. Think about the actual operations that are involved. You certainly never want to render directly into the front-buffer (you'd end up scanning out partially rendered scenes). So you render to a back buffer. Which you then blit to the front buffer (assuming you're in windowed mode; in full-screen mode the compositor goes out of the way anyway and lets you just flip). The only difference between the various modes of operation is who does that final blit. In plain X, it's the X server. In Wayland, it's the Wayland compositor. In X with a compositor, it's the compositor. Now granted, some compositors might be silly and re-composite the whole screen each frame, but clearly that can be avoided in most cases. Depending on the setup, there can also be some issues with the scheduling of the game's jobs vs. the compositor's jobs on the GPU. Valve are working on this currently for VR, since the problem is much more noticeable there -- clearly it can be fixed one way or another (on Radeon GPUs you could do a blit via asynchronous compute if need be, for example), but note that compositing actually doesn't change this issue (since the X server's jobs also need to compete with the game's jobs for scheduling time). So if compositing has worse latency, it's because nobody has cared enough to polish the critical paths. Conversely, compositing clearly does have advantages in overall image quality. So why not fix the (entirely fixable) technical problems with compositing?
- badsectoracula 9y ago
  There is a very good practical reason why the compositor is in no position to fix that even if it could theoretically be possible. A major source for the compositor latency (or actually, the increased response time you get with a compositor) is that the "render to back buffer" (ie. compositor's texture, at the best case) and the "blit to the front buffer" (which happens by the compositor by drawing the window geometry) do not happen at the same time. From a technical perspective it is perfectly possible for a compositor to create a tight integration between a program and the compositor itself: simply synchronize the program's updates with the compositor updates. Every time the program says "i'm done drawing" (either via an explicit notification to the compositor, via glXSwapBuffers or whatever), issue a screen update. The problem however here is the compositor has to take into account multiple windows from multiple programs so you cannot have a single window dictating the compositor updates. Imagine for example two windows with animations running, one at 100fps and another at 130fps. Depending which window is active (assuming that the compositor syncs itself with the active window), it would affect the perception of other window's updates (since what the user will see will be at the rate of the foreground window's update rate). Moreover, beyond just the perception, it will also affect the other windows' animation loops themselves - if a background window finishes drawing itself and notifies the compositor while the compositor is in the middle of an update, the background window will have to wait until the update is finished - thus having the foreground window indirectly also affect the animation loops of the background windows. This can be avoided through triple buffering, but that introduces an extra frame of latency - at least for background windows. So to avoid the above problems, what all compositors do is to decouple window update notifications from screen updates and instead perform the screen updates at some predefined interval - usually the monitor refresh rate and they do the updates synchronized to it. However that creates the increased response time you get with the compositor being on a few milliseconds behind the user's actions, with the most common example would be window manipulation like resizing and moving windows lagging behind the mouse cursor (which is drawn by the GPU directly, thus bypassing the compositor). Hence the linked article recommending a 144Hz monitor to avoid this, although this is just a workaround that makes the problem less visible but doesn't really solve it.

_pmf_ 9y ago

> the most likely difference will be that the game GUI will be redrawn every frame whereas the desktop GUI wont (and there are game GUI frameworks that cache their output to avoid redrawing the entire widget tree every frame). Modern UI toolkits (WPF, QML, JavaFX) operate on a scene graph, so they work exactly the same. Android is slowly catching up; it's a disgusting mix of the worst of both worlds.

Jasper_ 9y ago

The reason that apps on Windows can bypass the compositor is that you can use them as the scanout buffer directly on fullscreen. On Linux (both Xorg and Wayland), this same exact behavior is supported with a compositor. For strange legacy reasons, it's known as "fullscreen unredirection". If you're running windowed on all three OSes, you see the same compositor latency.

badsectoracula 9y ago
Note that on Linux you are not forced to use a compositor, personally i do not use one and so i do not have any such penalty when running games in windowed mode.

AnIdiotOnTheNet 9y ago

I'll take a frame of latency and Wayland's not-completely-insane model over X any day.

cat199 9y ago
yet strangely, wayland removes the frames-of-latency known as 'remote windowing' in it's quest for 'improvement'..
- AnIdiotOnTheNet 9y ago
  Because it vastly simplified many things, and oh look, compositors have managed to add remoting back in somehow without it! I mean christ, windows has no support for remote windowing in it's API and still has RDP, even for single windows. Linux people just love to complain about any change that actually makes the system better because then they can't feel quite as elitist for using it.

nhaehnle 9y ago

The funny thing is that there is no technical reason at all for compositing to have worse latency, even for games. Think about the actual operations that are involved. You certainly never want to render directly into the front-buffer (you'd end up scanning out partially rendered scenes). So you render to a back buffer. Which you then blit to the front buffer (assuming you're in windowed mode; in full-screen mode the compositor goes out of the way anyway and lets you just flip). The only difference between the various modes of operation is who does that final blit. In plain X, it's the X server. In Wayland, it's the Wayland compositor. In X with a compositor, it's the compositor. Now granted, some compositors might be silly and re-composite the whole screen each frame, but clearly that can be avoided in most cases. Depending on the setup, there can also be some issues with the scheduling of the game's jobs vs. the compositor's jobs on the GPU. Valve are working on this currently for VR, since the problem is much more noticeable there -- clearly it can be fixed one way or another (on Radeon GPUs you could do a blit via asynchronous compute if need be, for example), but note that compositing actually doesn't change this issue (since the X server's jobs also need to compete with the game's jobs for scheduling time). So if compositing has worse latency, it's because nobody has cared enough to polish the critical paths. Conversely, compositing clearly does have advantages in overall image quality. So why not fix the (entirely fixable) technical problems with compositing?

badsectoracula 9y ago
There is a very good practical reason why the compositor is in no position to fix that even if it could theoretically be possible. A major source for the compositor latency (or actually, the increased response time you get with a compositor) is that the "render to back buffer" (ie. compositor's texture, at the best case) and the "blit to the front buffer" (which happens by the compositor by drawing the window geometry) do not happen at the same time. From a technical perspective it is perfectly possible for a compositor to create a tight integration between a program and the compositor itself: simply synchronize the program's updates with the compositor updates. Every time the program says "i'm done drawing" (either via an explicit notification to the compositor, via glXSwapBuffers or whatever), issue a screen update. The problem however here is the compositor has to take into account multiple windows from multiple programs so you cannot have a single window dictating the compositor updates. Imagine for example two windows with animations running, one at 100fps and another at 130fps. Depending which window is active (assuming that the compositor syncs itself with the active window), it would affect the perception of other window's updates (since what the user will see will be at the rate of the foreground window's update rate). Moreover, beyond just the perception, it will also affect the other windows' animation loops themselves - if a background window finishes drawing itself and notifies the compositor while the compositor is in the middle of an update, the background window will have to wait until the update is finished - thus having the foreground window indirectly also affect the animation loops of the background windows. This can be avoided through triple buffering, but that introduces an extra frame of latency - at least for background windows. So to avoid the above problems, what all compositors do is to decouple window update notifications from screen updates and instead perform the screen updates at some predefined interval - usually the monitor refresh rate and they do the updates synchronized to it. However that creates the increased response time you get with the compositor being on a few milliseconds behind the user's actions, with the most common example would be window manipulation like resizing and moving windows lagging behind the mouse cursor (which is drawn by the GPU directly, thus bypassing the compositor). Hence the linked article recommending a 144Hz monitor to avoid this, although this is just a workaround that makes the problem less visible but doesn't really solve it.