5 ms·

Display devices (usually part of your GPU) still have "explicit display hardware just for the mouse" in form of cursor planes. Later this has been generalized a

Display devices (usually part of your GPU) still have "explicit display hardware just for the mouse" in form of cursor planes. Later this has been generalized as overlay planes. Planes can be updated and repositioned without redrawing the rest of the screen (the regular screen image is on the primary plane), so moving the cursor is just a case of committing the new plane position. The input latency introduced by GNOME's Mutter (the Wayland server used here) is likely simply a matter of their input sampling and commit timing strategy. Different servers have different strategies and priorities there, which can be good and bad. Wayland, which is a protocol, is not involved in the process of positioning regular cursors, so this is entirely display server internals and optimization. What happens on the protocol level is allowing clients to set the cursor image, and telling clients where the cursor is.

smallmancontrov 2y ago
Protocols can bake in unfortunate performance implications simply by virtue of defining an interface that doesn't fit the shape needed for good performance. Furthermore, this tends to happen "by default" unless there is a strong voice for performance in the design process. Hopefully this general concern doesn't apply to Wayland and the "shape" you have described doesn't sound bad, but the devil is in the details.
- jchw 2y ago
  Yeah, Wayland isn't designed in such a way that would require any additional latency on cursor updates. The Wayland protocols almost entirely regard how applications talk to the compositor, and don't really specify how the compositor handles input or output directly. So the pipeline from mouse input coming from evdev devices and then eventually going to DRM planes doesn't actually involve Wayland.
- gizmo686 2y ago
  I don't think the Wayland protocol is actually involved in this. Wayland describes how clients communicate with the compositor. Neither the cursor, nor the mouse are a client, so no where in the path between moving the mouse and the cursor moving on screen is Wayland actually involved. The story is different for applications like games that hide the system cursor to display their own. In those cases, the client needs to receive mouse events from the compositor, then redraw the surface appropriately, all of which does go through Wayland.
  - gf000 2y ago
    According to Asahi Lina, X does async ioctl that can update the cursor even during the scanout of the current frame, while Wayland does atomic, synced updates on everything, cursor involved, which has the benefit of no tearing and the cursor's state is always in sync with the content, but it does add an average of 1 more frame latency (either updates just in time for the next frame), or it will go to the next frame.
    - arghwhat 2y ago
      This is not what Wayland does, it is what a particular display server with Wayland support decided to do. Second, just to be clear, this only discusses mouse cursors on the desktop - not the content of windows, and in particular not games even if they have cursors. Just the white cursor you browse the Web with. Anyway, what you refer to is the legacy drm interface that was replaced by the atomic one. The legacy interface is very broken and does not expose new hardware features, but it did indeed handle cursors as its own magical entity. The atomic API does support tearing updates, but cursor updates are currently rejected in that path as drivers are not ready for that, and at the same time, current consensus is that tearing is toggled on when a particular fullscreen game demands it, and games composite any cursors in their own render pass so they're unaffected. Drivers will probably support this eventually, but it's not meant to be a general solution. The legacy API could let some hardware swap the cursor position mid-scanout, possibly tearing the cursor, but just because the call is made mid-scanout does not mean that the driver or hardware would do it. > but it does add an average of 1 more frame latency If you commit just in time (display servers aim to commit as late as possible), then the delay between the commit and a tearing update made just before the pixels were pushed is dependent on the cursor position - if the cursor is at the first line shown, it makes no difference, if on the last shown, it'll be almost a frame newer. Averaging cursor positions mean half a frame of extra latency, but with a steady sampling rate instead of rolling shutter. Proper commit timing is usually the proper solution, and more importantly helps every other aspect of content delivery as well.
      - gf000 2y ago
        Sure, it's what Gnome Wayland does, but the Wayland protocol does sort of mandate that every frame should be perfect, and the cursor has to match the underlying content, e.g. if it moves over a text it has to change to denote that it is selectable. I do believe it is a useful tradeoff, though.
wmanley 2y ago
Software works best when the developers take responsibility for solving user's problems. > Wayland, which is a protocol This is wayland's biggest weakness. The effect is diffusion of responsibility.
- wmf 2y ago
  You're kind of getting tripped up on terminology. The OP didn't measure Wayland; they measured GNOME Shell which does take responsibility for its performance. Also, I'm not aware of any latency-related mistakes in Wayland/Weston (given its goal of tear-free compositing).
  - wmanley 2y ago
    > You're kind of getting tripped up on terminology. I'm not. My comment doesn't address the latency of gnome shell. I understand the boring technical distinctions between wayland and wayland client libraries and wayland display servers and gnome shell and mutter and sway, blah blah blah. Much like I understand that Linux is a kernel. That it is inspired by UNIX, but it is technically not a UNIX. I also understand that if someone describes themselves as a Linux user they probably don't just mean that they have a Android phone or that the display controller in their dishwasher or wireless access point happens to include Linux the kernel. The "well acksually wayland is just the name of the protocol" that emerges whenever a problem is brought up is a symptom of the underlying problem with wayland the system. The confusion that gives rise to these deflections is also a symptom of that problem. By Conway's law systems end up resembling the organisations that produce them. In this way the design of wayland the system seems to be designed by people who don't want to work together. I can see a parallel with microservice architecture.
    - WhyNotHugo 2y ago
      The distinction between protocol and implementation IS significant here. Imagine comparing HTTP1.1 vs HTTP3. These are protocols, but in practice one compares implementations. I can pick curl for http1.1, but Python for http3, and http3 would very likely measures as slower. Is that the protocols fault?
redmajor12 2y ago
Wayland as a just protocol... Isn't that the same argument they used when it shipped without copy/paste or a screensaver?
- arghwhat 2y ago
  It's not an "argument", it's just a description of what Wayland is. But no, the correct protocol has had copy-paste since day one, and I dont remember there being issues with screensavers. In the metaphor of a web server and a web browser, Wayland would be the HTTP specification. What you're usually interested in is what server you're running, e.g. GNOME's Mutter, KDE's Kwin, sway, niri, or what client you're running, e.g. Gtk4, Qt6, etc.
  - Vilian 2y ago
    Wayland don't support screensaver and clipboard is managed by portals no?, idk why the complain tho, no one uses screensaver and clipboard being managed by a portal dsounds more logical than creating a protocol for that

smallmancontrov 2y ago

Protocols can bake in unfortunate performance implications simply by virtue of defining an interface that doesn't fit the shape needed for good performance. Furthermore, this tends to happen "by default" unless there is a strong voice for performance in the design process. Hopefully this general concern doesn't apply to Wayland and the "shape" you have described doesn't sound bad, but the devil is in the details.

jchw 2y ago
Yeah, Wayland isn't designed in such a way that would require any additional latency on cursor updates. The Wayland protocols almost entirely regard how applications talk to the compositor, and don't really specify how the compositor handles input or output directly. So the pipeline from mouse input coming from evdev devices and then eventually going to DRM planes doesn't actually involve Wayland.
gizmo686 2y ago
I don't think the Wayland protocol is actually involved in this. Wayland describes how clients communicate with the compositor. Neither the cursor, nor the mouse are a client, so no where in the path between moving the mouse and the cursor moving on screen is Wayland actually involved. The story is different for applications like games that hide the system cursor to display their own. In those cases, the client needs to receive mouse events from the compositor, then redraw the surface appropriately, all of which does go through Wayland.
- gf000 2y ago
  According to Asahi Lina, X does async ioctl that can update the cursor even during the scanout of the current frame, while Wayland does atomic, synced updates on everything, cursor involved, which has the benefit of no tearing and the cursor's state is always in sync with the content, but it does add an average of 1 more frame latency (either updates just in time for the next frame), or it will go to the next frame.
  - arghwhat 2y ago
    This is not what Wayland does, it is what a particular display server with Wayland support decided to do. Second, just to be clear, this only discusses mouse cursors on the desktop - not the content of windows, and in particular not games even if they have cursors. Just the white cursor you browse the Web with. Anyway, what you refer to is the legacy drm interface that was replaced by the atomic one. The legacy interface is very broken and does not expose new hardware features, but it did indeed handle cursors as its own magical entity. The atomic API does support tearing updates, but cursor updates are currently rejected in that path as drivers are not ready for that, and at the same time, current consensus is that tearing is toggled on when a particular fullscreen game demands it, and games composite any cursors in their own render pass so they're unaffected. Drivers will probably support this eventually, but it's not meant to be a general solution. The legacy API could let some hardware swap the cursor position mid-scanout, possibly tearing the cursor, but just because the call is made mid-scanout does not mean that the driver or hardware would do it. > but it does add an average of 1 more frame latency If you commit just in time (display servers aim to commit as late as possible), then the delay between the commit and a tearing update made just before the pixels were pushed is dependent on the cursor position - if the cursor is at the first line shown, it makes no difference, if on the last shown, it'll be almost a frame newer. Averaging cursor positions mean half a frame of extra latency, but with a steady sampling rate instead of rolling shutter. Proper commit timing is usually the proper solution, and more importantly helps every other aspect of content delivery as well.
    - gf000 2y ago
      Sure, it's what Gnome Wayland does, but the Wayland protocol does sort of mandate that every frame should be perfect, and the cursor has to match the underlying content, e.g. if it moves over a text it has to change to denote that it is selectable. I do believe it is a useful tradeoff, though.

wmanley 2y ago

Software works best when the developers take responsibility for solving user's problems. > Wayland, which is a protocol This is wayland's biggest weakness. The effect is diffusion of responsibility.

wmf 2y ago
You're kind of getting tripped up on terminology. The OP didn't measure Wayland; they measured GNOME Shell which does take responsibility for its performance. Also, I'm not aware of any latency-related mistakes in Wayland/Weston (given its goal of tear-free compositing).
- wmanley 2y ago
  > You're kind of getting tripped up on terminology. I'm not. My comment doesn't address the latency of gnome shell. I understand the boring technical distinctions between wayland and wayland client libraries and wayland display servers and gnome shell and mutter and sway, blah blah blah. Much like I understand that Linux is a kernel. That it is inspired by UNIX, but it is technically not a UNIX. I also understand that if someone describes themselves as a Linux user they probably don't just mean that they have a Android phone or that the display controller in their dishwasher or wireless access point happens to include Linux the kernel. The "well acksually wayland is just the name of the protocol" that emerges whenever a problem is brought up is a symptom of the underlying problem with wayland the system. The confusion that gives rise to these deflections is also a symptom of that problem. By Conway's law systems end up resembling the organisations that produce them. In this way the design of wayland the system seems to be designed by people who don't want to work together. I can see a parallel with microservice architecture.
  - WhyNotHugo 2y ago
    The distinction between protocol and implementation IS significant here. Imagine comparing HTTP1.1 vs HTTP3. These are protocols, but in practice one compares implementations. I can pick curl for http1.1, but Python for http3, and http3 would very likely measures as slower. Is that the protocols fault?

redmajor12 2y ago

Wayland as a just protocol... Isn't that the same argument they used when it shipped without copy/paste or a screensaver?

arghwhat 2y ago
It's not an "argument", it's just a description of what Wayland is. But no, the correct protocol has had copy-paste since day one, and I dont remember there being issues with screensavers. In the metaphor of a web server and a web browser, Wayland would be the HTTP specification. What you're usually interested in is what server you're running, e.g. GNOME's Mutter, KDE's Kwin, sway, niri, or what client you're running, e.g. Gtk4, Qt6, etc.
- Vilian 2y ago
  Wayland don't support screensaver and clipboard is managed by portals no?, idk why the complain tho, no one uses screensaver and clipboard being managed by a portal dsounds more logical than creating a protocol for that