15 ms·

Hard numbers in the Wayland vs. X11 input latency discussion

• JimmaDaRustla 2y ago
  The title and conclusion are disingenuous - they've proven that Gnome Wayland has higher latency than Gnome Xorg. This could very well have nothing to do with Wayland at all - we need more testing.
• sylware 2y ago
  Only one wayland compositor and then it is all wayland compositors???
• sapiogram 2y ago
  Beautiful work. Could it be worth repeating the experiment with the monitor running at a very low refresh rate, i.e. 30hz? If Wayland is always a frame slower than X11, it should be much easier to observe.
• jchw 2y ago
  Note that the results will differ between compositors, different GPUs, and different configurations. This is somewhat less the case with X11 since there is only one X server implementation (that Linux desktop systems use, anyhow.) I think there may still be the issue that many compositor/GPU combinations don't get hardware cursor planes, which would definitely cause a latency discrepancy like this.
• tapoxi 2y ago
  This is probably compositor specific, so I wonder if this is a mutter issue or if it can be replicated in kwin.
  • ChocolateGod 2y ago
    Mutter has one of the laggiest Wayland implementations imho.
• deleted 2y ago
  [deleted]
• superkuh 2y ago
  The various wayland compositors all have huge problems with keyboards, mice, and other input devices because the wayland spec and reference implementation(s) chose not to support such things. So each wayland compositor picks a different solution. Popular ones are libei and libinput but often there's just no support for "advanced" mouse/keyboard inputs at all (looking at you, weston). And you can never tell if a particular piece of linux software will work on any given wayland. The fragmentation is terrible. On X with the strong X11 reference that actually implements everything you can be sure something that works on one will work on another. Hell, it's been 12 years and not a single wayland compositor supports screen readers for the visually disabled yet. They are toys.
  • bsnnkv 2y ago
    This is why I ultimately ended up on Windows after switching from macOS and focusing my free time development work on improving the power user/developer experience on Windows with a tiling window manager ecosystem.[1] One of the really cool things that has come out of this is seeing how people are adding accessibility workflows on top of the ecosystem. There is one person who controls their entire desktop via the tiling window manager using voice control![2] [1]: https://github.com/LGUG2Z/awesome-komorebi https://github.com/LGUG2Z/awesome-komorebi [2]: https://youtu.be/fiPJLmhnnXM https://youtu.be/fiPJLmhnnXM
    • BearOso 2y ago
      Windows has added several things with the newer WDDMs to help alleviate compositor problems: flip model, overlays, etc. They've definitely identified the problem areas and are trying to correct them. It's been quite the turnaround since Vista.
  • Hizonner 2y ago
    I'm told they left out color calibration support, too (as I read it, there's no way for calibration apps to control, or even know, what compensation is applied when they're trying to measure things). Oh, and client side window decorations still around, giving up on the chance to fix an ancient security hole. So no good for games, no good for professional graphics, no good if you don't see well... basically no good if you're any different from the people who hacked it together. But, hey, with any luck they cut down on the screen tearing that I've never noticed.
• henry700 2y ago
  Has implications for competitive FPS gaming on modern Linux — one more problem to fix. For example "The Finals" allows Linux players, but imagine having this much input delay and having to revert to classic x11 to play games, lol.
  • riskable 2y ago
    It's actually just cursor latency specific to the windowed environment you're running. From what I've experienced (with a 4060 Ti) it doesn't seem to impact FPS games at all. I haven't tried any games that use a cursor with Wayland yet so I don't know if it would have an impact there. I think it has to do with whether or not the game in question is reading the mouse device directly (e.g. through SSL) or via the compositor. If it's reading the device directly it stands to reason that there would be less latency.
  • rez9x 2y ago
    I have trouble believing that 6.5ms in increased latency would be perceptible to any more than a fraction of a percent of the most elite gamers. Most the people claiming that this level of difference is impacting their gameplay are victims of confirmation bias.
    • xyst 2y ago
      You may think 6.5 ms of input latency is imperceptible. But combine it with the rest of the stack (monitor refresh rate, local network latency, RTT between client and server, time for server to register input from client and calculate “winner”), and it becomes the diff between an L and W. In the case of pros, the diff between a multimillion dollar cash prize and nil.
    • sapiogram 2y ago
      I would postulate that 100% of professional (i.e. elite) competitive gamers would be able to tell the difference. See this old touchscreen demonstration: https://www.youtube.com/watch?v=vOvQCPLkPt4 https://www.youtube.com/watch?v=vOvQCPLkPt4
    • Strilanc 2y ago
• deleted 2y ago
  [deleted]
• mikenew 2y ago
  I get major lag spikes when the gpu is under heavy load (like doing Stable Diffusion inference or something). TBF I haven't A/B tested with X11, but I don't ever remember it being like that. An extra frame of latency isn't great on it's own, but the occasional spikes in lag are really irritating.
  • cma 2y ago
    May still happen especially if it is thrashing vram in and out of system memory or something, but have you tried lowering priority of the stable diffusion process?
    • cloudwalk9 2y ago
      I can also attest to horrific lagspikes on an Optimus laptop even if Intel is driving the desktop. Memory pressure is definitely the problem here. Lagspikes actually lessened when I switched to Wayland Gnome. I think they lessened further with PREEMPT_RT on kernel 6.12. Nvidia requires an environment variable to build on real time kernels but it plays surprisingly nice as of driver 570. But if you have this config, you need at least 11th gen Intel iGPU or AMD APU, because i915 does not build for real-time kernels. Only the Xe driver works and only if you force_probe the ID if it's Tiger Lake. ...Which I don't get because the Xe driver is said to explicitly support, at minimum, Tiger Lake. I played Minecraft on the iGPU with Xe and it was perfectly fine. It... drew 3D graphics at expected framerates.
• bigbones 2y ago
  would love to see this test repeated with a non-compositing WM on X11
• deleted 2y ago
  [deleted]
• sxp 2y ago
  For anyone who uses ffmpeg for this type of per frame analysis, `ffmpeg -skip_frame nokey -i file -vsync 0 -frame_pts true out%d.png` will get the "presentation time" of each frame in the video. That's more precise than just dumping frames and calculating timestamps. You can also do something similar in a web browser by playing a <video> and using `requestVideoFrameCallback()`. Though, you might need to set `.playbackRate` to a low value if the computer can't decode all the frames fast enough. > With my 144Hz screen,....Wayland, on average, has roughly 6.5ms more cursor latency than X11 on my system...Interestingly, the difference is very close to 1 full screen refresh. I don't know whether or not that's a coincidence. The fact that the latency is almost 1/144th of a second means that it might become 1/60th of a second on standard 60Hz monitors. This is hard to notice consciously without training, but most people can "feel" the difference even if they can't explain it.
  • goalieca 2y ago
    I found low latency terminals make a big improvement to even simple tasks like typing.
    • daef 2y ago
      what would you count as low latency terminal?
      • layer8 2y ago
        See https://beuke.org/terminal-latency/ https://beuke.org/terminal-latency/. Single-digit milliseconds I’d say. These numbers are minus the keyboard and display latency.
        • rustc 2y ago
          Has anyone run this test on Ghostty?
          • _emacsomancer_ 2y ago
• sheepdestroyer 2y ago
  It would be interesting to have numbers under gamescope instead, Gnome doesn't care much about gamers & power users that mesure latencies.
  • dijit 2y ago
    Games don’t depend on a compositor for anything, they draw direct to the framebuffer. Anything related to the compositor won’t affect the game, unless it’s in windowed mode, then there can be some strange interactions.
    • talldayo 2y ago
      Games don't, but xWayland does.
      • deleted 2y ago
        [deleted]
      • dijit 2y ago
        I thought we were talking about gamers, xWayland doesn’t affect gamers at all either.
        • mort96 2y ago
          What do you mean? Most games run on XWayland when you're in a Wayland session, wouldn't input latency caused by XWayland therefore affect gamers who use Wayland?
          • dijit 2y ago
            Games don’t run via X11, they write direct to the framebuffer using a framework like Vulkan. In windowed mode they can be captured, then things get tricky, but in full screen they draw direct to the gpu, vulkan itself is completely headless.
            • 2y ago
• hirako2000 2y ago
  Was Wayland not architected to fix latency issues? I also find Wayland more laggy and more buggy without concrete evidence of course. It has significantly improved lately though, far more stable than it used to be.
  • vander_elst 2y ago
    Wayland is just a protocol that is probably implemented over and over and each implementation might have issues. It's mostly likely the different implementations having issues.
    • hirako2000 2y ago
      Was Wayland not architected to be simpler to implement than the (antique) X11 protocol? Not arguing though, you are right it is just issues: Drivers, the Wayland implementations, how some plethora of apps and libraries have been battle tested then optimized, years over years, for X11. Not as much for Wayland display yet.
      • KerrAvon 2y ago
        Architecture contributes, but all it takes to blow input latency is some inefficient code almost anywhere in the input or rendering pipeline. The Apple IIe didn’t really have very many layers between the keyboard and the CRT. Wayland might have an optimal architecture in theory for latency — I don’t know — and latency could still be ruined by not fully optimizing the other components in the stack.
      • mrweasel 2y ago
        > Was Wayland not architected to be simpler to implement than the (antique) X11 protocol? I think it's safe to assume that it is actually simpler, given that we already have multiple Wayland implementations, but still basically just the one X11 server implementation. Can one or more of those implementations shave off one or two milliseconds overs the next 40 years... Probably yes.
        • 2y ago
• BearOso 2y ago
  This boils down to software cursor vs hardware cursor.
  • mort96 2y ago
    I'm pretty sure this is incorrect? I half-remember having cursor rendering bugs on GNOME Wayland with NVidia ages ago which I had to disable something h hardware cursor related to fix. But I don't know if that's what you're talking about or not. Could you link to a source which says that Mutter doesn't make use of hardware cursor? (Hi, author here by the way! ... Don't worry, that disabling hardware cursors thing was at least one OS re-install ago)
    • mariusor 2y ago
      I wonder if the latency is due to the compositor receiving a damage event for the area of the cursor, thus being a frame late to render. But probably that wouldn't be an issue with hardware cursors.
    • BearOso 2y ago
      Yes, it should use hardware cursor by default on most hardware. But I don't think that RPi's VideoCore (in the linked article) supports hardware cursor or multiple planes. The difference between X11 and Wayland might then be that the cursor isn't vsynced as part of the composite process on X11, where it is on Wayland. Mutter definitely throttles cursor updates on Wayland, too, which will contribute slightly to latency even with a hardware cursor. In general, with Wayland, the compositor is responsible for cursor updates, and I'm not sure which other ones throttle. But that would be where the difference comes from when using hardware cursors.
      • dividuum 2y ago
        Pis VideoCore has like 48 planes split up between writeback and HDMI output CRTCs. There’s one explicit Cursor plane per CRTC.
        • 2y ago
• kelseyfrog 2y ago
  > I don't have the statistics expertise necessary to properly analyze whether this difference is statistically significant or not, The two sample t-test statistic is -4.74 with a p-value of 4.20e-05.
  • pwr22 2y ago
    and that means?
    • kelseyfrog 2y ago
      Depends on your alpha ;) . But it's significant.
      • viraptor 2y ago
        That really does not answer the question.
    • jampekka 2y ago
      That there's a probability of 4.20e-05 that the observed difference of that large would happen by chance due to observation noise if there would be no real difference (given your assumptions about the data generating process holds).
      • tstenner 2y ago
        The normality assumption is a large stretch, especially since there an absolute lower limit near the observation and a somewhat discrete distribution, so a t-test isn't appropriate. But then again it looks significant, so there's no real need for a test
        • jampekka 2y ago
          Something like Mann-Whitney U would be safer. But it's indeed quite obvious from the samples that the difference is robust.
• guerrilla 2y ago
  I knew it! And when it does stall it takes way longer than X11.
• whatever1 2y ago
  Can’t we port whatever android uses? Clearly the community cannot execute GUI related things at the level that a focused company can. Possibly because the best contributors don’t care about GUI anyway.
  • itvision 2y ago
    Surface Flinger currently doesn't support well or at all multi-monitor configurations which are essential for the PC user. Otherwise, it's actually far more advanced than Wayland, and it had essential modern features implemented years ago, including HDR, VRR, and others. Why hasn't it been ported to Linux? Probably NiH, probably it's Google's child.
    • dTal 2y ago
      Ported to Linux? It runs on Linux. What exactly would need porting? What would be required to make surfaceflinger a practical Linux desktop compositor is merely some translation layer, equivalent to XWayland, that supports hardware acceleration. Such things have been written, but not open source and they never got traction. The idea has been toyed with for over a decade: https://news.ycombinator.com/item?id=5317638 https://news.ycombinator.com/item?id=5317638
      • yjftsjthsd-h 2y ago
        > Ported to Linux? It runs on Linux. What exactly would need porting? It runs on Android/Linux, not GNU/Linux. I would be surprised if there wasn't at least a bit of work needed to handle things like bionic vs glibc, and you'd really want it to act "normal" by being installed to FHS paths instead of Android's weird filesystem layout. All doable, I expect, but it would be a port.
  • tuna74 2y ago
    Just do it!
• ChuckMcM 2y ago
  This is excellent. Too often people guess at things when they could be more empirical about them. Ever since I learned the scientific method (I think 3rd or 4th grade) I was all about the 'let's design an experiment' :-). Let me state up front that I have no idea why Wayland would have this additional latency. That said, having been active in the computer community at the 'birth' of X11 (yes I'm that old) I can tell you that there was, especially early on, a constant whine about screen latency. Whether it was cursor response or xterm scrolling. When "workstations" became a thing, they sometimes had explicit display hardware for just the mouse because that would cut out the latency of rendering the mouse in the frame. (not to mention the infamous XOR patent[1]) As a result of all this whinging, the code paths that were between keyboard/mouse input and their effect on the screen, were constantly being evaluated for ways to "speed them up and reduce latency." Wayland, being something relatively "new" compared to X11, has not had this level of scrutiny for as long. I'm looking forward to folks fixing it though. [1] https://patents.google.com/patent/US4197590 https://patents.google.com/patent/US4197590
  • dralley 2y ago
    GNOME has unredirection support, so I don't think this test is applicable to actual in-game performance. A fullscreen app ought to be taking a fast path through / around the compositor.
    • deleted 2y ago
      [deleted]
    • eqvinox 2y ago
      This isn't about in-game performance, this is about the desktop feeling sluggish.
  • AtlasBarfed 2y ago
    How old is Wayland? I'll be reading a dream of spring in my grave at this rate. I understand I'm complaining about free things, but this is a forced change for the worse for so long. Wayland adoption should have been predicated on a near universal superiority in all input and display requirements. Intel and AMD and Nvidia and Arm makers should be all in on a viable desktop Linux as a consortium. Governments should be doing the same because a secure Linux desktop is actually possible. It is the fastest path to showcasing their CPUs and 3d bling, advanced vector /computer hardware. Wayland simply came at a time to further the delay of the Linux desktop, at a time when Windows was attempting to kill Windows with its horrid tiles and Apple staunchly refused a half billion in extra market cap by offering osx on general x86.
• t3rra 2y ago
  [flagged]
  • itvision 2y ago
    Answer to what? DLSS was designed to solve performance issues, not architecture problems and poor design. And you absolutely wouldn't want your fonts or UI to be upscaled by it. You will hate every second of your experience.
    • t3rra 2y ago
      It was a joke. Why are you so serious? Did you have a bad day?
  • soganess 2y ago
    My response assumes that you are talking about the "frame warping" feature of Reflex 2 (saying DLSS 4 was misquoting Nvidia's PR jumble) not simply frame-gen/MFG or upscaling. MFG, while producing even more interpolated frames, does not improve latency and will probably make it worse. I suppose an argument could be made for upscaling in a roundabout may (lower internal resolution = more frames = faster response time) but that would be more true for DLSS 3 as it has a lower GPU overhead than the transformer-based DLSS 4 Anyways, even if it were the case that graphic stack produces a response to the input by warping the frame during interpolation / before rendering, the input that triggers said response would (certius paribus) still go through libinput (and most likely the Wayland compositor) so timing would remain the same. For any features to improve on response time, it would likely have to circumvent one (if not both) libraries.
    • t3rra 2y ago
      I know I know. I just made a joke. And I know you cannot get it. Seriously, you two guys are so serious. What is the problem with you?
• ray023 2y ago
  I also have no ideas how it works, but my guess would be that is actually translates exactly like that into games. Why would the mouse on the desktop have a latency that games do not have. Linux does not have a real fullscreen mode like Windows has (that I do not use anymore anyway for faster alt-tabbing). So my guess is, the cursor or mouse input gets into games very much the same way as it gets in the desktop.
  • gsich 2y ago
    Hardware cursor might be used in games. You notice the difference on Windows if a GPU does not support it, even on the desktop.
  • zamalek 2y ago
    Wayland has support for games completely taking over the GPU; notoriously critical for VR support. This is basically the same as Windows exclusive full screen. Wine/proton would need to support it, XWayland would need to support it (Wine/Proton are one major step away from native Wayland support: Vulkan), and finally the compositor would need to support it. Gnome is about the worst compositor that you could be testing any of this stuff on, they are comically hostile towards ideas not their own. The chances of this ever working on Gnome are near zero. KDE is pretty good for support, Hyprland seems to be trying to support every under the sun.
    • debugnik 2y ago
      > notoriously critical for VR support. This is basically the same as Windows exclusive full screen. D3D12 dropped support for exclusive fullscreen, and I don't think headsets even go through DXGI but their own swap chain APIs. Why do VR games on Linux need the equivalent from the Wayland compositor?
      • account42 2y ago
        VR on X.Org also already bypasses the X server for the VR display.
      • 2y ago
• mixmastamyk 2y ago
  Looked up the mouse to see how fast it updates and read that it is wireless. Kind of disappointing and might explain some of the variance. Although if X11 is consistently faster the test should still have some validity. Would like to see a test with gnome and the wireless mouse removed from the equation.
  • IshKebab 2y ago
    Yeah testing this with a wireless mouse is dumb. The proper way is to use an Arduino or similar as a USB mouse, and connect a photodiode to it too. USB mice update at 1 kHz.
    • viraptor 2y ago
      It depends on the model. Wireless mice don't have to be slower than wired. But even then, for this test it's irrelevant. It's the same mouse in both scenarios, and we only care about the difference. Even if a wired mouse stripped X ms on both sides, the result would be the same.
      • IshKebab 2y ago
        It does matter because the latency of a wireless mouse isn't necessarily consistent, so it adds noise into an already very high noise set-up.
        • viraptor 2y ago
          That's only a problem if the noise of wireless is higher than the signal (measured delay difference). And we can see from the data it isn't. Unless you're claiming it can be consistently different by ~5ms between those two tests... but that's very unlikely. Like, "you should demonstrate it" unlikely.
  • gsich 2y ago
    You need a setup like this guy: https://www.youtube.com/watch?v=nbu3ySrRNVc https://www.youtube.com/watch?v=nbu3ySrRNVc
• SaintSeiya 2y ago
  Why this does not surprise me? every attempt to rewrite a working solution to make it more "modern, easy to maintain and future proof" rarely do so. It always end up slower, with more latency and lifted by faster hardware, not by faster software. Every 20 years a new generation comes weaker, pampered with the abtsractions of the previous generations who did the grunt work for them. After 3 generations of software developers all we have is library/framework callers and nobody really knows about performance and optimization.
  • kombine 2y ago
    This is not my experience. When I upgraded several of my computers to KDE 6 switching to Wayland, overall responsiveness and snappiness of the system increased very visibly. There are still features that Wayland lacks compared to X11, but I am willing to compromise in favor of its other benefits.
    • mtlmtlmtlmtl 2y ago
      On the other hand, I just did a fresh install of Artix Linux. Installed KDE just to have something functional while I get my tiling setup working. Boot into Plasma(Wayland) and it utterly shits itself, turning my main monitor on and off repeatedly until it finally crashes. So I pick Plasma(X11) instead and that just works. In fact, in almost 2 decades of using Linux every day, I can't remember X doing anything similar. It's always just worked.
      • gf000 2y ago
        > In fact, in almost 2 decades of using Linux every day, I can't remember X doing anything similar. It's always just worked. Well, we have very different memories then. Sure, X worked reliably once configured. But configuring it was a marathon in hell, as per by my memory, and all the litany of forum posts crying out for help all across the internet. Like, I have at one point had to rescue an install by changing back the config file without a screen at all!
        • 2y ago
• haunter 2y ago
  Does the Nvidia FrameView app works under Linux (or more like under X11 and Wayland)? You can measure general system latency with that https://www.nvidia.com/en-us/geforce/technologies/frameview/ https://www.nvidia.com/en-us/geforce/technologies/frameview/
• PeakKS 2y ago
  This is pretty bad methodology for someone to call it "Hard numbers"
  • sapiogram 2y ago
    If you're unhappy about their methodology, you're welcome to write productively about your specific gripes.
  • jay_kyburz 2y ago
    Yes, I was expecting some software that listens directly to the mouse input, and watches a pixel on the screen. Messing around with cameras and counting frames introduces a whole bunch of other variables like the quality of the monitor and mouse and phone used for recording.
    • ggreer 2y ago
      The camera, mouse, and monitor all stayed the same for the tests, but there was a significant difference in latency. Out of the 16 times the experiment was run, only once did Wayland have lower latency. It would be an amazing coincidence if the monitor, mouse, and/or camera were the reason for this. Figuring out why there's increased latency is a job for software tooling, but I think this guy's experiment is one of the best ways to measure what users care about: The time it takes for an input (such as mouse movement) to result in a change (such as cursor movement). Note that this doesn't mean that the Wayland protocol itself is the reason for the higher latency. It may be Gnome's implementation (testing with a wlroots compositor might shed light on this). It may be differences in default configuration options. It may be that Wayland and X11 start up different services, and the Wayland helper processes increase load on the machine. But I seriously doubt the reason for the difference in latency was because the same hardware was used throughout the experiment.
      • deleted 2y ago
        [deleted]
    • tuna74 2y ago
• cakealert 2y ago
  Would have been nice to see if the issue is present on wlroots compositors, GNOME rolled their own.
• delusional 2y ago
  I did some work to create a simple device to measure this some yeas ago. The intention was for developers to use it to regularly benchmark their software. It costs like 20 bucks and takes 10 minutes to build. I might get to run some benchmarks myself next week. https://jnsn.dev/posts/frametime/ https://jnsn.dev/posts/frametime/ with a follow-up in https://jnsn.dev/posts/fastisslow/ https://jnsn.dev/posts/fastisslow/ or if you just want to see the code without my jabbering: https://github.com/DelusionalLogic/Frametime https://github.com/DelusionalLogic/Frametime
• bitwize 2y ago
  Composited desktops have their drawbacks. With a display system like X11 or old-school GDI-based Windows, draws to the display are next to immediate, whereas with composited desktops, everybody has to draw to a back buffer, then the back buffers have to be composited together on the final display, adding at least one frame of latency. Mouse cursors have been a hardware sprite since the 90s, but Wayland compositors may favor using the GPU to draw a soft cursor. It'd be interesting to see a compositor that can speak Wayland, yet performs immediate, region-clipped draws of all windows like X11 conventionally does.
  • unit149 2y ago
    [dead]
• goodburb 2y ago
  It's much worse with Nvidia 30+ series on Wayland, feels worse than a Bluetooth mouse, using Nouveau eliminates lag for the most part.
• Cthulhu_ 2y ago
  TIL Wayland is 16 years old already; in a few years (I can't math) it'll be as old as X was when Wayland came out, but it seems it's still considered mediocre or not as good as X was.
  • tmtvl 2y ago
    If X is 40 and Wayland is 16,that means a difference of 24 years. Hence Wayland compositors have 8 years to work out the kinks. I am currently using Wayland via Plasma 6 and it works well enough but I don't have special needs, so I don't know how well... say... screen readers work.
    • DonHopkins 2y ago
      That means in two more years it will be legal for X11 and Wayland to have a baby!
      • guappa 2y ago
        Maybe like we did we pulseaudio (throw it away and make pipewire) needs to be done with wayland.
        • account42 2y ago
          I'd rather get something that actually works reliably this time.
      • josefx 2y ago
        We can call it V to signify another step backwards and make it the default, people can just use the terminal until the first implementation crops up.
      • tmtvl 2y ago
        What consenting adults do is their business, but still such an age gap feels a bit icky to me.
  • craftkiller 2y ago
• soganess 2y ago
  Great work, I hope the gnome/wayland devs appreciate it. I wonder if there is an issue with high frame rate support in Gnome (or possibly wayland?). It seems like they are actively still working on it: https://9to5linux.com/gnome-47-3-improves-frame-rate-for-monitors-attached-to-secondary-gpus https://9to5linux.com/gnome-47-3-improves-frame-rate-for-mon...
  • PeakKS 2y ago
    No, that article is about copying frames between gpus
    • soganess 2y ago
      Agreed. That was just the latest related article I saw. I was bring it up to support my claim of "high framerate support is still new/developing in Gnome"... embarrassingly my old man brain compressed time... it's been 6 years[1], oh well. [1]:https://www.phoronix.com/news/GNOME-Still-Dropping-Latency https://www.phoronix.com/news/GNOME-Still-Dropping-Latency
• light_hue_1 2y ago
  Wayland has just been a disaster for the open source desktop. It's absorbed all of the effort into something horrifically subpar while breaking compatibility with the past. Accessibly is far worse to the point of embarrassment. Latency is worse. Compatibility is gone. Everyone in the ecosystem needs to do more work to end up with fewer features for end users. This is the opposite of systemd in every way. Systemd upset a lot of people because it was entirely user focused and didn't care about the beauty of the architecture. Wayland is only obsessed with how nice their apis are, not what the end user experience is. Let's end this disaster. It's been 16 years. It's almost old enough to vote. When will we accept it has failed?
  • prmoustache 2y ago
    x11 failed in other areas, and earlier. So what is your proposal, starting again from a blank sheet? I don't think it will help.
    • Gormo 2y ago
      > x11 failed in other areas, and earlier. Still works great for me!
      • prmoustache 2y ago
        as do wayland for the majority of its users. Fine, we are all happy then.
        • Gormo 2y ago
          Exactly -- so nothing has failed at all!
  • tuna74 2y ago
    "Let's end this disaster. It's been 16 years. It's almost old enough to vote. When will we accept it has failed?" Use what you think is better then.
  • 2y ago
• sprash 2y ago
  It should be easy to create something better than X11. And if not pour more resources into X11 developement. Wayland on the other hand is a complete failure on all fronts. It should be abandoned ASAP.
  • PeakKS 2y ago
    Well it's already better than X11, so task accomplished.
    • sprash 2y ago
      What exactly is better? I can't even think of a single thing. It has worse performance on almost all metrics. Heavy fragmentation. Very limited and incomplete API measured in functionality but still extremely complicated to work with as developer. Extremely slow development. The only thing that it has over X11 is better funding (for whatever reason).
      • mixedCase 2y ago
        Are we really pretending multi-monitor on X is not an absolutely shit show? Try mixed DPI and VRR in one of the monitors and see how you get that working well.
        • sprash 2y ago
          The VRR issue can be solved with a simple patch that never got merged into mainline (no idea why). Hence it's not inherently a X11 problem and honestly also a rather niche problem compared to the humongous deficits Wayland brings. Mixed DPI works perfectly fine on X11 if the toolkit supports it. The xrandr protocol provides all the necessary dpi information. GNOME simply chose not to support it on X11.
        • Gormo 2y ago
          I have zero issues with multiple monitors, with mixed DPIs and refresh rates, and different scaling factors on each display, using XFCE under Xorg.
• perihelions 2y ago
  For those of you who are application developers on Linux/Wayland, do you use any other methods for benchmarking true latencies between input events and screen? Less heroic than the high-speed camera method. I'm embarrassed I've still never figured this out properly.
  • freehorse 2y ago
    A photocell is I think the standard (or at least "the standard" in psychophysics applications I have also used). You need, though, to make sure your event causes a change that a simple photocell would detect (eg you have a corner of your screen or application window having a small square that turns from black to white and vice versa). You can connect that to an arduino or whatever you can use to get both signals in some low latency way (one from user interaction or whatever causes the event, and one from the event itself with that light intensity change). There are also photocells with a higher granularity for luminosity changes than just light/dark, so maybe you do not need to have the black/white square thing, just something that causes enough luminosity change to be detected. This should, in principle, give you very accurate timings if you can also get the event initiator somehow with low latency (or you can do stuff like simulate an event periodically and look for skipped frames/variance in the screen presentation rather than latency).
  • dsissitka 2y ago
    You might like Typometer. I believe it was created by a JetBrains developer to improve input latency in IDEA. > Typometer works by generating OS input events and using screen capture to measure the delay between a keystroke and a corresponding screen update. Hence, the measurement encompasses all the constituents of processing latency (i. e. OS queue, VM, editor, GPU pipeline, buffering, window manager and possible V-Sync). https://pavelfatin.com/typing-with-pleasure/ https://pavelfatin.com/typing-with-pleasure/
• sweeter 2y ago
  Its very interesting, but I would love to see other Wayland implementations other than GNOME's. Its kind of hard to tell if its the GNOME specific implementation or if this is going to be a trend across KDE X vs KDE wayland and specific window managers like sway -> i3 and hyprland etc... But still a really cool blog post.
• shmerl 2y ago
  > GNOME Wayland session I wouldn't use that. Try KDE if you want a decent gaming experience. Gnome was very slow in even implementing something like adaptive sync, so they hardly are prioritizing latency issues for the end user. I.e. Gnome's problems ≠ Wayland problems in the broader sense. Conflating the two is wrong.
  • sprash 2y ago
    Wayland lacks a strong reference implementation. The resulting fragmentation in the ecosystem turns the most popular compositors into de-facto standards every application developer will have to use as a reference. This means Gnome's problems = Wayland problems. Furthermore, the latency issues with Wayland are inherent. Compositors will always have worse latency than writing directly to the front buffer.
    • shmerl 2y ago
      The kind of issues referenced in the article are not related to compositors being compositors though, but to bad implementations. Gnome is big, but it's not the most popular DE out there, so it can't be treated even as a de-facto standard in the Wayland world. Some things might be treated that way, but they are lower level (let's say Pipewire, libinput etc.). And regardless, it can't be used an example that blames Wayland for Gnome's issues, unless literally every single compositor has this problem and the real reason is Wayland's inherent limitations. There are such limitations in general indeed due to Wayland still evolving, but this isn't one of them.
    • ChocolateGod 2y ago
      > Compositors will always have worse latency than writing directly to the front buffer. This is not really true, if a full screen application is detected, the compositor can tell the GPU to instead just display the contents of the applications buffer, skipping the compositor. IIRC this is broken on Xwayland GNOME.
    • bandrami 2y ago
      > Compositors will always have worse latency than writing directly to the front buffer. I mean... in the sense that "ALSA has to be faster than Jack ultimately", I guess; that fact may not be useful for a given setup
• mattbee 2y ago
  I've used https://isitsnappy.com/ https://isitsnappy.com/ on the iPhone to do these kinds of measurements pretty quickly, to substantiate why I sucked at Steam Deck rhythm games when connected to a TV. (I do not suck, my setup had 100ms lag)
  • abhinavk 2y ago
    Great app. I hope next iPhones have improved slow-motion camera modes. 1000fps will be a long shot but maybe 480/500fps.
  • davet91 2y ago
    What was the issue with your setup? TV not in game mode?
    • mattbee 2y ago
      Never got to the bottom of it. TV was set to game mode, I tried various scaler settings on the deck itself, couldn't get it below about 90-100ms. I've tried on two TVs. You can learn to play Space Channel 5 by tapping 100ms ahead of the beat. But I worried that if I succeeded too much, I'd end up doing everything that way: jumping red lights, taking espresso shots while they were still being poured etc.
      • ramses0 2y ago
        Living With Lag: https://www.fastcompany.com/3029918/what-its-like-to-experience-video-game-lag-in-real-life https://www.fastcompany.com/3029918/what-its-like-to-experie... ...what it's like to experience video game lag in real life.
  • lucb1e 2y ago
    Cool site! I didn't know this was an established way of measuring input lag but it's how I've been doing it since I found out my Samsung phone came with a 960 fps camera. Manually counting frames with ffmpeg is a pain, but I never even thought to look for an app that does this. Thanks for sharing, I'll take a look around :)
• rezmason 2y ago
  This reminds me of the early work of the structuralists in psychology. Measuring the time it takes for a person (or in this case Wayland) to react to a sensation.
• colonial 2y ago
  Funny, I didn't even know this was a debate. I didn't even notice when Fedora switched from X to Wayland by default - maybe because I'm still a 60hz plebian? I have experienced issues with my mouse input locking up/jumping - especially noticeable when I'm playing a shooter - but I'm pretty sure that's just my poor ancient mouse giving out.
• okr 2y ago
  Hm. I am currently using Ubuntu 24.10 and it also has Wayland and gnome as a default, with the nvidia 565 driver. And it is really, really bad. For example, i use chrome and the tabs, man, it takes ages to switch from one tab to another and to get focus in the location bar again. So much for the whining. I do not know what to do. I could record this behaviour, but it somehow needs to show the timestamp of when i intend a behaviour and the time, that behaviour becomes actualized. I used to be quick with all the hotkeys but i do make so many mistakes now, because the system is simply not responsive and all my instincts await instant action.
  • hinoki 2y ago
    https://ui.perfetto.dev/ https://ui.perfetto.dev/ Won’t get you end to end latency, but should be able to trace the input events through chrome, and show the context swaps of all the Wayland/dbus stuff. EDIT: looks like Linux system tracking is more work, but possible https://perfetto.dev/docs/quickstart/linux-tracing https://perfetto.dev/docs/quickstart/linux-tracing
    • okr 2y ago
      Thank you!
• markhahn 2y ago
  honestly, I still don't get the point of wayland.
  • sam_lowry_ 2y ago
    Kill all the fun in writing window managers?
• Asooka 2y ago
  Why is there latency beyond a single frame at all?? I did a very quick replica of the camera frame counting on Windows with a 144Hz display and it's definitely below the camera's 60FPS, i.e. the cursor moves on the same frame that the mouse moves. Moving a window also has 0 lag in that test.
• phendrenad2 2y ago
  I don't like the design of Wayland at all, so it'll take a lot more than performance analytics to get me to switch. I think that one of the core issues in Linux is nothing is ever consolidated, only moved toward more fragmentation. Wayland had the opportunity to consolidate, but instead to not fix fragmentation (each app is responsible for drawing its own window borders) and introduce more (multiple compositors each with their own quirks and non-standard APIs)
• FrostKiwi 2y ago
  Somebody should point LDAT at that [1] or the custom setup with LED triggered by the mouseclick + a high FPS camera that Battlenonsense used [2] before LDAT released. [1] https://developer.nvidia.com/nvidia-latency-display-analysis-tool https://developer.nvidia.com/nvidia-latency-display-analysis... [2] https://www.youtube.com/watch?v=_MaeJbd1xaM https://www.youtube.com/watch?v=_MaeJbd1xaM
• a-dub 2y ago
  i run nvidia/intel/optimus/wayland in optimus/integrated mode on my laptop. i have anecdotally noticed perceptually salient increased response times when: 1) dpi scaling is in use 2) external displays are used via usb-c/displayport 3) the nvidia memory clock has been scaled down by power management that said there have been fantastic improvements overall to both stability and response times since kde6 and nvidia-open have stabilized and i haven't noticed any of it in a long time now. it really is pretty great these days.
• sho_hn 2y ago
  Note that Wayland is a protocol, not an implementation. This testing was done with Gnome's implementation of Wayland, which doesn't mean much for any other Wayland desktop.
• adeon 2y ago
  I used to be a NetHack speedrunner. Pressing the keys real fast. All of it in terminals. I had the fastest run for almost 10 years. Input lag was a thing I thought sometimes. Everything below is based on pure gut feelings because I never measured any of it. I've long wondered why some terminals and operating systems felt like they were pretty laggy. In the sense that if I press my character to move, how long until they actually move? iTerm2 on MacOS is worst offender. But I can't tell if it's iTerm2 or MacOS itself. I remember trying other terminals on Mac, and it was mostly the same. iTerm2 itself also had bunch of rendering options, but I couldn't get them to do anything I could actually feel affected input lag. One thought I had was: maybe iTerm2 added 1 frame lateness, and MacOS itself added another? But that would be ~30ms on a 60fps monitor which I can easily already tell on a server-connection NetHack. I also have no idea if measuring "N frames late" is actually a sensible way to think about this. I assume computers know when a key is pressed at a higher frequency? Linux xterm and rxvt were the fastest that I remember, on X11 (I've never used Wayland like in the article, well not for NetHack anyway). I have no idea how compositors work on any of these operating systems or X11/Wayland to say anything smart about them or why they would be slower or faster. Reading the article...I realized I have that same iPhone 15 Pro with 240fps recording. I could replicate that experiment, but testing out terminals on various operating systems instead I used to play on. So I could now test if my gut feeling was right all along or not. or maybe it lied to me all these years. I wrote the idea in the article down on my notes, maybe when I'm bored enough I'll try it :) and I can stop saying propaganda about iTerm2 being slow if it turns out I was completely wrong. Maybe I have easier time because all my NetHack days were 60fps, so I have more leeway in measurements. I'm not active in NetHack playing anymore, although I sometimes play it as a comfort game.
  • patal 2y ago
    iTerm2 is on 45ms and more in this measuring: https://danluu.com/term-latency/ https://danluu.com/term-latency/ There, it's distinct in that other terminals have lower latency on the same system.
    • adeon 2y ago
      Ooh that is interesting. Also noticed this passage: "When people measure actual end-to-end latency for games on normal computer setups, they usually find latencies in the 100ms range." That is really surprising to me. 100ms is HUGE. On NetHack, sshing into servers the difference between 50ms and 150ms is enormous. If do my experiment and I'm not too lazy I want to check on that 100ms figure, your link points to more sources which I didn't check on right now. I don't know if I'm blind but I can't tell when that article was written exactly. But it mentioned MacBook 2014 and Lubuntu 16.04 so maybe it's around mid 2010, 2016 possibly? (I should check properly if I do my experiments) The author in the Wayland vs X11 used mouse flicks on the video. Just now while writing I kind of had a thought if this needs some more deep thinking: how do I accurately measure "my finger touched this button and X milliseconds later character moved". Wondering what to consider as the "finger touched this button" event in a 240fps video. Maybe I can do what the author did and map a key to a mouse instead, because maybe there's a much-easier-to-see physical movement with a mouse. But then, am I really measuring faithfully my old NetHack-like environment? Or maybe this isn't really an issue and it'll be easy to tell. Too much for my small brain to think in the moment :) that link gave me at least some validation that iTerm2 maybe really is a bit slow, or at least was at that time. There's also bunch of terminals on that page I've never heard of. Today iTerm2 is my daily driver, but xterm is still really fast on my Fedora Linux system I have on the side.
• Grieverheart 2y ago
  Isn’t 3 frames of latency already way too much? I would say they are both too slow!
• oskenso 2y ago
  I'm glad more people are looking into this. Maybe one day we'll discover the cause of the Laggy Mouse bug which has been a problem since 2015 https://bugzilla.gnome.org/show_bug.cgi?id=745032 https://bugzilla.gnome.org/show_bug.cgi?id=745032 https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/749 https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/749
• chme 2y ago
  If you just want to measure user space software input lag, shouldn't it be more accurate if you print out the time when the event arrives in the kernel, and again when the mouse moves within a application window?
  • account42 2y ago
    Depends what you want to measure. But generally, people are more sensitive to latencies in movement of the visual cursor than they are to applications responding to it.
• BitPirate 2y ago
  Asahi Linas explanation: https://lobste.rs/s/oxtwre/hard_numbers_wayland_vs_x11_input_latency#c_edq7tn https://lobste.rs/s/oxtwre/hard_numbers_wayland_vs_x11_input...
  • pbasista 2y ago
    An excerpt from the linked comment's own TL;DR summary: > X11 has no cursor vsync and tears. Wayland compositors use atomic cursor updates synchronized with vsync.
• snvzz 2y ago
  Mouse is moved. Preemption happens. Hardware cursor is repositioned. The Amiga could do it in 1985. Linux has realtime scheduler classes (i.e. SCHED_FIFO and SCHED_RR). These days, even PREEMPT_RT. Some Wayland composers are already leveraging hardware cursors when possible. Why is it so hard? What's stopping them from accomplishing this?
• fx1994 2y ago
  Huh, that is one of the things why I never liked Linux Desktop at all. Switching from Windows to Linux Desktop feels... weird, laggy. People told me I was the problem, but it seems there are more people who noticed this lag. Maybe someone can fix this :)
• torginus 2y ago
  I have another somewhat unrelated pet peeve - when I started using Linux, the desktop was software composited, essentially what I assume was going on was that the UI was drawn by the CPU writing into the RAM which got blitted into the GPU's framebuffer, as it was done forever. When compositors (even with X11 composite) and GPU renderers becamse fashionable, the latency became awful, literally 100s of msecs. Which was weird considering, I grew up playing DOS games that didn't have this issue. Things improved over time but it's not clear to me if we ever got back to the golden days of CPU rendering.
• xbar 2y ago
  Cursor input latency is the continuous galling impediment to my satisfaction with an application or operating system. IDE text cursor delay? Unacceptable. Shell text cursor delay? Unacceptable. GUI mouse cursor delay? Unacceptable. All are immediate deal-breakers for me.
  • mcv 2y ago
    Yeah, I'm baffled this is so bad. Especially in Linux. Surely moving the mouse should take precedence over everything else? Even highlighting the thing you move the mouse over shouldn't slow down the mouse cursor itself. I recently had a problem with extreme mouse slowdowns on KDE/Plasma 6 on X11 with NVidia. I noticed the slowdown was particularly extreme over the tabs of my browser. The fix, in case anyone else also has this problem, is to disable OpenGL flipping. I have no idea what OpenGL flipping does, but disabling it (in nvidia-settings and in /etc/X11/xorg.conf) fixed the problem. I'd love it if someone could explain why. Or why this isn't the default.
    • kllrnohj 2y ago
      OpenGL flipping is iirc the weird name nvidia gives for vsync. Disabling it means you're going to have tearing artifacts, hence why it's not the default.
      • mcv 2y ago
        But why does it slow down the mouse? Is there no way to get both? Maybe I should try if Wayland gets me both, although at the moment it just gives me a black screen.
  • jonathaneunice 2y ago
    One of the places that macOS has always excelled, all the way back to classic Mac OS. Even System 1 on almost infinitely less powerful gear 40+ years ago super-prioritized cursor responsiveness, outshining all competitors. The OS and UI underpinnings are now 100% different, but the priority remains.
• bArray 2y ago
  > Interestingly, the difference is very close to 1 full screen refresh. I don't know whether or not that's a coincidence. Some sort of double buffering? To ensure it's consistently one frame, it would make sense to test on a slower computer via the same monitor. There was some discussion on HN in 2017 regarding latency differences: https://news.ycombinator.com/item?id=15748146 https://news.ycombinator.com/item?id=15748146 As mentioned in those comments, it could be from the compositor
• t43562 2y ago
  The latency issue isn't blocking me from trying Wayland. Wayland seems fine on my Raspberry Pi. The issue is screen-sharing in MS Teams. Does anyone know if that works now? Since I use Artix, I expect a potentially painful switch and don't really want to have to then do it all again to switch back immediately.
  • gmokki 2y ago
    Few options. Use a browser that has supported screen sharing on Wayland for many years: Firefox. Or petition Microsoft to enable the Wayland-enable bits in their electron build. Electron has supported screen sharing on Wayland for soon 5 years.
• dartos 2y ago
  The unfortunate thing about Wayland, imo, is that most people think it’s a single implementation like x11
• phkahler 2y ago
  Since this is all mouse pointer relates and there is hardware for that, I can only assume it's a fixable problem. OTOH the gnome developers have been talking about implementing triple buffering in the compositor. Tripple buffering is only needed when your renderer can't complete a frame faster than the scanout of a frame. Given that any modern GPU should be able to composite hundreds of frames per second this make me think something about Wayland or the gnome compositor in particular isn't designed as well as it could be.
  • PeakKS 2y ago
    Dynamic triple buffering is basically just giving the hardware time to clock itself up when there is a sudden change in workload, e.g. idling then opening the overview (drawing basically nothing -> redrawing basically the whole desktop).
• nbf_1995 2y ago
  Reposting my analysis I performed with a Rpi pico, oscilloscope and a photo diode from another site: I created a test setup using a Raspberry Pi Pico running CircuitPython and a USB HiD library to emulate mouse clicks. I connected button switch to a GPIO pin and polled the state of the pin to send mouse click events over usb to my laptop. I connected a photo diode to the power rail of the pico and pointed it at the screen. Finally I connected the push button to channel 1 of my oscilloscope and the photo diode to channel 2. I created small application with raylib which would toggle the screen from black to white when clicked. When the screen changes from white to black, the voltage across of the photo diode would drop, which would create a falling edge I could trigger on using the oscilloscope. Test Setup: I ran the following tests on my AMD Framework 13, under CachyOS with all packages up to date. For each test I set the Raylib application to run at 800x600@60Hz and left the photo diode in place near the bottom right corner of the screen. For each test, the display output scaling was set to 1x (2256x1504), and variable refresh rate was disabled. Using the oscilloscope I measured from the first trigger buffer sample that the voltage of the button reached steady state to the first sample where the voltage on the photo diode dropped below steady state to 1.5v Caveats: The times seem high, but they encompass the entire chain of Push button -> App rendering next frame. There is some loss in time resolution because of the abysmally small size of my cheap oscilloscope’s trace buffer. I only collected 9 samples for each configuration because I’m clumsy on the controls of the oscilloscope and this took a little bit. The raylib application was running under Xwayland for the Wayland tests which somewhat skews the results. Results: KDE Plasma (X11): 154ms, 134ms, 136ms, 137ms, 126ms, 144ms, 135ms, 142ms, 150ms => Avg 139ms KDE Plasma (Wayland): 181ms, 188ms, 139ms, 148m, 137ms, 156ms, 163ms, 164ms, 151ms => Avg 158ms Sway (Wayland): 172ms, 175ms, 168ms, 153ms, 144ms, 152ms, 161ms, 172ms, 156ms => Avg 161ms
• zwolf24 2y ago
  [dead]