13 ms·
Improving Xwayland window resizing
- anotherhue 2y agoI like to run xeyes and see if they move when I interact with the test application. It's not better but they're as charming as they always were. Also consider running an xwayland app under gamescope which smooths out some issues.
- yjftsjthsd-h 2y agoI would argue that it is better; it works regardless of this fix and on all compositors.
- anyfoo 2y agoThat’s actually something you don’t want nowadays. A random app like xeyes should not be able to know mouse position at all times, for privacy reasons. (Unless you very explicitly gave xeyes super-extra permission to do that.) We’ve long stopped living in a world where you should need to fully trust every piece software that you run as your user on your computer. And even probably trustworthy software can go bad at an update due to supply-chain attacks.
- ChocolateGod 2y ago> A random app like xeyes should not be able to know mouse position at all times, for privacy reasons. (Unless you very explicitly gave xeyes super-extra permission to do that.) The thing about xeyes isn't that its privacy invasive, is that it shows that xeyes knows what you're doing in other clients. Got a gnome terminal root shell open? That's a privilege escalation method for any other client running on the desktop under Xorg. This itself, isn't really a problem, but chained with other attacks could be (e.g. browser escape).
- lupusreal 2y ago> Got a gnome terminal root shell open? That's a privilege escalation method for any other client running on the desktop under Xorg. This itself, isn't really a problem, but chained with other attacks could be (e.g. browser escape). Unless you're sandboxed up the ass, Wayland won't save you when that browser escape happens. Something I did 20 years ago to a friend as a prank that still works today on a typical Linux desktop with Wayland; wrap sudo to log the users password the next time they use it. I didn't use a browser exploit for that, but it can easily be done if you have write access to the user's environment however that happened. Wayland won't protect you from that sort of thing unless you're willing to commit to extensive sandboxing.
- bitwize 2y agoWayland is a critical step in sandboxing everything on the Linux desktop up the ass. Flatpak is also part of this effort. This is where desktop computing is headed; and why what Drew DeVault called "anti-Wayland horseshit" is actually derailing a secure, easy-to-use Linux desktop.
- dgfitz 2y agoIsn’t is possible to get something like mouse position from almost any stdlib of most languages?
- seba_dos1 2y agoNo, not really. Modern APIs usually don't even let you access that kind of information without additional privileges. Some older toolkits have functions that are supposed to do this, but it doesn't work everywhere.
- dgfitz 2y agoCan’t you get the cursor position from the windows c++ stdlib? Or like pyautogui? Or the Java stdlib?
- adzm 2y agoWindows' pointer position is available to even the most limited GUI application; it is not protected information. For win32 at least, I am not familiar enough to say that about the newer app packages which are much more locked down, but I would be surprised.
- anyfoo 2y agoHopefully only if the given app has focus.
- funcDropShadow 2y agoWhy? I want to have some applications that can always see the mouse cursor like, xeyes. Because that allows me to implement a better customized desktop environment.
- anyfoo 2y agoThen give that piece of code extra-special permissions. As I’ve said in another comment, the days where you downloaded your software from the sunsite or tsx11.ai.mit.edu FTP servers and could be confident that it and all its dependencies were trustworthy are unfortunately gone for a very long time.
- superkuh 2y agoIt really is something I want. But most people do live in your described world. That's the smartphone and "run's every javascript application sent to my web browser automatically" kind of computing security model. But there do exist personal behaviors of desktop operating system use where you can actually trust the applications installed and not put up walls between everything. And then things like keyboard/mouse sharing work, windows return to their places, and screen readers work. Those are all very important to me. For them I'm willing to browse with javascript temp-whitelist-only and many other such tedious things. What I'm trying to get across is that the need for that kind of intense security model, every process a threat, is not intrinsic to modern computing.
- kaba0 2y agoBut what’s the limiting factor of doing the sane and safe thing by default? The most popular operating systems all do that (ios and android), and they have carved out safe APIs for all of that to work. You can’t patch up a Swiss cheese after the fact. Is it hard to create standard APIs in a bazaar style of development? Yeah. But that doesn’t mean that it’s not the correct approach.
- realusername 2y agoI think their concern is valid, it's difficult to do something which is both secure AND not limited at the same time. Sure Android and iOS are secure but in practice they kind of suck for making anything non-standard which limits creativity and freedom. Can we have both a secure and extendable system? Maybe but none of them exist yet. I'm really worrying that Linux mainstream distros will become like Android or iOS.
- jcelerier 2y agoNo one wants to use android and iOS for serious desktop work though. Like it's cool when your only interaction with the device is consuming content, definitely not for creating.
- kaba0 2y ago
- kaba0 2y agoAnd an often under appreciated tenet of security — even a “good” software can be exposed to “bad” data, and you only need a bug (especially a memory bug, which is exceedingly common because linux userspace can’t get rid of c for the life of it) to have arbitrary code executed. Like, your pdf reader is surely not evil, but do you trust every single pdf file you open?
- account42 2y agoI expect my PDF reader to be secure. If the PDF format is too complex to implement safely then the renderer should be sandboxed in the reader itself instead of preventing me from scripting using xdotool and similar. And unless you fully sandbox your PDF reader then an exploit is going to have access to your user directory without any display server involvement anyway. X11 vs. Wayland doesn't even come into the picture.
- kaba0 2y agoIt shouldn’t complicate the program itself, everything should be sandboxed by default. And they should simply not have access to my home folder, it should be given access to a specific file only it is about to read.
- account42 2y agoThat severy limits the usability and even functionality that programs can implement. If you want a phone os then go use one but don't make Desktop Linux into one.
- ykonstant 2y ago> That’s actually something you don’t want nowadays. No, that is something you don't want. I and many others, do want this functionality.
- consteval 2y agoEven you don't want this, you only want it sometimes, for some apps. Which is exactly what they said. I mean, would you like VSCode tracking your mouse movements across the entire desktop and your keypresses and then sending them off to Microsoft? Probably not, so we're all in agreement.
- account42 2y agoI don't use VSCode or other user hostile programs like it so I don't care what kind of anti-features it enables. I don't want my actually useful tools hobbled in order to deal with such programs.
- anyfoo 2y agoxz wasn’t “user hostile”, and so weren’t countless other pieces of software affected by supply chain attacks. Nothing is hobbled if you can give it explicit permission (which you may well do on xeyes). The days where you downloaded your software from the sunsite or tsx11.ai.mit.edu FTP servers and could be confident that it and all its dependencies were trustworthy are unfortunately gone for a very long time.
- account42 2y agoThe xz-utils hack didn't care about the window system at all. It also hasn't actually caused any known damage and I'm sure if the american three letter agencies cared the perpetrator would have been dealt with by now. Project takeovers of that kind is not something regular users need to be worried about because the cost of pulling one off is too righ to waste on petty crime. It's yet another boogeyman and scaring people into giving up their computing freedom for "security" migh as well have been the goal of the operation.
- yxhuvud 2y agoHmm. Even with the fix it didn't look very smooth even if it is a lot better. I wonder if that is due to the recording, or due to the mouse settings (mouse poll rate and acceleration settings are very impactful on smoothness), due to X or the toolkit, or due to it not being as good as it can be.
- GrantMoyer 2y agoI think issue is the screen capture tool or a framerate mismatch in transcoding. The video is 59fps, but the mouse cursor only updates every few frames. Usually it's every other frame, but sometimes it takes three frames or more to update, and sometimes it updates in two or more consecutive frames. On most of the frames where the cursor moves, the window size also changes.
- Arainach 2y agoExcellent writeup, thanks for posting. I was involved with efforts to achieve similar things in Windows (Direct Composition, etc.). In practice, it was enlightening to debug various apps and frameworks and discover just how long repainting can take and how many edge cases there are around slow apps, hung apps, and so on. Who knows, maybe someone there still has some of the malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.). It's definitely a hard problem space that's basically incompatible with how most GUI apps and frameworks worked from 1985-2015.
- ack_complete 2y agoIMO, the neatest trick that the DWM does is avoiding jitter/lag between the mouse cursor and window when dragging a window. Sadly, DXGI flip model seems to have reintroduced some issues -- it is difficult to avoid the kind of jank shown in this article when resizing a window drawn using flip mode presentation. Which is unfortunate, since it's the highest performing mode.
- jchw 2y agoThat trick is fun, but I wish Windows did a better job of synchronizing the visibility of the cursor sprite versus the presentation of the frames where the cursor is rendered in the frame. It's impossible for me to not notice the brief period where the cursor is missing!
- mouse_ 2y agoI think it's cute. They're doing "disable hardware cursor, enable software cursor" instead of the other way around. Seeing that happen is somehow fun for me.
- jchw 2y agoI think they do both simultaneously, it's just that the cursor sprite/plane is overlaid during scan out and the software cursor is rendered in the next frame to be rendered by the compositor. The cursor sprite/plane can turn on or off pretty much mid-frame, whereas the frame that's about to be rendered could be two frames away from the one that's about to be scanned out. What you actually need to do is track when the first frame with the software cursor is about to be presented, then turn off the cursor sprite/plane. I think technically speaking all of the information needed to implement this is present somewhere, but I'm sure it's quite complicated to actually implement, possibly made worse by interface layering issues :)
- o11c 2y agoFor me, the quickest way to tell that I'm running on Wayland is seeing if multi-lingual keyboard input is broken. I have right-alt-while-pressed set to switch to a Greek layout (because who doesn't math?), and it reliably works on X11 but breaks in all sorts of weird ways under Wayland (sometimes it simply fails to enable/disable, sometimes it shows a notification as if it changed but switches back when input actually happens, ...). On the bright side I'm finally learning how Vim's digraphs work.
- porphyra 2y agoMultilingual keyboard input is such a mess haha. I need to type a lot of Chinese using pinyin and still haven't found a reasonably polished implementation on Wayland (Sway) that works on both Xwayland and true Wayland applications. So I reverted to i3 with ibus.
- przmk 2y agoFor what it's worth, the latest 1.10 release of sway introduced support for IME popups. Maybe that would help
- pmontra 2y agoReally? My laptop has my national keyboard but I switch to USA for programming because of {} [] and all the other characters that can be typed with minimal use of extra keys. My fingers know the keys and the switch combination, maybe windows + space. I didn't expect Wayland not to have such basic functionality after so many years. But frames must be perfect, no tearing (is that the word?)
- ryandrake 2y agoNice results. It's kind of wild that the "before" behavior would be considered acceptable/shippable by anyone. That kind of jank should be a show stopper.
- porphyra 2y agoLinux desktop is full of such jank. Fortunately, thanks to contributors like OP, the jank is gradually getting fixed. The whole point of Wayland is to eliminate a whole class of jank (screen tearing) that was essentially unfixable with X.
- o11c 2y agoAnd yet, empirically, jank is far more common on Wayland than X11. With X11, on the rare case I noticed, it was usually just a matter of "okay, which toolkit or driver do I need to look up the vsync environment variable for?" I have no opposition to Wayland in theory; my concerns are entirely practical and unignorable.
- porphyra 2y agoYeah I have experienced all sorts of flickering and artifacts on Wayland, especially on nvidia. Recently it has gotten a lot better though.
- seba_dos1 2y agoFWIW, Nvidia problems on Wayland were Nvidia's problems, not Wayland's. Wayland was just a victim of Nvidia's bugs and lacking implementations.
- p_l 2y agoExcept for tearing, which turned out to be a combination of Mesa/DRM bugs and Wayland naively assuming the behaviour of those bugs. Then nVidia decided to switch to GBM/EGL way of doing things and it turned out everyone had incorrect assumptions...
- deleted 2y ago[deleted]
- whywhywhywhy 2y agoHuge respect to any dev who goes out of their way to fix flickering/glitching/black regions when resizing or moving windows, I honestly can't stand it when I see it because I know it can be solved with effort.
- shmerl 2y agoNice! It would be cool for Wine Wayland driver to also fix window resizing (regedit currently is blinking all over when being resized).
- snvzz 2y agoIs this really wine on wayland, or is it the default wine on xwayland on wayland?
- sph 2y agoHoping this makes Xwayland Emacs much smoother to resize. It currently resizes at 0.5 fps for me on KDE. If you're wondering why I don't use the native Wayland build of Emacs, it's because it is massively more laggy on HiDPI screens than the X11 version. I reported the issue upstream, probably won't be fixed for a decade when they'll decide to port to GTK4 and its hardware accelerated rendering backend. It took me a long while to notice that the typing lag I was experiencing was not because of native compilation or single-threading, but pgtk being a bit weak at rendering 3840x2160 pixels at 60 fps. In fact, it was not until I tried the Xwayland build that I experienced how much faster Emacs can feel.