15 ms·

Isolating Xwayland in a VM

111 points by pmarin 5y ago

temptemptemp111 5y ago
razemio 5y ago
Amazing! Thank you for sharing. I have to admit that I lack a lot in security related stuff. Reading your post is inspiring. I am thinking about experimenting with proxmox to run a similar setup. Currently I isolate each customer in its own vm. This works well but comes with the overhead to run a remote session for each customer. Using your method I could have a much more streamlined work flow while maintaining a high level of isolation.
hhh 5y ago
Spectacular article, I couldn’t express more gratitude for the deep context provided around every piece.
xvilka 5y ago
Useful approach, given the troves of applications that cannot be ported to the native Wayland in years to come - all GTK2 apps, e.g. GIMP, all Java apps, and so on... Regarding HiDPI and XWayland, there is work-in-progress PR[1] that so far is being ignored by the Wayland developers. [1] https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/733 https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests...
- CyberRabbi 5y ago
  > Regarding HiDPI and XWayland, there is work-in-progress PR[1] that so far is being ignored by the Wayland developers. Just curious, what are the Wayland developers expected to do? This seems like an Xwayland and compositor implementation issue.
  - deleted 5y ago
    [deleted]
- pjmlp 5y ago
  Java does not depend on Gtk 2, unless you're stuck with some pre-historic version. As for Wayland, https://wiki.openjdk.java.net/display/wakefield/OpenJDK+Project+Wakefield+-+Wayland+desktop+support+for+JDK+on+Linux https://wiki.openjdk.java.net/display/wakefield/OpenJDK+Proj...
  - xvilka 5y ago
    I know, those were two separate categories. Regarding the Wayfield project, citing them: > It is very unlikely that distros will be ready to ship all the pieces we need in the next 12 months. So the "short term" goal may actually need to wait for somewhat longer than that. https://wiki.openjdk.java.net/display/wakefield/Meeting+Notes https://wiki.openjdk.java.net/display/wakefield/Meeting+Note...
    - pjmlp 5y ago
      Given the progress adopting Wayland I don't see that much of a problem that it will take some time to deliver a full working solution. The way you made your remark could be understood as if nothing was being done at all.
      - xvilka 5y ago
        It's indeed not a problem if the Wayland sessions wouldn't become the default in mainstream distributions and these pesky HiDPI displays didn't become more affordable and popular among users. Don't get me wrong, the blame is squarely on the Java ecosystem that is too slow to keep up with the progress, Wayland developers did great job. The problem is that we still use some of that Java software, or some of that GTK2 software (GIMP), so it's better to accomodate the needs of that software, even if just temporarily.
        pjmlp 5y ago
        Other than ChromeOS and WSL, hardly any distribution is Wayland only, so one can't really blame the Java community to focus improving Swing on Wayland, specially with other more relevant areas still in need. If anything the whole GNU/Linux ecosystem is too slow adopting Wayland.
- emersion 5y ago
  > ignored by the Wayland developers Feel free to improve and/or review it. Maybe it's very important to you. But it may no be very important to other contributors, and said contributors are just volunteers.
  - xvilka 5y ago
    To be honest, it's not very important to me - I need this particular case on the one machine where I use JetBrains IDE. On this machine I just switched to the Xorg. Everywhere else, I am quite happy with the pure Wayland as is, kudos to developers. It is important [1] for many JetBrains users, it seems though. [1] https://youtrack.jetbrains.com/issue/JBR-3206 https://youtrack.jetbrains.com/issue/JBR-3206
- 1_player 5y ago
  > PR that so far is being ignored by the Wayland developers The reviewers have asked for some details about the changes and no response yet, it seems unfair to blame the maintainers here. But of course that goes against the narrative of "Wayland bad".
  - xvilka 5y ago
    I never said "Wayland is bad". Some cases aren't handled properly before making it mainstream in Linux distributions. It should have been done before, not after.
    - shatteredgate 5y ago
      Open source doesn't really work like that, there is no top-down decision maker setting priorities for what should be done in any given distribution. Usually what happens is you wait until it becomes enough of a problem that some distribution fixes it, but this one is low priority given that distributions would probably prefer to focus on native Wayland ports for the software they have control over that is packaged in their distribution. And for the external software they don't have control over, it's much harder to find people to work on those.
      - chaosharmonic 5y ago
        ^ further, there's the fact that distributions (and the various parts therein) frequently duplicate one another's work in the name of taking different approaches at the same problem, and usually the solutions that become more mainstream are the ones that several of them have collectively settled on. Android has a similar, if more centralized feedback loop: AOSP drops, device makers build out customizations on top of it, Google adopts some of these features upstream (whether by building new implementations or merging contributions from vendors), and then those vendors go on to building the next differentiating thing now that they don't have to dedicate those resources to individually maintaining stylus support/multiwindow/notification badges/whatever.
- kirbyfan64sos 5y ago
  That's my MR and I haven't had time to complete it (particularly a compositor impl to go along with it), has nothing to do with being ignored.
  - floatboth 5y ago
    > particularly a compositor impl to go along with it Well, there is a wlroots impl, so I've just added a couple lines to wlroots to invoke this support from an env variable without having to modify the actual compositor :D https://github.com/unrelentingtech/wlroots/commit/5b8308651cb6942ac3079cbdab7fcbc7a41bd577 https://github.com/unrelentingtech/wlroots/commit/5b8308651c... but the wlroots impl isn't merged either, so I think there's kind of a mutual dependency here. Either your MR or the wlroots MR should go in first!
myelin 5y ago
This is very very cool. I had coincidentally just spent my evening trying to figure out how to get Wayland and Sommelier working in a slightly odd environment (Crouton, on Chrome OS), and this article showed up on HN just in time for me to appreciate it :)
- pa7ch 5y ago
  Just curious, why did you set that up on crouton vs just using crostini?
Kototama 5y ago
His blog is really great but last time I didn't find a way to post a comment or to contact the author. Do somebody knows better?
- jeroenhd 5y ago
  The author links to his github account, you should be able to extract _some_ kind of email address from the git commit log. To me it seems that the author doesn't list contact details on purpose. There's an extensive "about me" section without any contact information, so I think the author might wish not to be contacted. Edit: actually, his Gmail is in one of the screenshots in this article.
- talex5 5y ago
  I usually notice if my blog gets on Hacker News fairly soon, so I'll see any comments posted here. I prefer not to use email for most things because then the reply only benefits one person, whereas replies in a public forum can be read by others and get indexed by search engines. I've been thinking about creating a Matrix room for each blog post as a discussion forum, but so far most posts have ended up on other discussion sites anyway.
  - gostsamo 5y ago
    I think that some blogs have email lists as discussion forum with each thread devoted to one post, but I can't come up with examples right now. If the email list archive is public, it might be the perfect solution.
  - Kototama 5y ago
    Regarding the performances issues that you had with Qubes: I once disabled power management on one of my laptop and it moves Qubes from barely usable to usable. Still close to unusable: Web conferences in the browser (webex for example).
kelnos 5y ago
This is a bit off topic, but the article reminded me of something that's always bothered me: one of the (several) motivations for creating Wayland was to fix the security issues with X11, namely that any X client can read and manipulate the windows belonging to any other X client. And I've heard claims that this problem is just unsolvable. It... doesn't seem like it is, really. Couldn't we modify the X server so clients just can't do that anymore? And then introduce a new X extension that provides for cases where this does need to happen, like the window manager (which could be granted blanket permissions), and other apps that would cause a dialog to pop up where the user could approve or deny access (think apps that take screenshots, or video conferencing apps that share the screen or specific windows with other participants). This permissions dialog could be handled by the window manager, or perhaps some standalone app that is somehow securely registered with the X server via this new X extension. I expect there are some other issues (and things that will break[0]) with apps not knowing about other apps' windows, but I'm not convinced those issues can't be solved. I could even imagine that this new regime might only implement partial isolation, where clients can't read pixels from or fake events to other clients' windows (among other things), but can know the sizes and positions of other windows, and maybe even read some whitelisted set of window properties. I think that would still address most, if not all, security concerns with the current model. (Just to present some credentials so this doesn't come off as a clueless, "duh, this is so easy, you morons"-type post: I've written an X11 compositor, hacked on a window manager, built a simple toy WM for fun, and was once a co-maintainer of a reasonably popular desktop environment. So I'm not a complete noob here.) Certainly there are other benefits to Wayland over X11, not just security: greatly simplifying the graphics situation, moving the compositor into the "server" where IMO it belongs, etc. But I'm still not completely sold on the idea that this is worth a multi-decade project to completely replace this critical bit of the Linux GUI stack. I'm also not convinced that the X server can't be extended in other ways to fix other deficiencies. Of course, at the end of the day, I'm not doing the work, and the nature of open source means that if everyone wants to work on Wayland and no one wants to work on X11 or the xorg server, that's just the way it will be. But it feels like if we'd put even a fraction of the Wayland effort toward making X11 better, we'd be done by now, and people writing X11 applications would mostly not have had to do all that much work to migrate (and many wouldn't have to do any). Would it be as objectively "good" as Wayland supposedly is? No, probably not, but perhaps that doesn't matter. [0] One thing I can think of here is that some applications need to open more than one connection to the X server, and the X server would see them as separate clients. That could break an application that expects to be able to share and manipulate resources between the multiple connections. But perhaps that could be worked around by grouping permissions not solely by connection, but by process ID. There are probably still some edge cases here, but I think changing the apps here would be ok to do; at least, it seems like less work to accept this sort of breakage and try to fix the apps rather than develop an entirely new windowing system!
southerntofu 5y ago
Friendly reminder that QubesOS, mentioned in the article, is a great system if you have actual security needs. Their blog is full of treasures explaining what their research and development is about. https://www.qubes-os.org/news/ https://www.qubes-os.org/news/
- xvilka 5y ago
  There's also a project[1][2][3] to allow Windows-like containers inside, based on the ReactOS. [1] https://github.com/QubesOS/qubes-issues/issues/2809 https://github.com/QubesOS/qubes-issues/issues/2809 [2] https://reactos.org/forum/viewtopic.php?t=16480 https://reactos.org/forum/viewtopic.php?t=16480 [3] https://github.com/Jeeppler/QubesOS-notes/blob/master/ReactOS/Qubes-ReactOS.md https://github.com/Jeeppler/QubesOS-notes/blob/master/ReactO...
tyingq 5y ago
Interesting. WSLG seems to do this as well, bundling Xwayland, Pulse Audio, and Weston into a single separate VM that serves all the user-installed WSL distros on a box. https://devblogs.microsoft.com/commandline/wp-content/uploads/sites/33/2021/04/diagram-description-automatically-generated.png https://devblogs.microsoft.com/commandline/wp-content/upload...
ncmncm 5y ago
[Edit: I found the blog entry from March that mentions SpectrumOS.] Maybe you should know about the project SpectrumOS, which seems to have extremely similar goals to yours. https://spectrum-os.org/ https://spectrum-os.org/ I think instead of running persistent VMs for different roles, it spins up VMs very quickly to run individual apps in.
ncmncm 5y ago
It should be noted that Qubes is thus far the only actually secure X environment. The hardware X server runs in a secure X VM, which has exclusive access to input devices, the GPU, and the physical frame buffer memory. (Ordinary X gives all programs access to all traffic from all input devices, and the whole frame buffer.) Each client VM runs an X server that really only manages window contents. These windows are mmapped to memory the secure X VM exposes from its address space, It copies pixels from the shared regions to the real frame buffer. UI events that happen in each such window are forwarded to the client VM that owns it. Regular desktop "panel" widgets and other UI stuff is managed by the secure X VM. USB devices can be attached, via the secure X UI, to a chosen client VM, or taken back. Likewise, input devices such as microphones. The VM only gets PulseAudio streams forwarded from a microphone, and has no access to the sound chip. The only awkward thing is that client VMs have no access to any GPU acceleration. This is not a problem for normal use -- browsers all can use CPU-only graphics modes, and they are fast enough for the usual things, including youtube videos, albeit with increased power consumption.
ossusermivami 5y ago
Very good article, the linked code about vim is incredible tho (and not the good kind of it), I have actually spent a good part of my evening trying to understand how they end up with that implementation (I do have the same kind of spaghetti code and wonder the same about my code) : https://github.com/vim/vim/blob/9cd063e3195a4c250c8016fa340922ab21fda252/src/gui.c#L489 https://github.com/vim/vim/blob/9cd063e3195a4c250c8016fa3409...

111 points by pmarin 5y ago

Amazing! Thank you for sharing. I have to admit that I lack a lot in security related stuff. Reading your post is inspiring. I am thinking about experimenting with proxmox to run a similar setup. Currently I isolate each customer in its own vm. This works well but comes with the overhead to run a remote session for each customer. Using your method I could have a much more streamlined work flow while maintaining a high level of isolation.

hhh 5y ago

Spectacular article, I couldn’t express more gratitude for the deep context provided around every piece.

xvilka 5y ago

Useful approach, given the troves of applications that cannot be ported to the native Wayland in years to come - all GTK2 apps, e.g. GIMP, all Java apps, and so on... Regarding HiDPI and XWayland, there is work-in-progress PR[1] that so far is being ignored by the Wayland developers. [1] https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/733 https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests...

CyberRabbi 5y ago
> Regarding HiDPI and XWayland, there is work-in-progress PR[1] that so far is being ignored by the Wayland developers. Just curious, what are the Wayland developers expected to do? This seems like an Xwayland and compositor implementation issue.
- deleted 5y ago
  [deleted]
pjmlp 5y ago
Java does not depend on Gtk 2, unless you're stuck with some pre-historic version. As for Wayland, https://wiki.openjdk.java.net/display/wakefield/OpenJDK+Project+Wakefield+-+Wayland+desktop+support+for+JDK+on+Linux https://wiki.openjdk.java.net/display/wakefield/OpenJDK+Proj...
- xvilka 5y ago
  I know, those were two separate categories. Regarding the Wayfield project, citing them: > It is very unlikely that distros will be ready to ship all the pieces we need in the next 12 months. So the "short term" goal may actually need to wait for somewhat longer than that. https://wiki.openjdk.java.net/display/wakefield/Meeting+Notes https://wiki.openjdk.java.net/display/wakefield/Meeting+Note...
  - pjmlp 5y ago
    Given the progress adopting Wayland I don't see that much of a problem that it will take some time to deliver a full working solution. The way you made your remark could be understood as if nothing was being done at all.
    - xvilka 5y ago
      It's indeed not a problem if the Wayland sessions wouldn't become the default in mainstream distributions and these pesky HiDPI displays didn't become more affordable and popular among users. Don't get me wrong, the blame is squarely on the Java ecosystem that is too slow to keep up with the progress, Wayland developers did great job. The problem is that we still use some of that Java software, or some of that GTK2 software (GIMP), so it's better to accomodate the needs of that software, even if just temporarily.
      - pjmlp 5y ago
        Other than ChromeOS and WSL, hardly any distribution is Wayland only, so one can't really blame the Java community to focus improving Swing on Wayland, specially with other more relevant areas still in need. If anything the whole GNU/Linux ecosystem is too slow adopting Wayland.
emersion 5y ago
> ignored by the Wayland developers Feel free to improve and/or review it. Maybe it's very important to you. But it may no be very important to other contributors, and said contributors are just volunteers.
- xvilka 5y ago
  To be honest, it's not very important to me - I need this particular case on the one machine where I use JetBrains IDE. On this machine I just switched to the Xorg. Everywhere else, I am quite happy with the pure Wayland as is, kudos to developers. It is important [1] for many JetBrains users, it seems though. [1] https://youtrack.jetbrains.com/issue/JBR-3206 https://youtrack.jetbrains.com/issue/JBR-3206
1_player 5y ago
> PR that so far is being ignored by the Wayland developers The reviewers have asked for some details about the changes and no response yet, it seems unfair to blame the maintainers here. But of course that goes against the narrative of "Wayland bad".
- xvilka 5y ago
  I never said "Wayland is bad". Some cases aren't handled properly before making it mainstream in Linux distributions. It should have been done before, not after.
  - shatteredgate 5y ago
    Open source doesn't really work like that, there is no top-down decision maker setting priorities for what should be done in any given distribution. Usually what happens is you wait until it becomes enough of a problem that some distribution fixes it, but this one is low priority given that distributions would probably prefer to focus on native Wayland ports for the software they have control over that is packaged in their distribution. And for the external software they don't have control over, it's much harder to find people to work on those.
    - chaosharmonic 5y ago
      ^ further, there's the fact that distributions (and the various parts therein) frequently duplicate one another's work in the name of taking different approaches at the same problem, and usually the solutions that become more mainstream are the ones that several of them have collectively settled on. Android has a similar, if more centralized feedback loop: AOSP drops, device makers build out customizations on top of it, Google adopts some of these features upstream (whether by building new implementations or merging contributions from vendors), and then those vendors go on to building the next differentiating thing now that they don't have to dedicate those resources to individually maintaining stylus support/multiwindow/notification badges/whatever.
kirbyfan64sos 5y ago
That's my MR and I haven't had time to complete it (particularly a compositor impl to go along with it), has nothing to do with being ignored.
- floatboth 5y ago
  > particularly a compositor impl to go along with it Well, there is a wlroots impl, so I've just added a couple lines to wlroots to invoke this support from an env variable without having to modify the actual compositor :D https://github.com/unrelentingtech/wlroots/commit/5b8308651cb6942ac3079cbdab7fcbc7a41bd577 https://github.com/unrelentingtech/wlroots/commit/5b8308651c... but the wlroots impl isn't merged either, so I think there's kind of a mutual dependency here. Either your MR or the wlroots MR should go in first!

myelin 5y ago

This is very very cool. I had coincidentally just spent my evening trying to figure out how to get Wayland and Sommelier working in a slightly odd environment (Crouton, on Chrome OS), and this article showed up on HN just in time for me to appreciate it :)

pa7ch 5y ago
Just curious, why did you set that up on crouton vs just using crostini?

Kototama 5y ago

His blog is really great but last time I didn't find a way to post a comment or to contact the author. Do somebody knows better?

jeroenhd 5y ago
The author links to his github account, you should be able to extract _some_ kind of email address from the git commit log. To me it seems that the author doesn't list contact details on purpose. There's an extensive "about me" section without any contact information, so I think the author might wish not to be contacted. Edit: actually, his Gmail is in one of the screenshots in this article.
talex5 5y ago
I usually notice if my blog gets on Hacker News fairly soon, so I'll see any comments posted here. I prefer not to use email for most things because then the reply only benefits one person, whereas replies in a public forum can be read by others and get indexed by search engines. I've been thinking about creating a Matrix room for each blog post as a discussion forum, but so far most posts have ended up on other discussion sites anyway.
- gostsamo 5y ago
  I think that some blogs have email lists as discussion forum with each thread devoted to one post, but I can't come up with examples right now. If the email list archive is public, it might be the perfect solution.
- Kototama 5y ago
  Regarding the performances issues that you had with Qubes: I once disabled power management on one of my laptop and it moves Qubes from barely usable to usable. Still close to unusable: Web conferences in the browser (webex for example).

kelnos 5y ago

This is a bit off topic, but the article reminded me of something that's always bothered me: one of the (several) motivations for creating Wayland was to fix the security issues with X11, namely that any X client can read and manipulate the windows belonging to any other X client. And I've heard claims that this problem is just unsolvable. It... doesn't seem like it is, really. Couldn't we modify the X server so clients just can't do that anymore? And then introduce a new X extension that provides for cases where this does need to happen, like the window manager (which could be granted blanket permissions), and other apps that would cause a dialog to pop up where the user could approve or deny access (think apps that take screenshots, or video conferencing apps that share the screen or specific windows with other participants). This permissions dialog could be handled by the window manager, or perhaps some standalone app that is somehow securely registered with the X server via this new X extension. I expect there are some other issues (and things that will break[0]) with apps not knowing about other apps' windows, but I'm not convinced those issues can't be solved. I could even imagine that this new regime might only implement partial isolation, where clients can't read pixels from or fake events to other clients' windows (among other things), but can know the sizes and positions of other windows, and maybe even read some whitelisted set of window properties. I think that would still address most, if not all, security concerns with the current model. (Just to present some credentials so this doesn't come off as a clueless, "duh, this is so easy, you morons"-type post: I've written an X11 compositor, hacked on a window manager, built a simple toy WM for fun, and was once a co-maintainer of a reasonably popular desktop environment. So I'm not a complete noob here.) Certainly there are other benefits to Wayland over X11, not just security: greatly simplifying the graphics situation, moving the compositor into the "server" where IMO it belongs, etc. But I'm still not completely sold on the idea that this is worth a multi-decade project to completely replace this critical bit of the Linux GUI stack. I'm also not convinced that the X server can't be extended in other ways to fix other deficiencies. Of course, at the end of the day, I'm not doing the work, and the nature of open source means that if everyone wants to work on Wayland and no one wants to work on X11 or the xorg server, that's just the way it will be. But it feels like if we'd put even a fraction of the Wayland effort toward making X11 better, we'd be done by now, and people writing X11 applications would mostly not have had to do all that much work to migrate (and many wouldn't have to do any). Would it be as objectively "good" as Wayland supposedly is? No, probably not, but perhaps that doesn't matter. [0] One thing I can think of here is that some applications need to open more than one connection to the X server, and the X server would see them as separate clients. That could break an application that expects to be able to share and manipulate resources between the multiple connections. But perhaps that could be worked around by grouping permissions not solely by connection, but by process ID. There are probably still some edge cases here, but I think changing the apps here would be ok to do; at least, it seems like less work to accept this sort of breakage and try to fix the apps rather than develop an entirely new windowing system!

southerntofu 5y ago

Friendly reminder that QubesOS, mentioned in the article, is a great system if you have actual security needs. Their blog is full of treasures explaining what their research and development is about. https://www.qubes-os.org/news/ https://www.qubes-os.org/news/

xvilka 5y ago
There's also a project[1][2][3] to allow Windows-like containers inside, based on the ReactOS. [1] https://github.com/QubesOS/qubes-issues/issues/2809 https://github.com/QubesOS/qubes-issues/issues/2809 [2] https://reactos.org/forum/viewtopic.php?t=16480 https://reactos.org/forum/viewtopic.php?t=16480 [3] https://github.com/Jeeppler/QubesOS-notes/blob/master/ReactOS/Qubes-ReactOS.md https://github.com/Jeeppler/QubesOS-notes/blob/master/ReactO...

tyingq 5y ago

Interesting. WSLG seems to do this as well, bundling Xwayland, Pulse Audio, and Weston into a single separate VM that serves all the user-installed WSL distros on a box. https://devblogs.microsoft.com/commandline/wp-content/uploads/sites/33/2021/04/diagram-description-automatically-generated.png https://devblogs.microsoft.com/commandline/wp-content/upload...

ncmncm 5y ago

[Edit: I found the blog entry from March that mentions SpectrumOS.] Maybe you should know about the project SpectrumOS, which seems to have extremely similar goals to yours. https://spectrum-os.org/ https://spectrum-os.org/ I think instead of running persistent VMs for different roles, it spins up VMs very quickly to run individual apps in.

ncmncm 5y ago

It should be noted that Qubes is thus far the only actually secure X environment. The hardware X server runs in a secure X VM, which has exclusive access to input devices, the GPU, and the physical frame buffer memory. (Ordinary X gives all programs access to all traffic from all input devices, and the whole frame buffer.) Each client VM runs an X server that really only manages window contents. These windows are mmapped to memory the secure X VM exposes from its address space, It copies pixels from the shared regions to the real frame buffer. UI events that happen in each such window are forwarded to the client VM that owns it. Regular desktop "panel" widgets and other UI stuff is managed by the secure X VM. USB devices can be attached, via the secure X UI, to a chosen client VM, or taken back. Likewise, input devices such as microphones. The VM only gets PulseAudio streams forwarded from a microphone, and has no access to the sound chip. The only awkward thing is that client VMs have no access to any GPU acceleration. This is not a problem for normal use -- browsers all can use CPU-only graphics modes, and they are fast enough for the usual things, including youtube videos, albeit with increased power consumption.

ossusermivami 5y ago

Very good article, the linked code about vim is incredible tho (and not the good kind of it), I have actually spent a good part of my evening trying to understand how they end up with that implementation (I do have the same kind of spaghetti code and wonder the same about my code) : https://github.com/vim/vim/blob/9cd063e3195a4c250c8016fa340922ab21fda252/src/gui.c#L489 https://github.com/vim/vim/blob/9cd063e3195a4c250c8016fa3409...