6 ms·

The Linux Graphics Stack

284 points by Adrock 14y ago

antihero 14y ago
Absolutely fascinating. Some questions... Would this mean that window managers themselves would have to be re-implemented in order to support the Wayland protocol? Is anyone working to create a non-reference optimized implementation of libwayland? Have the Wayland developers said anything about supporting multiple monitors yet?
- exDM69 14y ago
  > Would this mean that window managers themselves would have to be re-implemented in order to support the Wayland protocol? Yes, an a lot more than that will have to be partially or fully rewritten. Thankfully, many if not most desktop apps use a widget toolkit like GTK+ or Qt for UI, so it's enough to make the widget toolkits work on Wayland.
- makomk 14y ago
  Yep, the Wayland developers' solution to window management is for anyone who wants a different window manager to replace Wayland with a different compositor+window manager speaking the same protocol. Their suggested way of supporting proprietary graphics drivers is also replacing the compositor+window manager with one specific to that hardware which speaks the Wayland protocol, sort of like in the early days of X when there was a seperate X server binary for every piece of hardware. So your window management choices under Wayland depend on which graphics drivers you're using.
  - 0x0 14y ago
    Wow, does that mean you could end up with a "KDE-only" nvidia driver and a "GNOME-only" ATI/AMD driver?
  - JoshTriplett 14y ago
    Not quite. Wayland itself doesn't require any graphics-driver-specific code; it uses the standard kernel interfaces and graphics stack (Mesa, DRI, libdrm, KMS, etc). That also works fine with radeon and nouveau, and it would work fine with the proprietary drivers as long as they support the standard graphics stack. It just won't work with proprietary drivers that implement their own entirely un-integrated graphics stack, which seems unsurprising to me. Your window manager of choice can just link into Wayland/Weston and ignore all the underlying graphics driver bits; they'll Just Work.
    - tedunangst 14y ago
      It's been years and years since I knew the details, but at one point the nvidia drivers replaced pretty everything from libX11 down, and I don't think you could hook in at a middle point. (ie, they don't support the "standard" graphics stack.) Would appreciate if anyone could comment on how things stand today.
      - JoshTriplett 14y ago
        Somewhat better, but still awful. They don't replace libX11, but they do replace libGL, and they bypass the entire kernel graphics stack in favor of their own.
    - wmf 14y ago
      You're describing how it should work, but AFAIK the proprietary drivers don't support KMS and have even talked about never supporting it. But rather than hacking Weston to support proprietary drivers, I would advocate using it as a carrot to beat AMD/Nvidia into submission.
  - alexlarsson 14y ago
    While its true that a new "WM" in wayland would be a diffent binary its not really the same as a different X server. All the stuff that X does wrt hardware and stuff is not really in the wayland compositor, but in the drivers, and they are the same with both WMs. And the WMs would link to the wayland server library, so a lot of code would be shared between them. I don't think properietary drivers need to have their own wayland server. However, they do need to add EGL support, so that wayland can use the driver outside of X, and a few specific GL extensions that wayland needs. See e.g: http://ppaalanen.blogspot.fr/2012/03/what-does-egl-do-in-wayland-stack.html http://ppaalanen.blogspot.fr/2012/03/what-does-egl-do-in-way...
- alexlarsson 14y ago
  libwayland being a reference implementation does not mean its not implemented. It just means that the specified thing is the protocol, not the library API. This is similar to libX11, its just a reference implementation, and other implementations like escher (a pure java X11 protocol implementation) exists. Most wayland servers will still use libwayland. There is nothing special about multiple monitors in wayland. The KMS/Mesa drivers allows setup and rendering to multiple monitors fine, then its up to the wayland server to decide what to draw on each monitor.
  - wmf 14y ago
    then its up to the wayland server to decide what to draw on each monitor This is the kind of "mechanism without any policy" thinking that I was hoping to escape from. What does Weston actually do in the presence of multiple monitors?
loudmax 14y ago
I can understand that Wayland does not want to pay for the complexity of network transparency, but I'd be sorry to see it go entirely. I'm one of the few people who do make use if X11's network transparency on a regular basis. Being able to open a web browser on a machine that is several ssh hops away is awfully convenient. I hope that Wayland has some provisions for reproducing or emulating X11 even if it isn't built into the core.
- kzahel 14y ago
  And while I've never actually needed to do this, it's comforting to know it's there.
- kleiba 14y ago
  Yes, I use that all the time too. There are certainly alternatives around for getting a remote window displayed locally but the simplicity of "ssh -X" is hard to beat.
- eru 14y ago
  I'm doing this at the very moment.
- simonh 14y ago
  That would be XWayland, essentialy running X on Wayland the way you'd run it on Windows or MacOS X now, which should go a long way towards easing the transition.
- krakensden 14y ago
  It's planned for the future, but it isn't done now.
- mbq 14y ago
  I used to as well, but I discovered that VNC completely beats X forwarding in terms of responsiveness, reliability, speed, clipboard management and OpenGL usability. Modern apps simply draw so much so complex things that the recipe is heavier than the effect. The philosophy is different, I admit, and it needs certain setup, but it is totally worth it.
- 14y ago
exDM69 14y ago
> While it’s very apparent that X is going to go away in favor of Wayland some time soon... Isn't it a bit early to be saying something like that? While I hope that Wayland will succeed, it has an equal chance of ending up as vaporware.
- chao- 14y ago
  At minimum we'll get at least one distro attempting to utilize Wayland eventually: Ubuntu. See the bottom of https://wiki.ubuntu.com/Wayland https://wiki.ubuntu.com/Wayland where it reads: Canonical is investing in making Compiz Wayland-enabled and running Unity on top. While it does not completely refute your concern of Wayland becoming vaporware, a commitment from Canonical is about as close as anything in the Linux Desktop world ever comes to being a hard milestone. Put another way: it's more reliable than a random user on the Arch forums* talking up grand plans to rebuild WMFS to work with Wayland. *A purely random example, and not a knock on Arch forums, which are an awesome/informative place.
  - hollerith 14y ago
    >a commitment from Canonical is about as close as anything in the Linux Desktop world ever comes to being a hard milestone Furthermore, most Xorg and desktop developers are pro-Wayland if the OP is to be believed.
    - Jach 14y ago
      I think if anyone really doubts this they should spend some time programming something directly on top of Xorg to convince themselves that when the ball of mud is gone there will be at least a bit more sanity.
      - 1346971608 14y ago
        I haven't used the raw stream protocol directly, but if you mean Xlib, I actually find it more usable than most toolkits. (Although a big part of that is being able to use select(), whereas most toolkits force you to do everything in callbacks while the toolkit drives the main loop.)
randallu 14y ago
I wish that unoccluded or maybe just the frontmost window could be unredirected too, even if it's not fullscreen. The decorations and shadow could still be drawn in the Compositor Overlay Window. I work on Linux on a program which is entirely drawn by OpenGL (I target ARM devices, but I test on the desktop before doing an ARM build). Application GL performance suffers immensely under a compositor, so on most machines I can't use GNOME3 (though I would like to). On machines where I get acceptable GL performance in my app, vsync is broken so there is constant tearing. I'm sure there are some caveats with unredirecting the frontmost window -- like when it becomes occluded it would have to be redirected and that requires a round trip to the client (which might hang, etc), but there are probably other situations like this which the window manager has to deal with already (resize timers?) so maybe there is a way to implement it...
- kelnos 14y ago
  Part of the problem is also that the compositor often can't know if the frontmost window (or any window, for that matter) is 100% opaque. If your window has partial transparency, or (for example) implements non-rectangularity by using an ARGB window and setting the alpha for some pixels to 0, then unredirecting it will break that. I suppose you could just restrict the optimization to windows with an RGB visual, though. But then you need to make sure there isn't something else modifying the opacity of the window. I've written a standalone compositor before (that is, a compositor that runs independently of the window manager, as a separate process), so you can't always assume that the compositor "knows best"; for example, the WM may want to make the window decorations semi-transparent. Or maybe the user has specified slight transparency for all their windows. Or something. Hmm... in the end, though, you could probably come up with a heuristic that works in most cases.
- timmaxw 14y ago
  I think that Mac OS X unredirects unoccluded windows; I've noticed windowed computer games suddenly getting much slower when I move another window in front of them.
  - randallu 14y ago
Avshalom 14y ago
So wait... Wayland does nothing, makes existing things harder and requires the Linux ecosystem to greenspun X in perpetuity?
- krakensden 14y ago
  X runs fine on top of Wayland, just like X runs fine on top of Quartz.
- fusiongyro 14y ago
  Not sure why you're being pessimistic. VNC demonstrates that graphical access can be networked at the application level rather than as a fundamental architecture decision we pay for constantly. Wayland is an attempt to cash in on the savings.
  - tedunangst 14y ago
    Except it doesn't look good, and copy/paste is even shittier.
deleted 14y ago
[deleted]
outworlder 14y ago
Now, this is the sort of article I want to see on Hacker News. Excellent read.
tedunangst 14y ago
instead of going over the network, they’ll go over a UNIX socket, and the kernel doesn’t have to copy data around. This isn't really true. Even if the kernel does cow pipes (on both ends), the writer is almost certainly going to trash the page and cause a fault. The data is just copied around without being bundled into TCP packets. There is also a SHM extension which does use shared memory, but that's separate and mostly only used for pixmaps and the like.

284 points by Adrock 14y ago

Absolutely fascinating. Some questions... Would this mean that window managers themselves would have to be re-implemented in order to support the Wayland protocol? Is anyone working to create a non-reference optimized implementation of libwayland? Have the Wayland developers said anything about supporting multiple monitors yet?

exDM69 14y ago
> Would this mean that window managers themselves would have to be re-implemented in order to support the Wayland protocol? Yes, an a lot more than that will have to be partially or fully rewritten. Thankfully, many if not most desktop apps use a widget toolkit like GTK+ or Qt for UI, so it's enough to make the widget toolkits work on Wayland.
makomk 14y ago
Yep, the Wayland developers' solution to window management is for anyone who wants a different window manager to replace Wayland with a different compositor+window manager speaking the same protocol. Their suggested way of supporting proprietary graphics drivers is also replacing the compositor+window manager with one specific to that hardware which speaks the Wayland protocol, sort of like in the early days of X when there was a seperate X server binary for every piece of hardware. So your window management choices under Wayland depend on which graphics drivers you're using.
- 0x0 14y ago
  Wow, does that mean you could end up with a "KDE-only" nvidia driver and a "GNOME-only" ATI/AMD driver?
- JoshTriplett 14y ago
  Not quite. Wayland itself doesn't require any graphics-driver-specific code; it uses the standard kernel interfaces and graphics stack (Mesa, DRI, libdrm, KMS, etc). That also works fine with radeon and nouveau, and it would work fine with the proprietary drivers as long as they support the standard graphics stack. It just won't work with proprietary drivers that implement their own entirely un-integrated graphics stack, which seems unsurprising to me. Your window manager of choice can just link into Wayland/Weston and ignore all the underlying graphics driver bits; they'll Just Work.
  - tedunangst 14y ago
    It's been years and years since I knew the details, but at one point the nvidia drivers replaced pretty everything from libX11 down, and I don't think you could hook in at a middle point. (ie, they don't support the "standard" graphics stack.) Would appreciate if anyone could comment on how things stand today.
    - JoshTriplett 14y ago
      Somewhat better, but still awful. They don't replace libX11, but they do replace libGL, and they bypass the entire kernel graphics stack in favor of their own.
  - wmf 14y ago
    You're describing how it should work, but AFAIK the proprietary drivers don't support KMS and have even talked about never supporting it. But rather than hacking Weston to support proprietary drivers, I would advocate using it as a carrot to beat AMD/Nvidia into submission.
- alexlarsson 14y ago
  While its true that a new "WM" in wayland would be a diffent binary its not really the same as a different X server. All the stuff that X does wrt hardware and stuff is not really in the wayland compositor, but in the drivers, and they are the same with both WMs. And the WMs would link to the wayland server library, so a lot of code would be shared between them. I don't think properietary drivers need to have their own wayland server. However, they do need to add EGL support, so that wayland can use the driver outside of X, and a few specific GL extensions that wayland needs. See e.g: http://ppaalanen.blogspot.fr/2012/03/what-does-egl-do-in-wayland-stack.html http://ppaalanen.blogspot.fr/2012/03/what-does-egl-do-in-way...
alexlarsson 14y ago
libwayland being a reference implementation does not mean its not implemented. It just means that the specified thing is the protocol, not the library API. This is similar to libX11, its just a reference implementation, and other implementations like escher (a pure java X11 protocol implementation) exists. Most wayland servers will still use libwayland. There is nothing special about multiple monitors in wayland. The KMS/Mesa drivers allows setup and rendering to multiple monitors fine, then its up to the wayland server to decide what to draw on each monitor.
- wmf 14y ago
  then its up to the wayland server to decide what to draw on each monitor This is the kind of "mechanism without any policy" thinking that I was hoping to escape from. What does Weston actually do in the presence of multiple monitors?

loudmax 14y ago

I can understand that Wayland does not want to pay for the complexity of network transparency, but I'd be sorry to see it go entirely. I'm one of the few people who do make use if X11's network transparency on a regular basis. Being able to open a web browser on a machine that is several ssh hops away is awfully convenient. I hope that Wayland has some provisions for reproducing or emulating X11 even if it isn't built into the core.

kzahel 14y ago
And while I've never actually needed to do this, it's comforting to know it's there.
kleiba 14y ago
Yes, I use that all the time too. There are certainly alternatives around for getting a remote window displayed locally but the simplicity of "ssh -X" is hard to beat.
eru 14y ago
I'm doing this at the very moment.
simonh 14y ago
That would be XWayland, essentialy running X on Wayland the way you'd run it on Windows or MacOS X now, which should go a long way towards easing the transition.
krakensden 14y ago
It's planned for the future, but it isn't done now.
mbq 14y ago
I used to as well, but I discovered that VNC completely beats X forwarding in terms of responsiveness, reliability, speed, clipboard management and OpenGL usability. Modern apps simply draw so much so complex things that the recipe is heavier than the effect. The philosophy is different, I admit, and it needs certain setup, but it is totally worth it.
14y ago

exDM69 14y ago

> While it’s very apparent that X is going to go away in favor of Wayland some time soon... Isn't it a bit early to be saying something like that? While I hope that Wayland will succeed, it has an equal chance of ending up as vaporware.

chao- 14y ago
At minimum we'll get at least one distro attempting to utilize Wayland eventually: Ubuntu. See the bottom of https://wiki.ubuntu.com/Wayland https://wiki.ubuntu.com/Wayland where it reads: Canonical is investing in making Compiz Wayland-enabled and running Unity on top. While it does not completely refute your concern of Wayland becoming vaporware, a commitment from Canonical is about as close as anything in the Linux Desktop world ever comes to being a hard milestone. Put another way: it's more reliable than a random user on the Arch forums* talking up grand plans to rebuild WMFS to work with Wayland. *A purely random example, and not a knock on Arch forums, which are an awesome/informative place.
- hollerith 14y ago
  >a commitment from Canonical is about as close as anything in the Linux Desktop world ever comes to being a hard milestone Furthermore, most Xorg and desktop developers are pro-Wayland if the OP is to be believed.
  - Jach 14y ago
    I think if anyone really doubts this they should spend some time programming something directly on top of Xorg to convince themselves that when the ball of mud is gone there will be at least a bit more sanity.
    - 1346971608 14y ago
      I haven't used the raw stream protocol directly, but if you mean Xlib, I actually find it more usable than most toolkits. (Although a big part of that is being able to use select(), whereas most toolkits force you to do everything in callbacks while the toolkit drives the main loop.)

randallu 14y ago

I wish that unoccluded or maybe just the frontmost window could be unredirected too, even if it's not fullscreen. The decorations and shadow could still be drawn in the Compositor Overlay Window. I work on Linux on a program which is entirely drawn by OpenGL (I target ARM devices, but I test on the desktop before doing an ARM build). Application GL performance suffers immensely under a compositor, so on most machines I can't use GNOME3 (though I would like to). On machines where I get acceptable GL performance in my app, vsync is broken so there is constant tearing. I'm sure there are some caveats with unredirecting the frontmost window -- like when it becomes occluded it would have to be redirected and that requires a round trip to the client (which might hang, etc), but there are probably other situations like this which the window manager has to deal with already (resize timers?) so maybe there is a way to implement it...

kelnos 14y ago
Part of the problem is also that the compositor often can't know if the frontmost window (or any window, for that matter) is 100% opaque. If your window has partial transparency, or (for example) implements non-rectangularity by using an ARGB window and setting the alpha for some pixels to 0, then unredirecting it will break that. I suppose you could just restrict the optimization to windows with an RGB visual, though. But then you need to make sure there isn't something else modifying the opacity of the window. I've written a standalone compositor before (that is, a compositor that runs independently of the window manager, as a separate process), so you can't always assume that the compositor "knows best"; for example, the WM may want to make the window decorations semi-transparent. Or maybe the user has specified slight transparency for all their windows. Or something. Hmm... in the end, though, you could probably come up with a heuristic that works in most cases.
timmaxw 14y ago
I think that Mac OS X unredirects unoccluded windows; I've noticed windowed computer games suddenly getting much slower when I move another window in front of them.
- randallu 14y ago

Avshalom 14y ago

So wait... Wayland does nothing, makes existing things harder and requires the Linux ecosystem to greenspun X in perpetuity?

krakensden 14y ago
X runs fine on top of Wayland, just like X runs fine on top of Quartz.
fusiongyro 14y ago
Not sure why you're being pessimistic. VNC demonstrates that graphical access can be networked at the application level rather than as a fundamental architecture decision we pay for constantly. Wayland is an attempt to cash in on the savings.
- tedunangst 14y ago
  Except it doesn't look good, and copy/paste is even shittier.

deleted 14y ago

[deleted]

outworlder 14y ago

Now, this is the sort of article I want to see on Hacker News. Excellent read.

tedunangst 14y ago

instead of going over the network, they’ll go over a UNIX socket, and the kernel doesn’t have to copy data around. This isn't really true. Even if the kernel does cow pipes (on both ends), the writer is almost certainly going to trash the page and cause a fault. The data is just copied around without being bundled into TCP packets. There is also a SHM extension which does use shared memory, but that's separate and mostly only used for pixmaps and the like.