15 ms·
Understanding the working of X11 from the ground up (2017)
- shmerl 7y ago> I've made a personal decision to leave the Linux, X11, and Wayland worlds mostly behind. I understand X11 has a lot of unfixable problems, but what's wrong with Wayland world?
- jchw 7y agoI'm running Sway WM right now, a Wayland compositor that implements an i3-compatible window manager, and in fact working on the code (hopefully merging my first contribution soon.) I'd say the biggest issue is really maturity. I actually find Wayland to be a surprisingly nice piece of software engineering. I don't know if I like the architecture really, but it comes together fairly nicely in the end. For example, there's unanswered security questions. It would be good if applications couldn't just go and manipulate and spy on other apps like they can with X11. Wayland indeed solves this issue, but it leaves other problems. How do I capture the desktop? wl_roots offers some extensions that help alleviate the problem, but they don't solve the full problem; for example, you can capture the screen, for which programs and an OBS plugin exists, but there's no upstream Wayland extension that can make WebRTC desktop capture work. I think what we really need is the ability to integrate policy and consent requests, but I'm not sure whether there's progress on that front, or how long it will take. Once the problem is solved, upstreamed to WebRTC, and that upstream version trickles down to Electron apps, Firefox and Chrome stable versions... well, things will be in a better place. Speaking of web browsers, today Firefox still has some issues to go before it is fully working in Wayland natively. You can do it today, but it is not perfect; there are sometimes places in the code where it assumes it is running on X11, and things simply break. The screenshot tool is just flat out not present when running under Wayland due to the above issue. Applications are slow on the uptake too. XWayland covers most of them, but it is not perfect. Like anything that captures the screen, of course, is not going to work under XWayland, but also, XWayland apps may behave a little off. In Sway WM, one bug is that when the mouse travels from an X window to a Wayland window, sometimes it behaves as though the mouse is still hovering over the X window at the last position it touched, which leads to tooltips showing up where they shouldn't. There's still a decent amount of GTK2 apps, and even Qt apps aren't all Wayland yet; Krita plans to start working on support I believe next year. In my case, here's the itch I'm currently scratching: Sway WM also does not have support for drawing tablets. I found this out, by, well, loading up a drawing program and noticing the lack of pressure sensitivity. So, that's what I'm currently working on. Needless to say, this is a non-starter for anyone that relies on a graphics tablet, although if you are just using GNOME 3 it is unlikely to be an issue since I'm sure GNOME 3 supports graphics tablets just fine. NVIDIA drivers are still a pain in the neck. I gave up and bought an AMD graphics card, and frankly I can't recommend it enough. Basically everything works better on Linux, but certainly Wayland works better. It's worth giving up some raw power. Still, for many people the proprietary NVIDIA drivers are going to need to work, and today support varies from lackluster to poor depending on what compositor you use.
- shmerl 7y agoI think capturing desktop and screenshots should be addressed by standard Wayland extensions with integration with something like Pipewire as far as I know. I didn't follow the progress of that though. Some details here: https://wiki.archlinux.org/index.php/PipeWire https://wiki.archlinux.org/index.php/PipeWire Some features though are surely missing. Such as support for adaptive sync. No one really is pushing that forward. See: https://github.com/swaywm/wlroots/issues/1406 https://github.com/swaywm/wlroots/issues/1406 As for Nvidia - don't waste your time on their proprietary drivers on Linux. They don't to play by the rules (refuse to upstream), so no need to spend time supporting their blobs.
- jchw 7y agoLike I said... I gave up and bought an AMD card. Not everyone is going to do this. It's a step down from my NVIDIA card. I think there is still hope. Obviously today's reality is still bleak with signed blobs and plenty of question marks, but I think open-gpu-doc is a sign that NVIDIA does actually want to square away this problem, maybe they're just not sure how. I do believe NVIDIA is mostly responsible for Wayland's poor adoption, and I firmly believe that this also lead to Wayland not getting as much developer attention as it could've gotten. The fact that one company wields this much power is a serious problem. I kind of understand the hostility Linux kernel developers seem to have towards proprietary drivers under Linux now.
- shmerl 7y agoKernel developers are right to push back against blobs. Nvidia is the bad player here. AMD is doing fine though. I have RX 5700 XT, and besides for some rough edges due to it being very new and drivers still fixing bugs, it's a very good card. I expect Nvidia usage on Linux to plummet, due to their refusal to provide open drivers. It's already happening, just will take some time for it to drop to the point where they have little influence. See: https://www.gamingonlinux.com/index.php?module=statistics&view=trends https://www.gamingonlinux.com/index.php?module=statistics&vi... So in the long term it's not a problem.
- 7y ago
- yjftsjthsd-h 7y agoFrom my perspective, Wayland "fixes" things that could allow bad things but were also useful. For instance, screenshots/recording took a long time to be implemented, and is still constrained, because of the security risk of arbitrary programs recording the screen. Because that is a risk... but users still want it. As another example, I use xdotool to inject input to programs in X; in Wayland, that would need to be explicitly supported per-compositer.
- shmerl 7y agoInstead of xdotool, you can use this: https://github.com/ReimuNotMoe/ydotool https://github.com/ReimuNotMoe/ydotool
- dyanaraps 7y agoNot exactly a 1:1 replacement though: > This program requires access to /dev/uinput. This usually requires root permissions. Source: https://github.com/ReimuNotMoe/ydotool#runtime https://github.com/ReimuNotMoe/ydotool#runtime Edit: I'll also add that only 5~ commands have been implemented so far in ydotool.
- zzzcpan 7y agoWell, removing or not implementing such features because of security is always a big fat sign of something sneaky or even worse, outright incompetent going on. As such security risks of features can always be addressed through some sort of authorization mechanism.
- roenxi 7y agoThe basic problem with X11 is it wraps a whole bunch of things into one software package. So in theory you have to know about fonts, OpenGL, graphics drivers & memory management, the window model and networking simultaneously to configure and run the thing. Wayland is a protocol for a well designed future where all that functionality is broken up into separate pieces of software. Wayland handles the window model and most of the other parts already live in their own software projects. The problem is that Wayland makes some changes to the basic window model on the way through that are a big deal and so far there is no great software written that uses it. Adoption is very slow because the best case scenario is there a Wayland compositor that has feature parity with an X11 window manage and a bunch of weird problems cropping up because screen recording is a to-be-solved problem. My position from the armchair is that Wayland really needed a complimentary official protocol for how to request another application's graphical output. Not coordinating that means that anything that supports Wayland is going to be practically inferior to an X11 solution. It'll get fixed sooner or later I expect, it isn't a technically hard problem.
- shmerl 7y agoI think this is intended for screen capturing with Wayland: * https://wiki.archlinux.org/index.php/PipeWire https://wiki.archlinux.org/index.php/PipeWire * https://pipewire.org https://pipewire.org
- roenxi 7y agoQuoth the wiki 'PipeWire is a rather new multimedia framework by GNOME'. There will be people who don't want to use GNOME frameworks and don't want to use the GNOME window manager. Wayland is enforcing a policy ('don't access other devices graphics output') where it should be providing capability ('an applications graphics output is private by default but that can be configured'). However it looks like the design of Wayland purposefully omitted that sort of configuration from the protocol. So instead of having a security model that is sufficiently complicated to allow screen recording, the security model is 'No'. To get a framework independent solution all the compositors would need to implement an identical API for accessing graphics output. That needs to be coordinated in the same fashion as the Wayland spec. As the spec stands, it is just asking for every framework, compositor or application to go implement its own different solution. The Wayland designers may even have tried to avoid explicitly creating a security model, but in practice it comes with an implicit security model that is inflexible, unconfigurable and difficult to coordinate around.
- sly010 7y agoIn one word, nvidia.
- shmerl 7y agoEasy to fix - don't use it. Linux users today stay away from it for good reasons.
- mnd999 7y agoThat’s fine if you’re already a Linux user and purchased hardware with Linux in mind.
- shmerl 7y agoFor Windows users coming to Linux with Nvidia, this usually simply involves one extra cycle of GPU upgrades. By the next one, it's AMD for them. So in the long run - still not an issue.
- iikoolpp 7y agoI specifically bought only NVIDIA GPUs because they were faster under Linux when I got them.
- shmerl 7y agoIn the past that was a common reason. Today it's not relevant anymore. And downsides of the blob make buying Nvidia a non starter. You can clearly observe this trend today among Linux users.
- krychu 7y agoDoes he mean he's no longer working/interested in writing about these topics, and moved to other things? Or does he mean he abandoned these techs and moved to something other alternatives? If the latter, I wonder what these other alternatives are?
- Jasper_ 7y agoAuthor here. I had been working on Linux professionally for about 6 years at that point, and got burned out on it. I have a lot of thoughts on the success of Linux as a general-use platform but mostly keep them to myself these days. I still have a deep love for graphics, and have since moved onto working in the games industry, where I am much happier. I also still have a passion for explaining graphics topics. Current efforts: https://noclip.website/ https://noclip.website/ https://blog.mecheye.net/2018/03/deconstructing-the-water-effect-in-super-mario-sunshine/ https://blog.mecheye.net/2018/03/deconstructing-the-water-ef... https://www.youtube.com/watch?v=8rCRsOLiO7k https://www.youtube.com/watch?v=8rCRsOLiO7k
- shmerl 7y agoWhat's your view on the recent progress of Linux gaming?
- pcr910303 7y agoThis is a bit of topic but: Every time I see the X11 protocol, I realize that the UNIX philosophy (everything is a file) is dead. It’s a pity that instead of improving the speed of the file system, everyone started to ignore it, hence making it’s speed even less important, and on and on... And now nobody uses the file system for these purposes.
- progman 7y ago> Every time I see the X11 protocol, I realize that the UNIX philosophy (everything is a file) is dead. As "dead" as Android (which is based on Linux, a Unix clone) and OSX (a BSD Unix) and iOS (child of OSX)? Why does Microsoft vibrantly support Linux in Windows 10 (an abomination to Ex-CEO Steve Ballmer) if Unix is "dead"? I think the very opposite is true. Some developers realized that the KISS principle (including the Unix philosophy where everything is a stream of bytes) is not a bad idea after all. https://blog.pragmaticengineer.com/software-architecture-is-overrated/ https://blog.pragmaticengineer.com/software-architecture-is-...
- fao_ 7y agoSupport of Linux != support of UNIX and the UNIX Philosophy. Lennart Poettering, who is the mind behind Systemd, Pulseaudio, and a lot of major new technologies explicitly disavows the UNIX philosophy.
- legulere 7y agoThe problem with file systems is that it is extremely difficult to build reliable systems on top of them. It’s shared mutable state with barely any error handling possibilities.
- pcr910303 7y ago> It’s shared mutable state with barely any error handling possibilities. And that makes me think about scoped file systems... (gotten the idea from global & scoped variables) Since the biggest advantage of ‘everything is a file’ is that it allows a consistent API for manipulating data, why didn’t this happen?
- sprash 7y agoMoney Quote: > So, a lot of the design of X11 is giving clients and apps a large set of tools to design and implement their app, without any real opinionated design about how an app should be made. An X client can use this tool or that tool to battle this tradeoff or that tradeoffs, and neither is fundamentally a wrong choice. That is something that can not be said about Wayland. Besides the "every frame is perfect" mantra Wayland has no design philosophy at all. It is also a weak argument to introduce a new protocol. With modern drivers it is rare to experience tearing on X11 and on some occasions it even makes sense to disable vsync (e.g. competitive gaming). The introduction of Wayland was a mistake because X11 has a huge ecosystem with lot of little programs and a large diverse users base. Almost every Unix OS supported X11 at some point. There is a reason why Intel still stuffs the i368 instruction set in their processors, Microsoft Windows still can execute programs from 1995 and IBM is still able to make money with vastly overpriced mainframes. Successful systems have the tendency to stay and accumulate cruft as a result. The abandonment of X11 will be a heavy blow to the FOSS community (and maybe that is exactly the intend).
- jdub 7y agoX11 and its ecosystem of applications isn't abandoned by Wayland, because you can run an X11 server on Wayland. By all means, debate the merits of Wayland itself, but don't lean on inaccuracies and conspiracy theories.
- sprash 7y agoMy window manager won't work, my hotkey daemon won't work, my screenshot tool won't work, any X11 program that draws to the root window won't work. Xwayland is obviously not a working solution to fix the ecosystem problem. Who is inaccurate now? > debate the merits Wayland has no merits. That is kind of the problem. It is just a reimplementation of stuff that worked.
- progman 7y agoThe wayland project exposes what marvelous piece of software X11 really is since Wayland is still not able to compete, otherwise the whole Linux community would use it already. I respect the intention and efforts of the Wayland community to make things better. However, I wished the community would have focused on making X11 better (easier to maintain with modularity etc.) instead of following the mainstream which is focused on eye candy and other sensual features to attract customers. The unfortunate developments of KDE3/4, Gnome2/3 and Win7/8 exposed that boring usable functionality is much more important than exciting sensual functionality.
- krychu 7y agoI haven't used linux as my primary OS for many years now. If not X11 or Wayland, what other alternative? Is there some other relevant development going on?
- sverige 7y agoOpenBSD is still actively developing xenocara, which is their fork of X11.
- 0xdeadb00f 7y agoThat still counts as "X11" IMO.
- tarruda 7y agoThere's Arcan: https://github.com/letoram/arcan https://github.com/letoram/arcan . In the words of the developer, it is a "multimedia engine" (a combination of display server, game engine and multimedia framework). It is an interesting project and the YouTube presentation is pretty convincing: https://m.youtube.com/watch?v=07nqZIFRDJg https://m.youtube.com/watch?v=07nqZIFRDJg A great example of what a motivated and disciplined developer can achieve.
- kebman 7y agoThis was a really nice demo! It can even be used to understand concepts used in the HTML5 canvas, such as aliasing. Hell, it could possibly used as en entry to learning how to program ray tracers. At the bare bones, construction of complex bodies in POV-Ray was used by Constructive Solid Geometry hearkening back to the basic union, xor, intersection, etc, operations done in the demo, at least on a conceptual level.
- Jasper_ 7y agoAuthor here. I stopped working on this a few years ago, so now's as good a time as any for an overly-long, rambly, unedited retrospective. When I started this, I was working on GNOME's window manager full-time, and wanted to learn intricately how X11's drawing model worked, so over the course of a few weeks in a hotel room, I recreated large parts of X11's drawing model in a web browser, fixing artifacts as I went along, until I feel I had a really good grasp of it. My initial test scene was a traditional desktop-like approach with a taskbar and xeyes, both of which are still in the codebase today, but untested [0]. I didn't know what I wanted to do with it, until I settled upon using snippets of it to build a long-form article. I learned a lot about the difficulty of writing, of pedagogy, of that blurred line between being technically correct 100% of the time vs. telling a few small lies here and there to keep the flow consistent and help people see the broader picture. At my day job, I had mostly moved onto Wayland, where some of the bits I picked up here really helped me design better protocols and systems. My goal with the series was to try to be as neutral as possible, and my original design was to have a giant caution sign around "Author Opinion Zones" where I would talk about how certain design features haven't held up well in practice. But quickly, people on Hacker News or Phoronix or Reddit seemed to skim the article, pick up a piece here or there, and go on straight to bashing Wayland, gleefully unaware that I was one of the people making it. So, the end result was that I basically stopped working on Xplain basically after the second article. The COMPOSITE article was one I made after a colleague was having trouble understanding COMPOSITE, and I figured it was easier to write with my framework than explain in a chatroom, and maybe some others would appreciate it. I have a deep passion for sharing my knowledge, and Xplain was the format I first really used to do it widely, so I tried to keep it exciting for me by changing it from "Xplain" to "Explanations", and opening up the topics from X11 to just about anything, but at some point I was just unhappy working on it. The last thing I was working on was a continuation of my Basic 2D rasterization article, where I had a fun code editor you could use to make your own graphics [1] [2], but as fun as the technology was, I couldn't find a satisfying flow to the article, so I stopped it. Parts of it were later recycled for an article on the histories of 2D and 3D graphics. [3] Around mid-2016, I had stopped working on Linux and open-source graphics entirely, and by 2017 I had exited the open-source industry completely and jumped ship to professional game development. I still have a deep love for graphics and a passion to explain things. I just released a new side project a few days ago for it, even. Here's some stuff I'm working on these days. It's much cooler than X11/Wayland flamewars, in my opinion. https://noclip.website/ https://noclip.website/ https://blog.mecheye.net/2018/03/deconstructing-the-water-effect-in-super-mario-sunshine/ https://blog.mecheye.net/2018/03/deconstructing-the-water-ef... https://www.youtube.com/watch?v=8rCRsOLiO7k https://www.youtube.com/watch?v=8rCRsOLiO7k -- [0] https://github.com/magcius/xplain/blob/gh-pages/src/clients/xeyes.js https://github.com/magcius/xplain/blob/gh-pages/src/clients/... [1] https://u.teknik.io/XdSbC.webm https://u.teknik.io/XdSbC.webm [2] Sort of up here at https://magcius.github.io/xplain/article/rast2.html https://magcius.github.io/xplain/article/rast2.html [3] https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harder-than-3d-graphics/ https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harde...
- hota_mazi 7y agoWhat's fascinating to me is that X11 and Windows started at complete opposite ends of the spectrum. Then X11 never managed to be a really effective window manager on a local workstation while Windows became really, really good at remote displays.
- josteink 7y agoThat can be explained by economics alone. There were no financial driver to make X11 a more optimal local solution. There were huge financial incentives for MS to make their terminal-server offering more appealing. And that’s all you need to explain the outcome.
- asveikau 7y agoI feel like this is short changing X. I have seen it as a good local solution over 20 years or so. Did you ever see those 90s commercial Unix workstations? Like from SGI? How about a Nokia N900? X on a smartphone. It ran pretty well. Today's smartphone hardware would run circles around that thing so I can only assume if somebody ported X to those it would also do fine. I feel like people see things they don't like or imperfections and bash the whole thing as useless. X had a lot of mileage. I use it frequently today and don't see a lot of problems.
- ttkciar 7y agoI am fascinated by the economy of packing three inaccuracies into that one sentence. (1) X11 is not a window manager, (2) There have been several really effective window managers for X11, (3) MS-Windows has never been good at remote display.
- raphlinus 7y agoLooking over this stuff, I get an uneasy feeling. X was a brilliant solution to problems 35 years ago, where you barely had enough memory for a full frame buffer and the graphics model was primitive (no antialiasing or transparency). From what I've seen, Wayland is a good solution for the model maybe 15 years ago, where you're willing to accept the frame-and-a-half latency hit from having a compositor in the path. Basically, my question is whether anybody is working on a solution for the hardware we have today, where there are hardware overlays, the GPU throughput is orders of magnitude higher than what's needed to composite a desktop, but we still care about memory bandwidth, largely because all of resolution, bit depth, and frame rate is still going up. And you really want to be able to schedule every pixel that gets presented with a high degree of timing accuracy. I've become familiar with Windows, but feel like it's a bunch of hacks. They will promote a window to an overlay, but it's all a heuristic, and I'm pretty sure latency will stutter while it does the transition in and out. And the new flip-model stuff actually regresses the ability to do smooth window resizing. I'm not even sure macOS is working on lowering latency. I haven't seen any mention from them of hardware overlays. Most of the attention these days seems to be in the VR/AR space, where latency is obviously important, and there's more of a green field with respect to legacy compatibility, but I'd love to see this problem space actually solved on the desktop. Doing this right is not easy - it probably requires some form of soft real time discipline across the graphics stack (including GPU scheduling) so you can race the beam.
- Jasper_ 7y agoI have a lot of replies to this, but the back-and-forth format that HN has isn't the best for collaborative discussion. Imagine us sitting in a room with a whiteboard, rather than bickering at each other at loud volume. If you're ever in south bay area and want to meet up at a coffee shop, let's :) > Basically, my question is whether anybody is working on a solution for the hardware we have today, where there are hardware overlays, the GPU throughput is orders of magnitude higher than what's needed to composite a desktop Hardware overlays are... not that popular on desktop devices? You just stomp on the gas pedal with bandwidth and power consumption on consumer desktop gaming GPUs. Besides a couple YUV planes and a cursor overlay, you don't see that much in the way of CRTC capabilities on desktop. All of that's happening in the mobile space, because of power consumption reasons, where you basically have an SNES PPU in your CRTC with multi-plane compositing. I remain unconvinced that compositors need to add a frame and a half of latency. 16ms is a lot of time to render a frame, and composition isn't too difficult of a task. It just comes down to buffer handoff, which can be smoothed out over time. Similarly, the new flip-model isn't the thing that makes smooth resizing difficult, buffer allocation and resizing has always been a pain and racy on Windows/X11. This is one of the things that Wayland fixes -- it was actually the thing that caused Kristian to start Wayland forever ago. Presentation and timing is something I think should be possible, but I don't think it makes sense for every app to have to handle it. Multi-monitor scenarios are what make global timing difficult, and having per-monitor surfaces is just a pain to deal with from an API standpoint; if we did the work to have per-monitor timing information and split surface rendering, nobody wants to go through the work to set up two devices and present different ones at the right time. You might protest that all monitor timing is roughly the same these days, but someone hooking up their laptop to a projector, you get radically different timing. In many respects, I think graphics is in a local minima right now: it works well enough and we understand it well enough that nobody's really interested in pushing us to the next thing. VR/AR research might, but that's been relatively unimportant as a target platform, commercially.