5 ms·

This is a great positive example of the (often negatively presented) double-edged sword that is extreme backwards compatibility. X11 is an incredibly backwards

by aroman 13y ago

This is a great positive example of the (often negatively presented) double-edged sword that is extreme backwards compatibility. X11 is an incredibly backwards compatible system, but this comes at the cost of positively absurd code complexity. I remember reading some release notes from a new version a few years ago that said 100,000+ lines of deprecated code were removed, and that was barely a drop in the bucket. And, of course, this is why projects like Wayland and Mir (Canonical's new display server) have been born — it's simply too cumbersome (and in some cases, impossible) to innovate on a behemoth like X11. But people forget why X11 has such a giant codebase. It's not just bloat — a lot of it exists for a very good reason, and this article is an excellent illustration of what that is.

derefr 13y ago
I will never understand why people insist on making new clients and servers that are backward-compatible with terribly old, deprecated protocols, instead of just preserving the old clients and servers that spoke those protocols natively within some sort of sandboxed emulation system. Take Trident (IE's rendering engine), for example. Why can only one version of IE be installed at a time? Why does IE11 come with IE7/8/9/10 "compatibility modes" (which, much as they might claim, aren't faithful recreations of IE7/8/9/10 behavior)? Wouldn't it make more sense to just ship an IE "browser chrome" that could load up the actual rendering engine DLLs from IE6-through-11 (similar to how Google made IE connect to Chrome Frame), with increasing levels of virtualization and sandboxing as you go further back to cope with the difference in environments? Apple and Microsoft have both got this right a few times: Apple had Rosetta for PowerPC emulation; Windows had WOW16 and WOW64. But besides these isolated instances, it really isn't a common idea.
- aroman 13y ago
  EDIT: I didn't see your edit which gave Apple's Rosetta and X11 as examples. Those are good counter-examples to my point. But then again, those strategies were not done by X11 itself, rather by compatibility layers as you suggested. And in fact this is precisely what a lot of companies are doing, for example the XMir project (http://mjg59.dreamwidth.org/26254.html http://mjg59.dreamwidth.org/26254.html). ____ I agree that that's a great strategy in general, but I don't think it translates well to display servers. You can't just "put it in a sandbox". You need driver support, and you'd need a sandboxed environment sophisticated enough to deal with all the idiosyncrasies of the X protocol. Even putting running an old version of X11/UNIX in a virtual machine is of limited benefit, because X11's core functionality is often tightly coupled with drivers and other things which would need a lot of special wrappers and hacks to work properly. I'm not saying it wouldn't work (and I am not an X11 developer), but I do think there are extremely compelling and legitimate reasons for people to not follow your suggestion (at least in this case).
  - recuter 13y ago
    Why not make a clean break? None of this sandbox stuff. X11 2: Electric Bugaloo - The list of major graphical linux programs in active use is very finite and most of it doesn't talk to X directly but relies on stuff like Gnome or QT which are libraries that have frequent churn anyway. No? I bet if you could get the Top 30 projects to target your clean shiny X replacement in one fell swoop you'll end up most of the way there in terms of adoption within a few years. Linux is command line centric and frequently updated after all with users who expect things to break anyway. "Old" X odds and ends that won't be updated can be thought of much like Java Applets are thought of now a days. Oh you need that abandoned Chemistry application from 2006 that depends on X11? Spin up a VM. We can afford to be more pragmatic nowadays. Nobody expects to run 5 year old apps on their smartphones, but desktop linux can't afford a clean break once every 2 decades?
    - derefr 13y ago
      > Oh you need that abandoned Chemistry application from 2006 that depends on X11? Spin up a VM. We can afford to be more pragmatic nowadays. That was exactly what I was suggesting with "sandbox stuff"--except that the OS can provide a stub library that recognizes what that "abandoned Chemistry application from 2006" is asking for, and spin up the VM itself, in the background, making it look as if it was just a compatibility layer. See, for example, Windows 7/8's "Windows XP Mode" -- or, as I mentioned before, Rosetta.
      - Someone 13y ago
        That's not as easy as it sounds. In a desktop GUI, customers expect seamless integration between such sandboxes. Making the clipboard and drag and drop work two-way between such VMs can be a lot of work. Even for simple text, that may involve encoding conversions. That, in turn, means that the host must be able to infer what encoding the guest expects. Styled text and graphics are way harder. The VMs also must react to changes in the parent OS such as a change in the keyboard layout, two apps running in different VMs may both want to have some control over hardware, etc. The approach works well in Windows because Microsoft spends lots of time on it and because the compatibility changes aren't that great. Of course, it also works well in systems where there is no need for the VMs to interact (other than via the network). VM (http://en.wikipedia.org/wiki/VM_(operating_system) http://en.wikipedia.org/wiki/VM_(operating_system)) is a nice example there.
- noselasd 13y ago
  People get upset when stuff don't work. If you want customers, you make stuff work. You don't break stuff, or change stuff so they have to call you and have you tell them how to make it work.
- ori_b 13y ago
  And now, you're supporting the old system and the new system. Maybe it's worth it. Maybe it's not. Just keep in mind that if you're keeping an emulated version alive, someone has to mind the emulator, and that's not free.
- pcwalton 13y ago
  Sandboxing doesn't remove the need to provide ongoing security maintenance for a Web browser engine. For one, sandboxes can be attacked through whatever IPC mechanism you provide (e.g. Pwnium). But more importantly, the Web browser engine enforces security mechanisms that OS-level sandboxing does not address (e.g. the same-origin policy and history sniffing countermeasures).
- FollowSteph3 13y ago
  How do you deal with security updates? You'd have to apply them to differen code bases all the time. Pas bad as it sounds it's easier to work with one for base
jmathai 13y ago
Joel Spolsky, Things You Should Never Do, Part I http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html Back to that two page function. Yes, I know, it's just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I'll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.
- Peaker 13y ago
  The right way to resolve such bugs is to have abstraction layers that hide these awful hairiness. If hair regarding low-memory conditions, Internet Explorer and other things all appears in the same function, some abstractions are definitely missing. Also, if the reason for these things isn't clear, that's what comments are for. Rewriting code, from my experience, has had only great results.
  - Locke1689 13y ago
    Rewriting can be great as long as you take all the things Joel notes into account and decide it is still the right decision for your project.
  - zanny 13y ago
    This is so absolutely true. What nobody recognizes is that most of these really fundamental systems we run on - unix system calls, our window managers, our shells, etc - were designed and implemented in an era where the supercomputers couldn't dent your Nexus 5 in terms of most computing performance metrics. They didn't have the luxury of having an API that might be a few thousand KB big loaded into memory to act as a single redirection to your underlying implementation. They worked in the confines of bytes of memory, not gigabytes. We can afford to be generic, to make extensible runtime programmable interfaces and runtime evaluation type dynamism, because we have the performance necessary. But our core APIs are still written like its 1980.
makomk 13y ago
Amusingly enough, Wayland is already developing backwards-compatibility cruft even though it's not shipping anywhere yet. The core Wayland support for window creation and management cannot support minimizing windows, so it's being essentially obsoleted by an extension called xdg_shell, but xdg_shell isn't mandatory so applications and Wayland compositors will wind up having to support both.
- asabjorn 13y ago
  I thought things like xdg_shell grow up as either extension or in Weston, and then it gets merged into the stable Wayland protocol after it is battle tested and ready. Is this not the case for xdg_shell?
- toyg 13y ago
  >The core Wayland support for window creation and management cannot support minimising windows One of the problems with Big Rewrite projects is that, inevitably, early releases will never get feature-parity with the latest version of whatever it is that's being rebuilt. Managing expectations then becomes critical to avoid this sort of legacy-support nightmare.
- alexlarsson 13y ago
  This is funny. You know that window minimization is not in the X11 spec either, right? Its something that X window managers handle internally (just like on Wayland where this is an internal compositor thing). Of course, desktop apps wanted to interact with minimization (read current state, etc), so the WM authors joined forces to create a spec that allowed this. See how it specs minimization here: http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html#id2731936 http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html#id... This is part of the "wm spec", and the thing corresponding to it in Wayland is called "xdg_spec", Both are produced by "xdg" (X Desktop Group) and are optional things that are used by "traditional desktop environments". It is a good thing that wayland does not enforce the existance of window minimization, because it is not always something that makes sense in all use cases of wayland. For instance in a phone like Jolla.
rthomas6 13y ago
This is probably a stupid question but why can't someone make an X11 version that speeds up and optimizes everything but breaks backwards compatibility?
- toyg 13y ago
  What are you going to run on it? 99% of userland programs expect compatibility with some historical version of X11. Are you going to rewrite GNOME and KDE and everything else?
  - ahomescu1 13y ago
    Many userland applications don't use X libraries directly, but use a toolkit like Qt and GTK. Both projects are working on Wayland support [1, 2], bypassing X completely. 1 - http://wayland.freedesktop.org/qt5.html http://wayland.freedesktop.org/qt5.html 2 - http://wayland.freedesktop.org/gtk.html http://wayland.freedesktop.org/gtk.html
    - krakensden 13y ago
      It's worth noting that GTK+'s abstraction over X11 is fairly leaky in places- hence the always shaky state of its ports to Win & Mac. Qt has less trouble in this regard.
- dsr_ 13y ago
  Because something you depend on every day needs one of those backward compatible features.
- stormbrew 13y ago
  There's really no point to doing that, since X is a protocol, and a pretty flexible one at that. Your new shiny can have an X server running in it to act as a go-between and talk something more modern with the new stuff instead. The OP would have been able to do the same stuff with Exceed on Windows or the X server that comes with OSX, after all. X11's survival on linux seems to be largely a matter of momentum and the fact that it's the least common denominator in a fragmented landscape. But now Ubuntu has approached the level of ubiquitousness that's necessary to push for a real change (thus mir) and the rest of the linux world is rallying behind wayland.

by aroman 13y ago

derefr 13y ago

I will never understand why people insist on making new clients and servers that are backward-compatible with terribly old, deprecated protocols, instead of just preserving the old clients and servers that spoke those protocols natively within some sort of sandboxed emulation system. Take Trident (IE's rendering engine), for example. Why can only one version of IE be installed at a time? Why does IE11 come with IE7/8/9/10 "compatibility modes" (which, much as they might claim, aren't faithful recreations of IE7/8/9/10 behavior)? Wouldn't it make more sense to just ship an IE "browser chrome" that could load up the actual rendering engine DLLs from IE6-through-11 (similar to how Google made IE connect to Chrome Frame), with increasing levels of virtualization and sandboxing as you go further back to cope with the difference in environments? Apple and Microsoft have both got this right a few times: Apple had Rosetta for PowerPC emulation; Windows had WOW16 and WOW64. But besides these isolated instances, it really isn't a common idea.

aroman 13y ago
EDIT: I didn't see your edit which gave Apple's Rosetta and X11 as examples. Those are good counter-examples to my point. But then again, those strategies were not done by X11 itself, rather by compatibility layers as you suggested. And in fact this is precisely what a lot of companies are doing, for example the XMir project (http://mjg59.dreamwidth.org/26254.html http://mjg59.dreamwidth.org/26254.html). ____ I agree that that's a great strategy in general, but I don't think it translates well to display servers. You can't just "put it in a sandbox". You need driver support, and you'd need a sandboxed environment sophisticated enough to deal with all the idiosyncrasies of the X protocol. Even putting running an old version of X11/UNIX in a virtual machine is of limited benefit, because X11's core functionality is often tightly coupled with drivers and other things which would need a lot of special wrappers and hacks to work properly. I'm not saying it wouldn't work (and I am not an X11 developer), but I do think there are extremely compelling and legitimate reasons for people to not follow your suggestion (at least in this case).
- recuter 13y ago
  Why not make a clean break? None of this sandbox stuff. X11 2: Electric Bugaloo - The list of major graphical linux programs in active use is very finite and most of it doesn't talk to X directly but relies on stuff like Gnome or QT which are libraries that have frequent churn anyway. No? I bet if you could get the Top 30 projects to target your clean shiny X replacement in one fell swoop you'll end up most of the way there in terms of adoption within a few years. Linux is command line centric and frequently updated after all with users who expect things to break anyway. "Old" X odds and ends that won't be updated can be thought of much like Java Applets are thought of now a days. Oh you need that abandoned Chemistry application from 2006 that depends on X11? Spin up a VM. We can afford to be more pragmatic nowadays. Nobody expects to run 5 year old apps on their smartphones, but desktop linux can't afford a clean break once every 2 decades?
  - derefr 13y ago
    > Oh you need that abandoned Chemistry application from 2006 that depends on X11? Spin up a VM. We can afford to be more pragmatic nowadays. That was exactly what I was suggesting with "sandbox stuff"--except that the OS can provide a stub library that recognizes what that "abandoned Chemistry application from 2006" is asking for, and spin up the VM itself, in the background, making it look as if it was just a compatibility layer. See, for example, Windows 7/8's "Windows XP Mode" -- or, as I mentioned before, Rosetta.
    - Someone 13y ago
      That's not as easy as it sounds. In a desktop GUI, customers expect seamless integration between such sandboxes. Making the clipboard and drag and drop work two-way between such VMs can be a lot of work. Even for simple text, that may involve encoding conversions. That, in turn, means that the host must be able to infer what encoding the guest expects. Styled text and graphics are way harder. The VMs also must react to changes in the parent OS such as a change in the keyboard layout, two apps running in different VMs may both want to have some control over hardware, etc. The approach works well in Windows because Microsoft spends lots of time on it and because the compatibility changes aren't that great. Of course, it also works well in systems where there is no need for the VMs to interact (other than via the network). VM (http://en.wikipedia.org/wiki/VM_(operating_system) http://en.wikipedia.org/wiki/VM_(operating_system)) is a nice example there.
noselasd 13y ago
People get upset when stuff don't work. If you want customers, you make stuff work. You don't break stuff, or change stuff so they have to call you and have you tell them how to make it work.
ori_b 13y ago
And now, you're supporting the old system and the new system. Maybe it's worth it. Maybe it's not. Just keep in mind that if you're keeping an emulated version alive, someone has to mind the emulator, and that's not free.
pcwalton 13y ago
Sandboxing doesn't remove the need to provide ongoing security maintenance for a Web browser engine. For one, sandboxes can be attacked through whatever IPC mechanism you provide (e.g. Pwnium). But more importantly, the Web browser engine enforces security mechanisms that OS-level sandboxing does not address (e.g. the same-origin policy and history sniffing countermeasures).
FollowSteph3 13y ago
How do you deal with security updates? You'd have to apply them to differen code bases all the time. Pas bad as it sounds it's easier to work with one for base

jmathai 13y ago

Joel Spolsky, Things You Should Never Do, Part I http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html Back to that two page function. Yes, I know, it's just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I'll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.

Peaker 13y ago
The right way to resolve such bugs is to have abstraction layers that hide these awful hairiness. If hair regarding low-memory conditions, Internet Explorer and other things all appears in the same function, some abstractions are definitely missing. Also, if the reason for these things isn't clear, that's what comments are for. Rewriting code, from my experience, has had only great results.
- Locke1689 13y ago
  Rewriting can be great as long as you take all the things Joel notes into account and decide it is still the right decision for your project.
- zanny 13y ago
  This is so absolutely true. What nobody recognizes is that most of these really fundamental systems we run on - unix system calls, our window managers, our shells, etc - were designed and implemented in an era where the supercomputers couldn't dent your Nexus 5 in terms of most computing performance metrics. They didn't have the luxury of having an API that might be a few thousand KB big loaded into memory to act as a single redirection to your underlying implementation. They worked in the confines of bytes of memory, not gigabytes. We can afford to be generic, to make extensible runtime programmable interfaces and runtime evaluation type dynamism, because we have the performance necessary. But our core APIs are still written like its 1980.

makomk 13y ago

Amusingly enough, Wayland is already developing backwards-compatibility cruft even though it's not shipping anywhere yet. The core Wayland support for window creation and management cannot support minimizing windows, so it's being essentially obsoleted by an extension called xdg_shell, but xdg_shell isn't mandatory so applications and Wayland compositors will wind up having to support both.

asabjorn 13y ago
I thought things like xdg_shell grow up as either extension or in Weston, and then it gets merged into the stable Wayland protocol after it is battle tested and ready. Is this not the case for xdg_shell?
toyg 13y ago
>The core Wayland support for window creation and management cannot support minimising windows One of the problems with Big Rewrite projects is that, inevitably, early releases will never get feature-parity with the latest version of whatever it is that's being rebuilt. Managing expectations then becomes critical to avoid this sort of legacy-support nightmare.
alexlarsson 13y ago
This is funny. You know that window minimization is not in the X11 spec either, right? Its something that X window managers handle internally (just like on Wayland where this is an internal compositor thing). Of course, desktop apps wanted to interact with minimization (read current state, etc), so the WM authors joined forces to create a spec that allowed this. See how it specs minimization here: http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html#id2731936 http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html#id... This is part of the "wm spec", and the thing corresponding to it in Wayland is called "xdg_spec", Both are produced by "xdg" (X Desktop Group) and are optional things that are used by "traditional desktop environments". It is a good thing that wayland does not enforce the existance of window minimization, because it is not always something that makes sense in all use cases of wayland. For instance in a phone like Jolla.

rthomas6 13y ago

This is probably a stupid question but why can't someone make an X11 version that speeds up and optimizes everything but breaks backwards compatibility?

toyg 13y ago
What are you going to run on it? 99% of userland programs expect compatibility with some historical version of X11. Are you going to rewrite GNOME and KDE and everything else?
- ahomescu1 13y ago
  Many userland applications don't use X libraries directly, but use a toolkit like Qt and GTK. Both projects are working on Wayland support [1, 2], bypassing X completely. 1 - http://wayland.freedesktop.org/qt5.html http://wayland.freedesktop.org/qt5.html 2 - http://wayland.freedesktop.org/gtk.html http://wayland.freedesktop.org/gtk.html
  - krakensden 13y ago
    It's worth noting that GTK+'s abstraction over X11 is fairly leaky in places- hence the always shaky state of its ports to Win & Mac. Qt has less trouble in this regard.
dsr_ 13y ago
Because something you depend on every day needs one of those backward compatible features.
stormbrew 13y ago
There's really no point to doing that, since X is a protocol, and a pretty flexible one at that. Your new shiny can have an X server running in it to act as a go-between and talk something more modern with the new stuff instead. The OP would have been able to do the same stuff with Exceed on Windows or the X server that comes with OSX, after all. X11's survival on linux seems to be largely a matter of momentum and the fact that it's the least common denominator in a fragmented landscape. But now Ubuntu has approached the level of ubiquitousness that's necessary to push for a real change (thus mir) and the rest of the linux world is rallying behind wayland.