9 ms·

> [...] reducing technical debt, improving security practices, etc Reducing technical debt by rewriting everything from scratch (almost?) never works. It's the

by elric 6y ago

> [...] reducing technical debt, improving security practices, etc Reducing technical debt by rewriting everything from scratch (almost?) never works. It's the baby and the bathwater. It's the sort of thing that seems attractive to junior developers, but more seasoned folks know that the legacy system contains years of embedded knowledge and workarounds for "real world" issues. The new, conceptually beautiful system always fails to capture that embedded knowledge, and ends up breaking many things in many ways for many people. Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does.

dataflow 6y ago
https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
- oytis 6y ago
  This is why you need tests, otherwise you're stuck with your legacy forever.
viraptor 6y ago
There are also cases where the embedded knowledge and workarounds collected over years are a problem and can be solved in better ways by nuking the whole section. It's a 36 years old project after all. But it takes deep understanding of the Xorg system to tell which case it is... which I expect most people on HN do not have.
yoz-y 6y ago
> Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does. I haven't followed the development at all but, Xorg is 16 years old and Wayland is 12. It seems counterintuitive that feature parity could not be attained in this time unless it's for philosophical reasons. Edit: Thanks for pointing out that Xorg is actually much older. My argument indeed doesn't hold.
- rmu09 6y ago
  Xorg was kind-of forked from a working solution (XFree) whereas wayland started from scratch.
- jchw 6y ago
  Ahh, 2004 is when Xorg forked off of XFree86, but XFree86 had already been going on since 1991, and already that was an implementation of X which already existed since 1984, and you can trace back even further than that if you want. It’s been a long ride home.
  - roenxi 6y ago
    Xorg development largely halted circa 2008 though, since then most of the "Xorg" features have been deleting code in X and moving it to Mesa/libdrm/the kernel. Wayland's latest release "mostly contains bug fixes and minor protocol updates." [0]. That is ~12 months worth of changes. Pretty similar to 1.18. It isn't like the protocol is a hotbed of massive changes, it looks largely complete according to the release notes. [0] https://lists.freedesktop.org/archives/wayland-devel/2021-January/041692.html https://lists.freedesktop.org/archives/wayland-devel/2021-Ja...
- mschuster91 6y ago
  > I haven't followed the development at all but, Xorg is 16 years old and Wayland is 12. It seems counterintuitive that feature parity could not be attained in this time unless it's for philosophical reasons. Xorg is a fork of Xfree86, which was released in 1991 (https://en.wikipedia.org/wiki/XFree86 https://en.wikipedia.org/wiki/XFree86).
- deleted 6y ago
  [deleted]
- brmgb 6y ago
  > Xorg is 16 years old Xorg is a fork of Xfree86 which started in 1991 but feature parity minus the cruft has been reached years ago. Some people disagree about what is cruft however. For example, I view 90% of the list in the posted articles to be minor softwares relying on undesirable behaviour and unwilling to adapt (the remaining 10% being misrepresentation of actually solved issues listed in bad faith). The original author would obviously disagree. Then again, I consider X.org to be and always have been completely unusable. How can people accept to use a display manager with constant screen tearing is beside me.
jchw 6y ago
I agree that rewriting things needlessly is bad. However I actually think the folks working on this knew exactly what they were doing and actually made a reasonable choice; they’re not rewriting Xorg, they’re making something completely and utterly foreign to what Xorg is. And I think it is justified regardless of whether Wayland ultimately succeeds or fails.
- qznc 6y ago
  Here is a reasonable overview from 2012: https://lwn.net/Articles/491509/ https://lwn.net/Articles/491509/
xorcist 6y ago
It feels important to note that in the case of Wayland, it is indeed designed by the very seasoned folks that have worked with Xorg. If someone like keithp who have worked with X since before X11 has opinions on the design, it is likely well informed. I think very few people involved underestimate the work involved to replace X, and that it is going to be many years before it is somehow done.
casept 6y ago
Well, if the protocol is fundamentally broken there's no way to go but an incompatible rewrite.
- vidarh 6y ago
  The protocol is extensible. Nothing would have stopped an approach of having new clients signal they support new behaviour and aggressively deprecate old functionality to iterate the protocol. The problem was not the protocol, but that the Wayland devs explicitly wanted to throw out a whole lot of functionality people actually depend on without having put thought into replacements.
WhyNotHugo 6y ago
> It's the sort of thing that seems attractive to junior developers, but more seasoned folks know that the legacy system contains years of embedded knowledge and workarounds for "real world" issues. Many seasoned Xorg developers work on Wayland. All the decades of lessons form Xorg _were_ carried over to Wayland.
- sokoloff 6y ago
  Many of the lessons are being carried over no doubt. Others are embodied in the heads of people not working on Wayland or in the codebase itself as emergent properties from interactions never consciously designed. There will be some loss of lessons with almost no doubt.
- phkahler 6y ago
  >> Many seasoned Xorg developers work on Wayland. All of them are either working on Wayland or moved on. There are no Xorg developers.
  - scaladev 6y ago
    Who is downvoting this and why? Face it, Xorg is on life support. The only patches it still receives are Xwayland-related fixes.
    - shock 6y ago
      I don't expect a stable application to receive many patches. How many patches do you think Xorg should receive nowadays and in what areas?
      - bluGill 6y ago
        There are a ton of long standing security issues. Which is why they are working on wayland: X11 isn't possible to secure as bad security is part of how it works.
        6y ago
majewsky 6y ago
> It's the sort of thing that seems attractive to junior developers You make it sound like Wayland has been pushed by some script kiddies, when in fact the lead developers on Wayland are the actual Xorg maintainers. To me, the migration from X11 to Wayland looks similar to the switch from DOS-based Windows to Windows NT. At the time, lots of programs were broken by the move because they assumed you could meddle in other application's internal memory. But breaking those applications was not malicious intent on the side of the Windows devs. It was a legimitate move towards a more sustainable architecture. I'm not intimately familiar with X11 internals, but my read of the situation is that X11 just exposes too many internals in its interfaces, and therefore it becomes increasingly impossible to adapt it to contemporary needs (in this case, application sandboxing).
- DyslexicAtheist 6y ago
  > when in fact the lead developers on Wayland are the actual Xorg maintainers. Daniel Stone[0] incredibly funny and educational talk[1] from 2013 could help many critics who missed the 3 decade long ongoing discussion of why X is bad. We've talked about this already in the mid 90ies (when there were no alternatives) and now 8 years after this popular linux.conf.au talk, people still having troubles understanding why X has always been a terrible tradeoff for both security and performance (well for literally everything). [0] https://www.fooishbar.org/about/ https://www.fooishbar.org/about/ [1] The Real Story Behind Wayland and X https://www.youtube.com/watch?v=RIctzAQOe44 https://www.youtube.com/watch?v=RIctzAQOe44 edit: BUT X IS THE UNIX WAY! << so what is the 1 thing X does, and what is it doing well?
  - posix_me_less 6y ago
    The talk was interesting but it also shows how the speaker is a little deluded that everything turns around technical quality of code and developers. X is used and preferred by many users and application authors because it is the standard and it works. > so what is the 1 thing X does, and what is it doing well? It provides functional and well-established(accepted standard) graphics server technology/libraries for any application that uses graphics. I don't care if it takes 10 liters of blood to compile one version of X or whatever. I'm a user. All I care is that my desktop works as robustly as did it yesterday or 5 years ago. If that's not happening, I'm not interested in macros, classes, variables, declarations, structs, or any other geeky CS technobabble. That's irrelevant. And a product that advertises itself as being created to be convenient for the people developing it is a paradox. Don't develop it, then. Makes things even easier. -- from https://www.dedoimedo.com/computers/fedora-25-wayland-vs-xorg.html https://www.dedoimedo.com/computers/fedora-25-wayland-vs-xor...
sz4kerto 6y ago
This is not a rewrite. It's a new product that has overlapping functionality with X11. Rewrites don't work when the following two statements are both true: - the rewrite should achieve feature parity at a higher generaly quality - the people who're doing the rewrite don't have most or all the knowledge that people working on the legacy system had (this could be written docs, automated tests, or just actual people hanging around)
tsss 6y ago
Let me tell you an anecdote: A big, old, crusty online retail company wants to bring their web shop up to par with the competitors. Due to many years of technical debt, feature development speed has come to a halt and it is determined that the web shop must be rewritten to remain competitive. Over a period of months, requirements are gathered from user focus groups and all the employees who maintained the content on the previous shop. Development starts and 3 years later, still not having implemented all the requested features, the project is deemed a failure. Another 3 years later, a new consulting company is brought in to attempt to realize the project. But this time, instead of trying to replace the old shop, a completely new shop is built iteratively and independently. Domain experts in the company complain: there is no way to review products, they can no longer put text wherever they want in the shop and users have to call the hotline to unsubscribe from newsletter because there is no settings menu. Still, the new shop is already outperforming the old one, despite having far less features. It turns out that many of those features weren't needed after all. The lesson here is that writing a direct replacement system rarely succeeds because the new system will hardly be able to implement all the features in a reasonable time. Instead you should take a step back and develop a completely new system that follows actual user requirements and gets rid of all the dead code.
ragnese 6y ago
Everything you said is correct. But, there are surely cases where even though rewriting everything from scratch almost never works, the status quo doesn't work either. I'm afraid that's where the Linux desktop is today. Linux as a platform actually sucks for running "untrusted" software. And "untrusted" doesn't just mean proprietary games. It also just means "can I please run the alpha version of this cool open source project without being afraid that it'll fuck up my whole home directory?" Android does a hacky thing where each program is its own "user". But then, having multiple (human) users is confusing (I don't know how Android does this, actually). I don't know if Wayland will succeed. In fact, I'm rather pessimistic about it. But I'm also pessimistic about desktop Linux "succeeding" (staying reasonably useful even to nerds) if Wayland doesn't.
- GekkePrutser 6y ago
  Android does a thing where it has its own display stack and very strict sandboxing so X11 isn't really a thing :) And for the sandboxing it uses SELinux which is available on desktop as well. It's just a royal PITA to configure. This is why many people don't bother with it. But in a highly restricted environment like a smartphone it's a lot easier. I wouldn't call that a 'hacky' thing, it's just using it in a very different environment.
  - ragnese 6y ago
    The Android thing was a bit of a tangent. I did know that it doesn't use X. The point was intended to be that Linux is becoming more and more lacking as a personal computing platform, relative to the gains elsewhere in the space. The display server is one part of that- the entire user/group permission model is another part. The part of Android that is "hacky" is using a separate user for every app. That's obviously not what the Linux/Unix guys had in mind for the user mechanism, thus it's a "hack". It's not necessarily a bad thing and it's totally hidden from the UX of your typical Android user, so it's fine. Just interesting.
  - rkangel 6y ago
    > And for the sandboxing it uses SELinux which is available on desktop as well I used to think that, and then I started using Fedora. They have put a lot of effort into their SeLinux policies. I do all sorts of things with my workstation and very rarely have to care. In the last 6 months the only thing I have had to do is add ":z" to a bind mount running a podman container so that labelling happened correctly.
stuaxo 6y ago
It doesn't ? Maybe we've worked on different projects, but sometimes the legacy is just that things were done differently in the past, and the APIs available were crap.
HeadsUpHigh 6y ago
>It's the sort of thing that seems attractive to junior developers, but more seasoned folks know that the legacy system contains years of embedded knowledge and workarounds for "real world" issues. The Wayland developers are the same people who have been building all these workarounds for xorg for decades. This isn't a new thing coming out of nowhere. When the old guard is telling you that their own work needs to be replaced, I'd listen. >Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does. Good thing nobody forces you to use it and it's just one click away on the login screen from switching to xorg. In the meantime people who want to hack on the project can do so and ship their changes to users to test them plus those who do suffer from xorg issues( because yes, xorg doesn't do well on a lot of things that aren't even that modern despite all the workarounds for the "real world" issues it has) can use wayland right away.
dreamcompiler 6y ago
> Reducing technical debt by rewriting everything from scratch (almost?) never works. Agreed but this is one of those "almost" exceptions. 1. Is the code built to 40+ year old requirements, many of which are irrelevant today? 2. Is it hard to make it work for today's requirements without widespread fundamental changes? 3. Is the code a huge pile of foo that's damn near impossible to navigate, much less understand, much less debug? 4. Would a hypothetical program that solved today's requirements (with an architecture where the most important backwards compatibility features could be hooked in as needed) be smaller, simpler, and easier to maintain by newcomers? If the answer to all of the above is "hell yes!", you might want to think about a wholesale rewrite.

by elric 6y ago

dataflow 6y ago

https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...

oytis 6y ago
This is why you need tests, otherwise you're stuck with your legacy forever.

viraptor 6y ago

There are also cases where the embedded knowledge and workarounds collected over years are a problem and can be solved in better ways by nuking the whole section. It's a 36 years old project after all. But it takes deep understanding of the Xorg system to tell which case it is... which I expect most people on HN do not have.

yoz-y 6y ago

> Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does. I haven't followed the development at all but, Xorg is 16 years old and Wayland is 12. It seems counterintuitive that feature parity could not be attained in this time unless it's for philosophical reasons. Edit: Thanks for pointing out that Xorg is actually much older. My argument indeed doesn't hold.

rmu09 6y ago
Xorg was kind-of forked from a working solution (XFree) whereas wayland started from scratch.
jchw 6y ago
Ahh, 2004 is when Xorg forked off of XFree86, but XFree86 had already been going on since 1991, and already that was an implementation of X which already existed since 1984, and you can trace back even further than that if you want. It’s been a long ride home.
- roenxi 6y ago
  Xorg development largely halted circa 2008 though, since then most of the "Xorg" features have been deleting code in X and moving it to Mesa/libdrm/the kernel. Wayland's latest release "mostly contains bug fixes and minor protocol updates." [0]. That is ~12 months worth of changes. Pretty similar to 1.18. It isn't like the protocol is a hotbed of massive changes, it looks largely complete according to the release notes. [0] https://lists.freedesktop.org/archives/wayland-devel/2021-January/041692.html https://lists.freedesktop.org/archives/wayland-devel/2021-Ja...
mschuster91 6y ago
> I haven't followed the development at all but, Xorg is 16 years old and Wayland is 12. It seems counterintuitive that feature parity could not be attained in this time unless it's for philosophical reasons. Xorg is a fork of Xfree86, which was released in 1991 (https://en.wikipedia.org/wiki/XFree86 https://en.wikipedia.org/wiki/XFree86).
deleted 6y ago
[deleted]
brmgb 6y ago
> Xorg is 16 years old Xorg is a fork of Xfree86 which started in 1991 but feature parity minus the cruft has been reached years ago. Some people disagree about what is cruft however. For example, I view 90% of the list in the posted articles to be minor softwares relying on undesirable behaviour and unwilling to adapt (the remaining 10% being misrepresentation of actually solved issues listed in bad faith). The original author would obviously disagree. Then again, I consider X.org to be and always have been completely unusable. How can people accept to use a display manager with constant screen tearing is beside me.

jchw 6y ago

I agree that rewriting things needlessly is bad. However I actually think the folks working on this knew exactly what they were doing and actually made a reasonable choice; they’re not rewriting Xorg, they’re making something completely and utterly foreign to what Xorg is. And I think it is justified regardless of whether Wayland ultimately succeeds or fails.

qznc 6y ago
Here is a reasonable overview from 2012: https://lwn.net/Articles/491509/ https://lwn.net/Articles/491509/

xorcist 6y ago

It feels important to note that in the case of Wayland, it is indeed designed by the very seasoned folks that have worked with Xorg. If someone like keithp who have worked with X since before X11 has opinions on the design, it is likely well informed. I think very few people involved underestimate the work involved to replace X, and that it is going to be many years before it is somehow done.

casept 6y ago

Well, if the protocol is fundamentally broken there's no way to go but an incompatible rewrite.

vidarh 6y ago
The protocol is extensible. Nothing would have stopped an approach of having new clients signal they support new behaviour and aggressively deprecate old functionality to iterate the protocol. The problem was not the protocol, but that the Wayland devs explicitly wanted to throw out a whole lot of functionality people actually depend on without having put thought into replacements.

WhyNotHugo 6y ago

sokoloff 6y ago
Many of the lessons are being carried over no doubt. Others are embodied in the heads of people not working on Wayland or in the codebase itself as emergent properties from interactions never consciously designed. There will be some loss of lessons with almost no doubt.
phkahler 6y ago
>> Many seasoned Xorg developers work on Wayland. All of them are either working on Wayland or moved on. There are no Xorg developers.
- scaladev 6y ago
  Who is downvoting this and why? Face it, Xorg is on life support. The only patches it still receives are Xwayland-related fixes.
  - shock 6y ago
    I don't expect a stable application to receive many patches. How many patches do you think Xorg should receive nowadays and in what areas?
    - bluGill 6y ago
      There are a ton of long standing security issues. Which is why they are working on wayland: X11 isn't possible to secure as bad security is part of how it works.
      - 6y ago

majewsky 6y ago

> It's the sort of thing that seems attractive to junior developers You make it sound like Wayland has been pushed by some script kiddies, when in fact the lead developers on Wayland are the actual Xorg maintainers. To me, the migration from X11 to Wayland looks similar to the switch from DOS-based Windows to Windows NT. At the time, lots of programs were broken by the move because they assumed you could meddle in other application's internal memory. But breaking those applications was not malicious intent on the side of the Windows devs. It was a legimitate move towards a more sustainable architecture. I'm not intimately familiar with X11 internals, but my read of the situation is that X11 just exposes too many internals in its interfaces, and therefore it becomes increasingly impossible to adapt it to contemporary needs (in this case, application sandboxing).

DyslexicAtheist 6y ago
> when in fact the lead developers on Wayland are the actual Xorg maintainers. Daniel Stone[0] incredibly funny and educational talk[1] from 2013 could help many critics who missed the 3 decade long ongoing discussion of why X is bad. We've talked about this already in the mid 90ies (when there were no alternatives) and now 8 years after this popular linux.conf.au talk, people still having troubles understanding why X has always been a terrible tradeoff for both security and performance (well for literally everything). [0] https://www.fooishbar.org/about/ https://www.fooishbar.org/about/ [1] The Real Story Behind Wayland and X https://www.youtube.com/watch?v=RIctzAQOe44 https://www.youtube.com/watch?v=RIctzAQOe44 edit: BUT X IS THE UNIX WAY! << so what is the 1 thing X does, and what is it doing well?
- posix_me_less 6y ago
  The talk was interesting but it also shows how the speaker is a little deluded that everything turns around technical quality of code and developers. X is used and preferred by many users and application authors because it is the standard and it works. > so what is the 1 thing X does, and what is it doing well? It provides functional and well-established(accepted standard) graphics server technology/libraries for any application that uses graphics. I don't care if it takes 10 liters of blood to compile one version of X or whatever. I'm a user. All I care is that my desktop works as robustly as did it yesterday or 5 years ago. If that's not happening, I'm not interested in macros, classes, variables, declarations, structs, or any other geeky CS technobabble. That's irrelevant. And a product that advertises itself as being created to be convenient for the people developing it is a paradox. Don't develop it, then. Makes things even easier. -- from https://www.dedoimedo.com/computers/fedora-25-wayland-vs-xorg.html https://www.dedoimedo.com/computers/fedora-25-wayland-vs-xor...

sz4kerto 6y ago

This is not a rewrite. It's a new product that has overlapping functionality with X11. Rewrites don't work when the following two statements are both true: - the rewrite should achieve feature parity at a higher generaly quality - the people who're doing the rewrite don't have most or all the knowledge that people working on the legacy system had (this could be written docs, automated tests, or just actual people hanging around)

tsss 6y ago

Let me tell you an anecdote: A big, old, crusty online retail company wants to bring their web shop up to par with the competitors. Due to many years of technical debt, feature development speed has come to a halt and it is determined that the web shop must be rewritten to remain competitive. Over a period of months, requirements are gathered from user focus groups and all the employees who maintained the content on the previous shop. Development starts and 3 years later, still not having implemented all the requested features, the project is deemed a failure. Another 3 years later, a new consulting company is brought in to attempt to realize the project. But this time, instead of trying to replace the old shop, a completely new shop is built iteratively and independently. Domain experts in the company complain: there is no way to review products, they can no longer put text wherever they want in the shop and users have to call the hotline to unsubscribe from newsletter because there is no settings menu. Still, the new shop is already outperforming the old one, despite having far less features. It turns out that many of those features weren't needed after all. The lesson here is that writing a direct replacement system rarely succeeds because the new system will hardly be able to implement all the features in a reasonable time. Instead you should take a step back and develop a completely new system that follows actual user requirements and gets rid of all the dead code.

ragnese 6y ago

Everything you said is correct. But, there are surely cases where even though rewriting everything from scratch almost never works, the status quo doesn't work either. I'm afraid that's where the Linux desktop is today. Linux as a platform actually sucks for running "untrusted" software. And "untrusted" doesn't just mean proprietary games. It also just means "can I please run the alpha version of this cool open source project without being afraid that it'll fuck up my whole home directory?" Android does a hacky thing where each program is its own "user". But then, having multiple (human) users is confusing (I don't know how Android does this, actually). I don't know if Wayland will succeed. In fact, I'm rather pessimistic about it. But I'm also pessimistic about desktop Linux "succeeding" (staying reasonably useful even to nerds) if Wayland doesn't.

GekkePrutser 6y ago
Android does a thing where it has its own display stack and very strict sandboxing so X11 isn't really a thing :) And for the sandboxing it uses SELinux which is available on desktop as well. It's just a royal PITA to configure. This is why many people don't bother with it. But in a highly restricted environment like a smartphone it's a lot easier. I wouldn't call that a 'hacky' thing, it's just using it in a very different environment.
- ragnese 6y ago
  The Android thing was a bit of a tangent. I did know that it doesn't use X. The point was intended to be that Linux is becoming more and more lacking as a personal computing platform, relative to the gains elsewhere in the space. The display server is one part of that- the entire user/group permission model is another part. The part of Android that is "hacky" is using a separate user for every app. That's obviously not what the Linux/Unix guys had in mind for the user mechanism, thus it's a "hack". It's not necessarily a bad thing and it's totally hidden from the UX of your typical Android user, so it's fine. Just interesting.
- rkangel 6y ago
  > And for the sandboxing it uses SELinux which is available on desktop as well I used to think that, and then I started using Fedora. They have put a lot of effort into their SeLinux policies. I do all sorts of things with my workstation and very rarely have to care. In the last 6 months the only thing I have had to do is add ":z" to a bind mount running a podman container so that labelling happened correctly.

stuaxo 6y ago

It doesn't ? Maybe we've worked on different projects, but sometimes the legacy is just that things were done differently in the past, and the APIs available were crap.

HeadsUpHigh 6y ago

>It's the sort of thing that seems attractive to junior developers, but more seasoned folks know that the legacy system contains years of embedded knowledge and workarounds for "real world" issues. The Wayland developers are the same people who have been building all these workarounds for xorg for decades. This isn't a new thing coming out of nowhere. When the old guard is telling you that their own work needs to be replaced, I'd listen. >Perhaps we did need a new display server, perhaps Wayland is conceptually better than Xorg, but it'll be many years before it reaches feature parity, if it ever does. Good thing nobody forces you to use it and it's just one click away on the login screen from switching to xorg. In the meantime people who want to hack on the project can do so and ship their changes to users to test them plus those who do suffer from xorg issues( because yes, xorg doesn't do well on a lot of things that aren't even that modern despite all the workarounds for the "real world" issues it has) can use wayland right away.

dreamcompiler 6y ago

> Reducing technical debt by rewriting everything from scratch (almost?) never works. Agreed but this is one of those "almost" exceptions. 1. Is the code built to 40+ year old requirements, many of which are irrelevant today? 2. Is it hard to make it work for today's requirements without widespread fundamental changes? 3. Is the code a huge pile of foo that's damn near impossible to navigate, much less understand, much less debug? 4. Would a hypothetical program that solved today's requirements (with an architecture where the most important backwards compatibility features could be hooked in as needed) be smaller, simpler, and easier to maintain by newcomers? If the answer to all of the above is "hell yes!", you might want to think about a wholesale rewrite.