14 ms·

Winding down my Debian involvement (2019)

181 points by ecliptik 6y ago

__s 6y ago
(2019) He's now experimenting with a distro focussing on a fast package manager: https://michael.stapelberg.ch/posts/2019-08-17-introducing-distri https://michael.stapelberg.ch/posts/2019-08-17-introducing-d...
- Semaphor 6y ago
  And here is the original HN discussion, 227 comments: https://news.ycombinator.com/item?id=19353010 https://news.ycombinator.com/item?id=19353010
- Ericson2314 6y ago
  It seems like he is not aware of Nix or Guix, which is a travesty.
  - CameronNemo 6y ago
    He is. Check out the Q&A of his 2020 ArchConf presentation. He explains why NixOS does not fit Hus needs.
    - Ericson2314 6y ago
      Buried after a talk, yay.
      - Ericson2314 6y ago
        I found https://static.conf.archlinux.org/2020/archconf/talks/distri%3A%20researching%20fast%20Linux%20package%20management/6387_distri.mp4 https://static.conf.archlinux.org/2020/archconf/talks/distri... Let me quote a thing starting at 26:49: > The performance improvements distri provides, definitely some of them can apply to Nix. I think there are some low hanging fruit in Nix. The other isuess do reflect some persistent rhetorical issues we've had with explaining Nix: > There was little differentiation with Nix vs NixOS. (Some of his philosophical difference could be resolved by just using Nix.) > There was no recognition that the "nix language part of Nix" is cleanly layered away from the layers that actually do the work of running jobs and moving files around, and can be replaced like Guix does. So I do want our materials to highlight this so experimenters realize Nix is less of an all-in proposition than it sounds (if one is already willing to do the extra work of blazing their own trail).
        makebackupstoo 6y ago
        A few blog posts on "how to make your own guix!" would be really neat. I'm not as deep as I'd like to be, but roughly it's a simply a matter of outputting a derivation that `nix-daemon/nix-build` can handle, right?
        Ericson2314 6y ago
        I do want to add a section to the Nix manual that describes what it is (data model, key abstractions, etc.). Everything today is some variation on "how to use it".
      - orhmeh09 6y ago
        What are you mad at? You seemed frustrated he was unaware, and now you find that he was aware, you’re mad that the information on his state of awareness was difficult for you to find? Why does it matter to you? Or are you disappointed that his assessment does not match your opinion of Nix et al.?
        Ericson2314 6y ago
        I'm relieved he is aware. I'm disappointed that the overhead of trying new things here is higher than I think it needs to be, because making these things from scratch means doing lots of not-innovative between the resesrchy bits. I care because there are very few people who care this much about packaging, and if our efforts were less divided it would go a lot further. I'm a little disappointed by what feels like a superficial take on Nix, but I am quite used to that now. And indeed, our documentation is bad about at explaining the essence of the thing.
    - makebackupstoo 6y ago
      I have yet to see Michael discuss Nix without spreading inaccurate, flatly wrong information about it (Nix does not have post-install hooks, and `nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling). Nix fulfills every single requirement Michael has put forward (except the squash>tar thing, which I still don't understand). That's all there is to this. I sympathize with Ericson's frustration. It's exhausting watching people re-invent inferior solutions to Nix, instead of just hopping in and fixing or using Nix. Of course, John Ericson is one of the few people motivated, qualified (and maybe has the buy-in) to make changes in Nix. I'm thankful for that on-going work.
      - st1ck 6y ago
        fast, but more typing: `nix-env -iA nixos.packageName`
      - Ericson2314 6y ago
        > I'm thankful for that on-going work. Thank you!
      - rofrol 6y ago
        >`nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling Can you provide more info on that?
        Ericson2314 6y ago
        It evaluates the build plan for everything we have, and then searches the things that were evaluated. The new unstable CLI has an evaluation cache at least. What most people do today is look for keys in the object and just evaluate what they need. (The Nix language is lazily evaluated so you can explore like this pretty well out of the box in the repl.) Or, they just grep Nixpkgs :D. ------- All of this, problems and solutions alike, is a weird situation to be in. I still stand by "just get rid of nix-env -i", but I want there to be better solutions too. Nixpkgs seems overly complex, and is in some ways, but the fact its trying to herd a gazillion upstream packages that don't meaningfully coordinate makes this harder to fix than it should be.
  - geofft 6y ago
    In https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... he measures NixOS and determines it doesn't solve his problems. More commentary at https://twitter.com/zekjur/status/1162779362682384384 https://twitter.com/zekjur/status/1162779362682384384 (though it sounds like part of it is the fact that the "right" was to install something is nix-env -iA nixpkgs.whatever instead of nix-env -i whatever, which I too find confusing).
    - Ericson2314 6y ago
      NIH hobby projects with good ideas that are doomed to go to waste rub me the wrong way. They are only a temporary reprieve from the burnout felt by for those that author them, anyways. Here's the thing, Nix is to slow (even ignoring `nix-env`'s terrible search functionality which should just be removed). What that requires is some good old boring profiling and optimization work. If he were to contribute that to a distro/package manager that basically shares his vision, this would be much more useful to the world. Cause, at the end of the day, the work isn't so much maintaining the package manager as maintaining the packages. That's simply too much work for anyone to do alone. http://blog.williammanley.net/2020/05/25/unlock-software-freedom-one-by-using-better-tools.html http://blog.williammanley.net/2020/05/25/unlock-software-fre... is good piece on why the ultimate issues with packaging are social, not technological. At this point, when the vast majority of devs don't seem to act as if there is a commons that even needs integration, I don't think any 1-person technological solution is going to be so good as to upend the social situation.
      - basicneo 6y ago
        Even if it were true that a hobby project is just a temporary reprieve from burnout... so what? They're free to do so. And maybe we'll all get something good out of it at the end. The author has a history of delivering quality OSS projects: i3, Debian Code Search, RobustIRC, gokrazy.
      - geofft 6y ago
        I don't think this is intended as a one-person technological solution. It's a one-person research project, trying to see if there are any architectural changes that can be applied to other distros that people actually use, such as Debian or Nix. It's true that the commons needs people willing to put in time and effort on boring things, but they have to be boring in the first place. If the author were to show up and say "Hey, Nix, if you rearchitect in this massive way it may or may not bring big improvements" and sent in a pull request, it would be rightly rejected. But it's still possible that a few days of rearchitecture can deliver the same results as a year of profiling and microoptimizations. The point of distri, as I understand it, is to have something to point to and say, this architectural change will actually work, and it's worth implementing in an actually-used distro.
  - deleted 6y ago
    [deleted]
  - soraminazuki 6y ago
    It looks like distri have very different design goals from Nix with an emphasis on speed rather than rock solid reproducibility. Although some of the key details do seem to be inspired by Nix, that's no reason to discourage experimentation by calling it a travesty.
- arendtio 6y ago
  Well, in my opinion the Arch Linux package manager is fast enough, but that might be influenced the years I was using Gentoo (Portage & Paludis) which aren't even part of his benchmark... Edit: For clarification: I am referring to this benchmark https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... For those who are unaware, Portage and Paludis are both magnitudes slower than the other packages managers. However, the comparison is not completely fair, as they both where source based package managers in the beginning, which have to work a bit differently. Nevertheless, Paludis is still a lot slower when being used with binary packages as it doesn't take all the shortcuts the others take.
Foxboron 6y ago
This probably got posted because he reworked the blog theme and fixed the post. This rebumped the post in the RSS feeds. AFAIK nothing new has been added?
- ecliptik 6y ago
  This is exactly what happened. Saw the post pop up in Newsblur this morning in Planet Debian feed, but I didn't notice the original date since I read it in "reader" view, otherwise I would have tagged it with a 2019. I didn't see it the first time around, and it caught my attention since I've had an interest in becoming a Debian Maintainer for about 15 years now, just never fully did it (even have a GPG key signed by a few Debian Devs) and a lot of the frustrations echoed with me.
  - secure 6y ago
    Thanks for clarifying :) I had not realized that my blog template update would result in a repost to various feed consumers. Will try and figure out why that happened. Maybe I can also add the year to the post header when the post is older than X days. Anyway, bringing this post up again is not a bad thing :)
ed25519FUUU 6y ago
> When I joined Debian, I was still studying, i.e. I had luxurious amounts of spare time. I felt so busy in college, what a tragedy! If only I know what it mean later in life. The best part about college for me is I could disappear for the summer, going somewhere and doing what I wanted, knowing that my life would wait for me until Fall.
aerovistae 6y ago
Not strictly related, but I was thinking about this the other day: how long will it be before some group or company creates a truly new operating system that takes off? I mean, the windows kernel is going on 20 years old. the Mac OS kernel is based on unix, which is even older. Linux is also based on unix. As are Android and ios. The more we add to these operating systems, the harder it becomes to walk away from them because we have so much invested in them. Does this mean that in 200 years we'll still be using the descendants of these early operating systems? Under what circumstances would someone decide to start something truly new? And what would it take to ever reach a feature parity with the existing options? And to be clear, I'm not saying there's a reason to walk away from these. I'm not an operating system programmer, I don't even really know that much about it. I'm just wondering if it will always make sense to just keep adding to what we already have.
- rstuart4133 6y ago
  > how long will it be before some group or company creates a truly new operating system that takes off? You won't get another Windows / Linux / BSD. They do their jobs very well, and as WSL emulating Linux proves they are near interchangeable. I have no doubt Linux could emulate the Windows kernel near perfectly too if we had the Windows source to see what is required. When you've already go a slew of interchangeable parts, we create another one? If something new is to displace them, it's going to have to do something very different. The only thing I can see is not a replacement, but a security kernel that allows us to establish a trust chain from the hardware to some application. It wasn't necessary when the hardware sits on your lap or desk, but now we are tending to rent out CPU cycles from some machine on the other side of the planet and yet still want to have some assurance of privacy, they are becoming kinda essential. There are a number of proprietary ones out there now, but I think that's doomed to fail. No one in the right mind would put their faith in a binary that could be in cahoots with anybody, from the USA government to a ransomware gang.
- wilsonthewhale 6y ago
  It's more likely for these systems to slowly evolve into something new rather than being replaced entirely. MacOS and Linux, at some level, are already very, very different than earlier unix systems. The BSD's are more conservative, but there's still systems being reworked across all of them (HAMMER in Dragonfly, pledge/unveil in openbsd, etc.). The ideas of unix will probably be with us forever, but the precise details of implementation are transient.
eclipseo76 6y ago
Reading this I'm glad I'm working on Fedora.: one git per package, one tool to rule them all (fedpkg), one tool to sync them all locally (grokmirror), (relatively) easy global changes through proven packagers who can ask for mass changes before each release. I don't know how I would do it if I had to deal with svn, mercurial or no scm at all. I understand the want of being decentralized but this is being done at the maintainers expense, who are often already stretched thing.
- ausjke 6y ago
  except that debian's apt repo style have been there for decades and still worked well, redhat/fedora's pkg and development model instead had changed who knows how many times, and of course the current one is the golden one, until next one arrives that is.
  - Conan_Kudo 6y ago
    The current RPM repository metadata format has existed since Fedora Core 2 in 2004. The build system infrastructure changed to Koji in 2007[1]. The development model has changed exactly once when Fedora switched from CVS to Git in 2009[2]. There have been no significant changes to Fedora packaging model until three years ago, when Modularity was introduced[3] and Pagure was deployed to ease contributions and support building modules[4]. And the modularity concept is primarily used for alternate software streams in Fedora, so the vast majority of Fedora packages don't use this feature. [1]: https://fedoraproject.org/wiki/Releases/FeatureNewBuildSystem https://fedoraproject.org/wiki/Releases/FeatureNewBuildSyste... [2]: https://fedoraproject.org/wiki/Dist_Git_Proposal https://fedoraproject.org/wiki/Dist_Git_Proposal [3]: https://docs.pagure.org/modularity/ https://docs.pagure.org/modularity/ [4]: https://fedoraproject.org/wiki/Changes/ArbitraryBranching https://fedoraproject.org/wiki/Changes/ArbitraryBranching
- CameronNemo 6y ago
  dnf/RPM is the biggest offender in his benchmarks when it comes to package manager speed. Not only is metadata for a package an order of magnitude larger, the package manager itself works slowly.
deleted 6y ago
[deleted]
gorgoiler 6y ago
OS work used to feel so ceremonial, back in the old days (ie 2003.) Hobbyists could cobble together hardware and ship it to a data center, but if you couldn’t afford a serial console you’d have to unrack the machine every time you messed up a kernel upgrade. Now I can just remotely blast a clean install on bare metal, and then build containers or VMs on that, and it’s so easy I can rebuild all infra every morning, from scratch, just for fun (ahem, to verify it’s idempotency.) Gone is the need for the high quality package management of Debian: The Universal Operating System. I lament this as much as I lament the decline in quality and commoditization of many many other things in life. Food, ISPs, journalism, education.
- core-questions 6y ago
  > Gone is the need for the high quality package management of Debian: The Universal Operating System. Well, almost. What's inside your container, though? For anything of moderate complexity, my containers always end up having some apt-get or yum installation in them; it's not like I want to Dockerfile up the manual from-source install steps for every small package, nor do I expect to have perfect upstream Dockerfiles to FROM that include all the exact bits I need....
  - gorgoiler 6y ago
    Yes, but they are usually single task hosts. All the effort that maintainers put into keeping /etc tidy when multiple pieces of software are installed on the same host are for nought.
    - toyg 6y ago
      So you install exactly one package in each container...? I find it hard to believe. You wouldn’t be able to boot with a single package. Feels very much like saying “all the stuff I don’t (think I) need, is useless”.
  - ryukafalz 6y ago
    Not to mention having stuff built for you for all sorts of different architectures. It generally doesn't matter if you're running x86, ARM, whatever if you're installing packages from Debian; by and large everything in the distro is packaged for every architecture and they generally just work.
debiandev 6y ago
A couple of healthy reminders to avoid drama and FUD: 1) People announce leaving publicly and the FLOSS community takes notice of it. This is a sign of health of Debian. In many other projects few people notices. 2) The number of Debian developers, projects, and packages has been increasing for decades. 3) For each person writing on mailing lists and blogs there are 10 people quietly contributing. 4) The same applies to the occasional flamewars. Vocal minorities are not representative of the thousands of DDs and contributors.
- zepto 6y ago
  This seems like damage control and image management. The piece was cogent, respectful, and constructive. How about addressing his points?
  - rstuart4133 6y ago
    I'm a DD. His points left me scratching my head. Yes, Debian's infrastructure is old, but so is Big Ben and just a Big Ben still does an admirable job of broadcasting the time to the locals, Debian Bug Tracker does an admirable job of tracking bugs. I get it that he personally might prefer to interact with it via a more modern web page, but that wouldn't alter how well it tracks bugs or facilitates discussions. And besides, I find email easier to interact with automatically than a web page. Oddly he goes on to list not being able to automate things as a complaint. The same goes for the rest of the things he lists. Yes, he might prefer to do them some other way, but the way they are done now has obviously been working very well for a long time. As for Debian being incapable of making big changes - that's just rubbish. He's been there for 10 years for pete's sake, systemd was a big change requiring many packages to updated spanning several years and several releases, while still delivering a working system as it happened. That's not big? How about altering the source package format, or moving away from sha1 for signing, or making everything build reproducibly, or moving all developers using its collaborative development platform from FusionForge to GitLab? Sorry, he's just plain wrong on the "can't make big changes point". Debian regularly makes big changes every major release. In Bullseye they will have made big strides in migrating away from Python2. Changes of this scale are things other projects regularly struggle with, but not Debian. His posts lists a whole pile of things about Debian he's discovered he no longer likes, which is fair enough. But as he says in his introduction it's him whose that's changed, he gone from a student with lots of time on his hands who was happy to be part of loosely collaborating group to being a member of a very focused and highly directed team at work, and he's discovered he prefers the latter. Great, I get it, happens to all of us. But that doesn't mean the Debian no longer works. It clearly works very well. It just means he's no longer a great fit for that way of doing things.
pelasaco 6y ago
Looking to openSUSE, community that i contribute to, i have the same feeling about almost all points. To keep having bugzilla around, makes me crazy. 2020 and you still not able to delete a comment.
- tlamponi 6y ago
  > 2020 and you still not able to delete a comment. or a report, no you need to create a bogus product (which is not a single click either), move the report there, and then delete the whole product. It's really a bit a PITA to use, the search is IMO quite good though.
  - phendrenad2 6y ago
    Wow, I'd love to know the story behind that. My guesses (A) No one cares enough to add it (B) some core maintainer is actively hostile to the idea (maybe someone deleted their bug report once and they've been getting revenge on the world ever since) or (C) the codebase is too fucked up for anyone to figure out how to add a delete button. But the truth is probably more interesting.
    - jodrellblank 6y ago
      Industry-wide institutionalised hoarding disorder that condemns us all to live atop the ever-expanding junk heap of the past, with the hoarder's justification "might come in useful one day". And that workaround is an accidental bug. A world where big important decisions made in person go completely undocumented, yet weeks of bickering and bikeshedding over nothing are kept forever.
toyg 6y ago
Debian is like democracy: the worst way of producing an OS, except all the others that have been tried from time to time. BeOS, AmigaOS, Solaris, most other 80s OSes - they’re effectively dead. Windows and macOS have effectively died once already. The BSDs can stall for years at times. Most Linux distributions (including RedHat) are typically only as good as the fortunes of the commercial (or occasionally public) entity they have behind. In all this, Debian endures, with its slow but inexorable progress, simply because its ideological foundations - not its technical ones - are eminently superior to all the others. Debian contributors don’t do it for the money, so they will be there when money runs out; and they don’t do it for being cool either, so they will be there when OS work is not cool. People will come and go, but the ideal of the “democratic OS” will always be there - hence, Debian will be too.
- smabie 6y ago
  Arch Linux is much superior to Debian, imo.
  - rovr138 6y ago
    Can you at least expand on why that's your opinion?
    - nsajko 6y ago
      Up-to-dateness, no unnecessary distro-specific patching. EDIT: My comment was ambiguous, I didn't mean that there are no Archlinux-specific patches; rather that there's more of an effort with Archlinux to let upstream be upstream.
      - soraminazuki 6y ago
        No "unnecessary" patching, like seriously? I see so many patches in the Arch packages repo I can't even count them all: https://github.com/archlinux/svntogit-packages/search?q=patch https://github.com/archlinux/svntogit-packages/search?q=patc... https://github.com/archlinux/svntogit-packages/search?q=sed https://github.com/archlinux/svntogit-packages/search?q=sed But putting that aside, all distros need huge amounts of patching to make each package get along with the rest of the system. Without patching, many of them won't even build in the first place.
gjvc 6y ago
ego blog post.
deleted 6y ago
[deleted]
rurban 6y ago
This is old news. Anybody knows (web.archive diff) which paragraphs were changed or added recently?
- secure 6y ago
  Nothing was changed in the content (only presentation). You can see the history of the post at https://github.com/stapelberg/hugo/commits/master/content/posts/2019-03-10-debian-winding-down.markdown https://github.com/stapelberg/hugo/commits/master/content/po... (you can get there by clicking the edit button in the article).
deleted 6y ago
[deleted]

181 points by ecliptik 6y ago

__s 6y ago

(2019) He's now experimenting with a distro focussing on a fast package manager: https://michael.stapelberg.ch/posts/2019-08-17-introducing-distri https://michael.stapelberg.ch/posts/2019-08-17-introducing-d...

Semaphor 6y ago
And here is the original HN discussion, 227 comments: https://news.ycombinator.com/item?id=19353010 https://news.ycombinator.com/item?id=19353010
Ericson2314 6y ago
It seems like he is not aware of Nix or Guix, which is a travesty.
- CameronNemo 6y ago
  He is. Check out the Q&A of his 2020 ArchConf presentation. He explains why NixOS does not fit Hus needs.
  - Ericson2314 6y ago
    Buried after a talk, yay.
    - Ericson2314 6y ago
      I found https://static.conf.archlinux.org/2020/archconf/talks/distri%3A%20researching%20fast%20Linux%20package%20management/6387_distri.mp4 https://static.conf.archlinux.org/2020/archconf/talks/distri... Let me quote a thing starting at 26:49: > The performance improvements distri provides, definitely some of them can apply to Nix. I think there are some low hanging fruit in Nix. The other isuess do reflect some persistent rhetorical issues we've had with explaining Nix: > There was little differentiation with Nix vs NixOS. (Some of his philosophical difference could be resolved by just using Nix.) > There was no recognition that the "nix language part of Nix" is cleanly layered away from the layers that actually do the work of running jobs and moving files around, and can be replaced like Guix does. So I do want our materials to highlight this so experimenters realize Nix is less of an all-in proposition than it sounds (if one is already willing to do the extra work of blazing their own trail).
      - makebackupstoo 6y ago
        A few blog posts on "how to make your own guix!" would be really neat. I'm not as deep as I'd like to be, but roughly it's a simply a matter of outputting a derivation that `nix-daemon/nix-build` can handle, right?
        Ericson2314 6y ago
        I do want to add a section to the Nix manual that describes what it is (data model, key abstractions, etc.). Everything today is some variation on "how to use it".
    - orhmeh09 6y ago
      What are you mad at? You seemed frustrated he was unaware, and now you find that he was aware, you’re mad that the information on his state of awareness was difficult for you to find? Why does it matter to you? Or are you disappointed that his assessment does not match your opinion of Nix et al.?
      - Ericson2314 6y ago
        I'm relieved he is aware. I'm disappointed that the overhead of trying new things here is higher than I think it needs to be, because making these things from scratch means doing lots of not-innovative between the resesrchy bits. I care because there are very few people who care this much about packaging, and if our efforts were less divided it would go a lot further. I'm a little disappointed by what feels like a superficial take on Nix, but I am quite used to that now. And indeed, our documentation is bad about at explaining the essence of the thing.
  - makebackupstoo 6y ago
    I have yet to see Michael discuss Nix without spreading inaccurate, flatly wrong information about it (Nix does not have post-install hooks, and `nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling). Nix fulfills every single requirement Michael has put forward (except the squash>tar thing, which I still don't understand). That's all there is to this. I sympathize with Ericson's frustration. It's exhausting watching people re-invent inferior solutions to Nix, instead of just hopping in and fixing or using Nix. Of course, John Ericson is one of the few people motivated, qualified (and maybe has the buy-in) to make changes in Nix. I'm thankful for that on-going work.
    - st1ck 6y ago
      fast, but more typing: `nix-env -iA nixos.packageName`
    - Ericson2314 6y ago
      > I'm thankful for that on-going work. Thank you!
    - rofrol 6y ago
      >`nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling Can you provide more info on that?
      - Ericson2314 6y ago
        It evaluates the build plan for everything we have, and then searches the things that were evaluated. The new unstable CLI has an evaluation cache at least. What most people do today is look for keys in the object and just evaluate what they need. (The Nix language is lazily evaluated so you can explore like this pretty well out of the box in the repl.) Or, they just grep Nixpkgs :D. ------- All of this, problems and solutions alike, is a weird situation to be in. I still stand by "just get rid of nix-env -i", but I want there to be better solutions too. Nixpkgs seems overly complex, and is in some ways, but the fact its trying to herd a gazillion upstream packages that don't meaningfully coordinate makes this harder to fix than it should be.
- geofft 6y ago
  In https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... he measures NixOS and determines it doesn't solve his problems. More commentary at https://twitter.com/zekjur/status/1162779362682384384 https://twitter.com/zekjur/status/1162779362682384384 (though it sounds like part of it is the fact that the "right" was to install something is nix-env -iA nixpkgs.whatever instead of nix-env -i whatever, which I too find confusing).
  - Ericson2314 6y ago
    NIH hobby projects with good ideas that are doomed to go to waste rub me the wrong way. They are only a temporary reprieve from the burnout felt by for those that author them, anyways. Here's the thing, Nix is to slow (even ignoring `nix-env`'s terrible search functionality which should just be removed). What that requires is some good old boring profiling and optimization work. If he were to contribute that to a distro/package manager that basically shares his vision, this would be much more useful to the world. Cause, at the end of the day, the work isn't so much maintaining the package manager as maintaining the packages. That's simply too much work for anyone to do alone. http://blog.williammanley.net/2020/05/25/unlock-software-freedom-one-by-using-better-tools.html http://blog.williammanley.net/2020/05/25/unlock-software-fre... is good piece on why the ultimate issues with packaging are social, not technological. At this point, when the vast majority of devs don't seem to act as if there is a commons that even needs integration, I don't think any 1-person technological solution is going to be so good as to upend the social situation.
    - basicneo 6y ago
      Even if it were true that a hobby project is just a temporary reprieve from burnout... so what? They're free to do so. And maybe we'll all get something good out of it at the end. The author has a history of delivering quality OSS projects: i3, Debian Code Search, RobustIRC, gokrazy.
    - geofft 6y ago
      I don't think this is intended as a one-person technological solution. It's a one-person research project, trying to see if there are any architectural changes that can be applied to other distros that people actually use, such as Debian or Nix. It's true that the commons needs people willing to put in time and effort on boring things, but they have to be boring in the first place. If the author were to show up and say "Hey, Nix, if you rearchitect in this massive way it may or may not bring big improvements" and sent in a pull request, it would be rightly rejected. But it's still possible that a few days of rearchitecture can deliver the same results as a year of profiling and microoptimizations. The point of distri, as I understand it, is to have something to point to and say, this architectural change will actually work, and it's worth implementing in an actually-used distro.
- deleted 6y ago
  [deleted]
- soraminazuki 6y ago
  It looks like distri have very different design goals from Nix with an emphasis on speed rather than rock solid reproducibility. Although some of the key details do seem to be inspired by Nix, that's no reason to discourage experimentation by calling it a travesty.
arendtio 6y ago
Well, in my opinion the Arch Linux package manager is fast enough, but that might be influenced the years I was using Gentoo (Portage & Paludis) which aren't even part of his benchmark... Edit: For clarification: I am referring to this benchmark https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... For those who are unaware, Portage and Paludis are both magnitudes slower than the other packages managers. However, the comparison is not completely fair, as they both where source based package managers in the beginning, which have to work a bit differently. Nevertheless, Paludis is still a lot slower when being used with binary packages as it doesn't take all the shortcuts the others take.

Foxboron 6y ago

This probably got posted because he reworked the blog theme and fixed the post. This rebumped the post in the RSS feeds. AFAIK nothing new has been added?

ecliptik 6y ago
This is exactly what happened. Saw the post pop up in Newsblur this morning in Planet Debian feed, but I didn't notice the original date since I read it in "reader" view, otherwise I would have tagged it with a 2019. I didn't see it the first time around, and it caught my attention since I've had an interest in becoming a Debian Maintainer for about 15 years now, just never fully did it (even have a GPG key signed by a few Debian Devs) and a lot of the frustrations echoed with me.
- secure 6y ago
  Thanks for clarifying :) I had not realized that my blog template update would result in a repost to various feed consumers. Will try and figure out why that happened. Maybe I can also add the year to the post header when the post is older than X days. Anyway, bringing this post up again is not a bad thing :)

ed25519FUUU 6y ago

> When I joined Debian, I was still studying, i.e. I had luxurious amounts of spare time. I felt so busy in college, what a tragedy! If only I know what it mean later in life. The best part about college for me is I could disappear for the summer, going somewhere and doing what I wanted, knowing that my life would wait for me until Fall.

aerovistae 6y ago

Not strictly related, but I was thinking about this the other day: how long will it be before some group or company creates a truly new operating system that takes off? I mean, the windows kernel is going on 20 years old. the Mac OS kernel is based on unix, which is even older. Linux is also based on unix. As are Android and ios. The more we add to these operating systems, the harder it becomes to walk away from them because we have so much invested in them. Does this mean that in 200 years we'll still be using the descendants of these early operating systems? Under what circumstances would someone decide to start something truly new? And what would it take to ever reach a feature parity with the existing options? And to be clear, I'm not saying there's a reason to walk away from these. I'm not an operating system programmer, I don't even really know that much about it. I'm just wondering if it will always make sense to just keep adding to what we already have.

rstuart4133 6y ago
> how long will it be before some group or company creates a truly new operating system that takes off? You won't get another Windows / Linux / BSD. They do their jobs very well, and as WSL emulating Linux proves they are near interchangeable. I have no doubt Linux could emulate the Windows kernel near perfectly too if we had the Windows source to see what is required. When you've already go a slew of interchangeable parts, we create another one? If something new is to displace them, it's going to have to do something very different. The only thing I can see is not a replacement, but a security kernel that allows us to establish a trust chain from the hardware to some application. It wasn't necessary when the hardware sits on your lap or desk, but now we are tending to rent out CPU cycles from some machine on the other side of the planet and yet still want to have some assurance of privacy, they are becoming kinda essential. There are a number of proprietary ones out there now, but I think that's doomed to fail. No one in the right mind would put their faith in a binary that could be in cahoots with anybody, from the USA government to a ransomware gang.
wilsonthewhale 6y ago
It's more likely for these systems to slowly evolve into something new rather than being replaced entirely. MacOS and Linux, at some level, are already very, very different than earlier unix systems. The BSD's are more conservative, but there's still systems being reworked across all of them (HAMMER in Dragonfly, pledge/unveil in openbsd, etc.). The ideas of unix will probably be with us forever, but the precise details of implementation are transient.

eclipseo76 6y ago

Reading this I'm glad I'm working on Fedora.: one git per package, one tool to rule them all (fedpkg), one tool to sync them all locally (grokmirror), (relatively) easy global changes through proven packagers who can ask for mass changes before each release. I don't know how I would do it if I had to deal with svn, mercurial or no scm at all. I understand the want of being decentralized but this is being done at the maintainers expense, who are often already stretched thing.

ausjke 6y ago
except that debian's apt repo style have been there for decades and still worked well, redhat/fedora's pkg and development model instead had changed who knows how many times, and of course the current one is the golden one, until next one arrives that is.
- Conan_Kudo 6y ago
  The current RPM repository metadata format has existed since Fedora Core 2 in 2004. The build system infrastructure changed to Koji in 2007[1]. The development model has changed exactly once when Fedora switched from CVS to Git in 2009[2]. There have been no significant changes to Fedora packaging model until three years ago, when Modularity was introduced[3] and Pagure was deployed to ease contributions and support building modules[4]. And the modularity concept is primarily used for alternate software streams in Fedora, so the vast majority of Fedora packages don't use this feature. [1]: https://fedoraproject.org/wiki/Releases/FeatureNewBuildSystem https://fedoraproject.org/wiki/Releases/FeatureNewBuildSyste... [2]: https://fedoraproject.org/wiki/Dist_Git_Proposal https://fedoraproject.org/wiki/Dist_Git_Proposal [3]: https://docs.pagure.org/modularity/ https://docs.pagure.org/modularity/ [4]: https://fedoraproject.org/wiki/Changes/ArbitraryBranching https://fedoraproject.org/wiki/Changes/ArbitraryBranching
CameronNemo 6y ago
dnf/RPM is the biggest offender in his benchmarks when it comes to package manager speed. Not only is metadata for a package an order of magnitude larger, the package manager itself works slowly.

deleted 6y ago

[deleted]

gorgoiler 6y ago

OS work used to feel so ceremonial, back in the old days (ie 2003.) Hobbyists could cobble together hardware and ship it to a data center, but if you couldn’t afford a serial console you’d have to unrack the machine every time you messed up a kernel upgrade. Now I can just remotely blast a clean install on bare metal, and then build containers or VMs on that, and it’s so easy I can rebuild all infra every morning, from scratch, just for fun (ahem, to verify it’s idempotency.) Gone is the need for the high quality package management of Debian: The Universal Operating System. I lament this as much as I lament the decline in quality and commoditization of many many other things in life. Food, ISPs, journalism, education.

core-questions 6y ago
> Gone is the need for the high quality package management of Debian: The Universal Operating System. Well, almost. What's inside your container, though? For anything of moderate complexity, my containers always end up having some apt-get or yum installation in them; it's not like I want to Dockerfile up the manual from-source install steps for every small package, nor do I expect to have perfect upstream Dockerfiles to FROM that include all the exact bits I need....
- gorgoiler 6y ago
  Yes, but they are usually single task hosts. All the effort that maintainers put into keeping /etc tidy when multiple pieces of software are installed on the same host are for nought.
  - toyg 6y ago
    So you install exactly one package in each container...? I find it hard to believe. You wouldn’t be able to boot with a single package. Feels very much like saying “all the stuff I don’t (think I) need, is useless”.
- ryukafalz 6y ago
  Not to mention having stuff built for you for all sorts of different architectures. It generally doesn't matter if you're running x86, ARM, whatever if you're installing packages from Debian; by and large everything in the distro is packaged for every architecture and they generally just work.

debiandev 6y ago

A couple of healthy reminders to avoid drama and FUD: 1) People announce leaving publicly and the FLOSS community takes notice of it. This is a sign of health of Debian. In many other projects few people notices. 2) The number of Debian developers, projects, and packages has been increasing for decades. 3) For each person writing on mailing lists and blogs there are 10 people quietly contributing. 4) The same applies to the occasional flamewars. Vocal minorities are not representative of the thousands of DDs and contributors.

zepto 6y ago
This seems like damage control and image management. The piece was cogent, respectful, and constructive. How about addressing his points?
- rstuart4133 6y ago
  I'm a DD. His points left me scratching my head. Yes, Debian's infrastructure is old, but so is Big Ben and just a Big Ben still does an admirable job of broadcasting the time to the locals, Debian Bug Tracker does an admirable job of tracking bugs. I get it that he personally might prefer to interact with it via a more modern web page, but that wouldn't alter how well it tracks bugs or facilitates discussions. And besides, I find email easier to interact with automatically than a web page. Oddly he goes on to list not being able to automate things as a complaint. The same goes for the rest of the things he lists. Yes, he might prefer to do them some other way, but the way they are done now has obviously been working very well for a long time. As for Debian being incapable of making big changes - that's just rubbish. He's been there for 10 years for pete's sake, systemd was a big change requiring many packages to updated spanning several years and several releases, while still delivering a working system as it happened. That's not big? How about altering the source package format, or moving away from sha1 for signing, or making everything build reproducibly, or moving all developers using its collaborative development platform from FusionForge to GitLab? Sorry, he's just plain wrong on the "can't make big changes point". Debian regularly makes big changes every major release. In Bullseye they will have made big strides in migrating away from Python2. Changes of this scale are things other projects regularly struggle with, but not Debian. His posts lists a whole pile of things about Debian he's discovered he no longer likes, which is fair enough. But as he says in his introduction it's him whose that's changed, he gone from a student with lots of time on his hands who was happy to be part of loosely collaborating group to being a member of a very focused and highly directed team at work, and he's discovered he prefers the latter. Great, I get it, happens to all of us. But that doesn't mean the Debian no longer works. It clearly works very well. It just means he's no longer a great fit for that way of doing things.

pelasaco 6y ago

Looking to openSUSE, community that i contribute to, i have the same feeling about almost all points. To keep having bugzilla around, makes me crazy. 2020 and you still not able to delete a comment.

tlamponi 6y ago
> 2020 and you still not able to delete a comment. or a report, no you need to create a bogus product (which is not a single click either), move the report there, and then delete the whole product. It's really a bit a PITA to use, the search is IMO quite good though.
- phendrenad2 6y ago
  Wow, I'd love to know the story behind that. My guesses (A) No one cares enough to add it (B) some core maintainer is actively hostile to the idea (maybe someone deleted their bug report once and they've been getting revenge on the world ever since) or (C) the codebase is too fucked up for anyone to figure out how to add a delete button. But the truth is probably more interesting.
  - jodrellblank 6y ago
    Industry-wide institutionalised hoarding disorder that condemns us all to live atop the ever-expanding junk heap of the past, with the hoarder's justification "might come in useful one day". And that workaround is an accidental bug. A world where big important decisions made in person go completely undocumented, yet weeks of bickering and bikeshedding over nothing are kept forever.

toyg 6y ago

Debian is like democracy: the worst way of producing an OS, except all the others that have been tried from time to time. BeOS, AmigaOS, Solaris, most other 80s OSes - they’re effectively dead. Windows and macOS have effectively died once already. The BSDs can stall for years at times. Most Linux distributions (including RedHat) are typically only as good as the fortunes of the commercial (or occasionally public) entity they have behind. In all this, Debian endures, with its slow but inexorable progress, simply because its ideological foundations - not its technical ones - are eminently superior to all the others. Debian contributors don’t do it for the money, so they will be there when money runs out; and they don’t do it for being cool either, so they will be there when OS work is not cool. People will come and go, but the ideal of the “democratic OS” will always be there - hence, Debian will be too.

smabie 6y ago
Arch Linux is much superior to Debian, imo.
- rovr138 6y ago
  Can you at least expand on why that's your opinion?
  - nsajko 6y ago
    Up-to-dateness, no unnecessary distro-specific patching. EDIT: My comment was ambiguous, I didn't mean that there are no Archlinux-specific patches; rather that there's more of an effort with Archlinux to let upstream be upstream.
    - soraminazuki 6y ago
      No "unnecessary" patching, like seriously? I see so many patches in the Arch packages repo I can't even count them all: https://github.com/archlinux/svntogit-packages/search?q=patch https://github.com/archlinux/svntogit-packages/search?q=patc... https://github.com/archlinux/svntogit-packages/search?q=sed https://github.com/archlinux/svntogit-packages/search?q=sed But putting that aside, all distros need huge amounts of patching to make each package get along with the rest of the system. Without patching, many of them won't even build in the first place.

gjvc 6y ago

ego blog post.

deleted 6y ago

[deleted]

rurban 6y ago

This is old news. Anybody knows (web.archive diff) which paragraphs were changed or added recently?

secure 6y ago
Nothing was changed in the content (only presentation). You can see the history of the post at https://github.com/stapelberg/hugo/commits/master/content/posts/2019-03-10-debian-winding-down.markdown https://github.com/stapelberg/hugo/commits/master/content/po... (you can get there by clicking the edit button in the article).

deleted 6y ago

[deleted]