5 ms·

(2019) He's now experimenting with a distro focussing on a fast package manager: https://michael.stapelberg.ch/posts/2019-08-17-introducing-distri https://mich

by __s 6y ago

(2019) He's now experimenting with a distro focussing on a fast package manager: https://michael.stapelberg.ch/posts/2019-08-17-introducing-distri https://michael.stapelberg.ch/posts/2019-08-17-introducing-d...

Semaphor 6y ago
And here is the original HN discussion, 227 comments: https://news.ycombinator.com/item?id=19353010 https://news.ycombinator.com/item?id=19353010
Ericson2314 6y ago
It seems like he is not aware of Nix or Guix, which is a travesty.
- CameronNemo 6y ago
  He is. Check out the Q&A of his 2020 ArchConf presentation. He explains why NixOS does not fit Hus needs.
  - Ericson2314 6y ago
    Buried after a talk, yay.
    - Ericson2314 6y ago
      I found https://static.conf.archlinux.org/2020/archconf/talks/distri%3A%20researching%20fast%20Linux%20package%20management/6387_distri.mp4 https://static.conf.archlinux.org/2020/archconf/talks/distri... Let me quote a thing starting at 26:49: > The performance improvements distri provides, definitely some of them can apply to Nix. I think there are some low hanging fruit in Nix. The other isuess do reflect some persistent rhetorical issues we've had with explaining Nix: > There was little differentiation with Nix vs NixOS. (Some of his philosophical difference could be resolved by just using Nix.) > There was no recognition that the "nix language part of Nix" is cleanly layered away from the layers that actually do the work of running jobs and moving files around, and can be replaced like Guix does. So I do want our materials to highlight this so experimenters realize Nix is less of an all-in proposition than it sounds (if one is already willing to do the extra work of blazing their own trail).
      - makebackupstoo 6y ago
        A few blog posts on "how to make your own guix!" would be really neat. I'm not as deep as I'd like to be, but roughly it's a simply a matter of outputting a derivation that `nix-daemon/nix-build` can handle, right?
        Ericson2314 6y ago
        I do want to add a section to the Nix manual that describes what it is (data model, key abstractions, etc.). Everything today is some variation on "how to use it".
    - orhmeh09 6y ago
      What are you mad at? You seemed frustrated he was unaware, and now you find that he was aware, you’re mad that the information on his state of awareness was difficult for you to find? Why does it matter to you? Or are you disappointed that his assessment does not match your opinion of Nix et al.?
      - Ericson2314 6y ago
        I'm relieved he is aware. I'm disappointed that the overhead of trying new things here is higher than I think it needs to be, because making these things from scratch means doing lots of not-innovative between the resesrchy bits. I care because there are very few people who care this much about packaging, and if our efforts were less divided it would go a lot further. I'm a little disappointed by what feels like a superficial take on Nix, but I am quite used to that now. And indeed, our documentation is bad about at explaining the essence of the thing.
  - makebackupstoo 6y ago
    I have yet to see Michael discuss Nix without spreading inaccurate, flatly wrong information about it (Nix does not have post-install hooks, and `nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling). Nix fulfills every single requirement Michael has put forward (except the squash>tar thing, which I still don't understand). That's all there is to this. I sympathize with Ericson's frustration. It's exhausting watching people re-invent inferior solutions to Nix, instead of just hopping in and fixing or using Nix. Of course, John Ericson is one of the few people motivated, qualified (and maybe has the buy-in) to make changes in Nix. I'm thankful for that on-going work.
    - st1ck 6y ago
      fast, but more typing: `nix-env -iA nixos.packageName`
    - Ericson2314 6y ago
      > I'm thankful for that on-going work. Thank you!
    - rofrol 6y ago
      >`nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling Can you provide more info on that?
      - Ericson2314 6y ago
        It evaluates the build plan for everything we have, and then searches the things that were evaluated. The new unstable CLI has an evaluation cache at least. What most people do today is look for keys in the object and just evaluate what they need. (The Nix language is lazily evaluated so you can explore like this pretty well out of the box in the repl.) Or, they just grep Nixpkgs :D. ------- All of this, problems and solutions alike, is a weird situation to be in. I still stand by "just get rid of nix-env -i", but I want there to be better solutions too. Nixpkgs seems overly complex, and is in some ways, but the fact its trying to herd a gazillion upstream packages that don't meaningfully coordinate makes this harder to fix than it should be.
- geofft 6y ago
  In https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... he measures NixOS and determines it doesn't solve his problems. More commentary at https://twitter.com/zekjur/status/1162779362682384384 https://twitter.com/zekjur/status/1162779362682384384 (though it sounds like part of it is the fact that the "right" was to install something is nix-env -iA nixpkgs.whatever instead of nix-env -i whatever, which I too find confusing).
  - Ericson2314 6y ago
    NIH hobby projects with good ideas that are doomed to go to waste rub me the wrong way. They are only a temporary reprieve from the burnout felt by for those that author them, anyways. Here's the thing, Nix is to slow (even ignoring `nix-env`'s terrible search functionality which should just be removed). What that requires is some good old boring profiling and optimization work. If he were to contribute that to a distro/package manager that basically shares his vision, this would be much more useful to the world. Cause, at the end of the day, the work isn't so much maintaining the package manager as maintaining the packages. That's simply too much work for anyone to do alone. http://blog.williammanley.net/2020/05/25/unlock-software-freedom-one-by-using-better-tools.html http://blog.williammanley.net/2020/05/25/unlock-software-fre... is good piece on why the ultimate issues with packaging are social, not technological. At this point, when the vast majority of devs don't seem to act as if there is a commons that even needs integration, I don't think any 1-person technological solution is going to be so good as to upend the social situation.
    - basicneo 6y ago
      Even if it were true that a hobby project is just a temporary reprieve from burnout... so what? They're free to do so. And maybe we'll all get something good out of it at the end. The author has a history of delivering quality OSS projects: i3, Debian Code Search, RobustIRC, gokrazy.
    - geofft 6y ago
      I don't think this is intended as a one-person technological solution. It's a one-person research project, trying to see if there are any architectural changes that can be applied to other distros that people actually use, such as Debian or Nix. It's true that the commons needs people willing to put in time and effort on boring things, but they have to be boring in the first place. If the author were to show up and say "Hey, Nix, if you rearchitect in this massive way it may or may not bring big improvements" and sent in a pull request, it would be rightly rejected. But it's still possible that a few days of rearchitecture can deliver the same results as a year of profiling and microoptimizations. The point of distri, as I understand it, is to have something to point to and say, this architectural change will actually work, and it's worth implementing in an actually-used distro.
- deleted 6y ago
  [deleted]
- soraminazuki 6y ago
  It looks like distri have very different design goals from Nix with an emphasis on speed rather than rock solid reproducibility. Although some of the key details do seem to be inspired by Nix, that's no reason to discourage experimentation by calling it a travesty.
arendtio 6y ago
Well, in my opinion the Arch Linux package manager is fast enough, but that might be influenced the years I was using Gentoo (Portage & Paludis) which aren't even part of his benchmark... Edit: For clarification: I am referring to this benchmark https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... For those who are unaware, Portage and Paludis are both magnitudes slower than the other packages managers. However, the comparison is not completely fair, as they both where source based package managers in the beginning, which have to work a bit differently. Nevertheless, Paludis is still a lot slower when being used with binary packages as it doesn't take all the shortcuts the others take.

by __s 6y ago

Semaphor 6y ago

And here is the original HN discussion, 227 comments: https://news.ycombinator.com/item?id=19353010 https://news.ycombinator.com/item?id=19353010

Ericson2314 6y ago

It seems like he is not aware of Nix or Guix, which is a travesty.

CameronNemo 6y ago
He is. Check out the Q&A of his 2020 ArchConf presentation. He explains why NixOS does not fit Hus needs.
- Ericson2314 6y ago
  Buried after a talk, yay.
  - Ericson2314 6y ago
    I found https://static.conf.archlinux.org/2020/archconf/talks/distri%3A%20researching%20fast%20Linux%20package%20management/6387_distri.mp4 https://static.conf.archlinux.org/2020/archconf/talks/distri... Let me quote a thing starting at 26:49: > The performance improvements distri provides, definitely some of them can apply to Nix. I think there are some low hanging fruit in Nix. The other isuess do reflect some persistent rhetorical issues we've had with explaining Nix: > There was little differentiation with Nix vs NixOS. (Some of his philosophical difference could be resolved by just using Nix.) > There was no recognition that the "nix language part of Nix" is cleanly layered away from the layers that actually do the work of running jobs and moving files around, and can be replaced like Guix does. So I do want our materials to highlight this so experimenters realize Nix is less of an all-in proposition than it sounds (if one is already willing to do the extra work of blazing their own trail).
    - makebackupstoo 6y ago
      A few blog posts on "how to make your own guix!" would be really neat. I'm not as deep as I'd like to be, but roughly it's a simply a matter of outputting a derivation that `nix-daemon/nix-build` can handle, right?
      - Ericson2314 6y ago
        I do want to add a section to the Nix manual that describes what it is (data model, key abstractions, etc.). Everything today is some variation on "how to use it".
  - orhmeh09 6y ago
    What are you mad at? You seemed frustrated he was unaware, and now you find that he was aware, you’re mad that the information on his state of awareness was difficult for you to find? Why does it matter to you? Or are you disappointed that his assessment does not match your opinion of Nix et al.?
    - Ericson2314 6y ago
      I'm relieved he is aware. I'm disappointed that the overhead of trying new things here is higher than I think it needs to be, because making these things from scratch means doing lots of not-innovative between the resesrchy bits. I care because there are very few people who care this much about packaging, and if our efforts were less divided it would go a lot further. I'm a little disappointed by what feels like a superficial take on Nix, but I am quite used to that now. And indeed, our documentation is bad about at explaining the essence of the thing.
- makebackupstoo 6y ago
  I have yet to see Michael discuss Nix without spreading inaccurate, flatly wrong information about it (Nix does not have post-install hooks, and `nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling). Nix fulfills every single requirement Michael has put forward (except the squash>tar thing, which I still don't understand). That's all there is to this. I sympathize with Ericson's frustration. It's exhausting watching people re-invent inferior solutions to Nix, instead of just hopping in and fixing or using Nix. Of course, John Ericson is one of the few people motivated, qualified (and maybe has the buy-in) to make changes in Nix. I'm thankful for that on-going work.
  - st1ck 6y ago
    fast, but more typing: `nix-env -iA nixos.packageName`
  - Ericson2314 6y ago
    > I'm thankful for that on-going work. Thank you!
  - rofrol 6y ago
    >`nix-env -i` being slow and unrecommended is a known meme that is addressed in upcoming tooling Can you provide more info on that?
    - Ericson2314 6y ago
      It evaluates the build plan for everything we have, and then searches the things that were evaluated. The new unstable CLI has an evaluation cache at least. What most people do today is look for keys in the object and just evaluate what they need. (The Nix language is lazily evaluated so you can explore like this pretty well out of the box in the repl.) Or, they just grep Nixpkgs :D. ------- All of this, problems and solutions alike, is a weird situation to be in. I still stand by "just get rid of nix-env -i", but I want there to be better solutions too. Nixpkgs seems overly complex, and is in some ways, but the fact its trying to herd a gazillion upstream packages that don't meaningfully coordinate makes this harder to fix than it should be.
geofft 6y ago
In https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... he measures NixOS and determines it doesn't solve his problems. More commentary at https://twitter.com/zekjur/status/1162779362682384384 https://twitter.com/zekjur/status/1162779362682384384 (though it sounds like part of it is the fact that the "right" was to install something is nix-env -iA nixpkgs.whatever instead of nix-env -i whatever, which I too find confusing).
- Ericson2314 6y ago
  NIH hobby projects with good ideas that are doomed to go to waste rub me the wrong way. They are only a temporary reprieve from the burnout felt by for those that author them, anyways. Here's the thing, Nix is to slow (even ignoring `nix-env`'s terrible search functionality which should just be removed). What that requires is some good old boring profiling and optimization work. If he were to contribute that to a distro/package manager that basically shares his vision, this would be much more useful to the world. Cause, at the end of the day, the work isn't so much maintaining the package manager as maintaining the packages. That's simply too much work for anyone to do alone. http://blog.williammanley.net/2020/05/25/unlock-software-freedom-one-by-using-better-tools.html http://blog.williammanley.net/2020/05/25/unlock-software-fre... is good piece on why the ultimate issues with packaging are social, not technological. At this point, when the vast majority of devs don't seem to act as if there is a commons that even needs integration, I don't think any 1-person technological solution is going to be so good as to upend the social situation.
  - basicneo 6y ago
    Even if it were true that a hobby project is just a temporary reprieve from burnout... so what? They're free to do so. And maybe we'll all get something good out of it at the end. The author has a history of delivering quality OSS projects: i3, Debian Code Search, RobustIRC, gokrazy.
  - geofft 6y ago
    I don't think this is intended as a one-person technological solution. It's a one-person research project, trying to see if there are any architectural changes that can be applied to other distros that people actually use, such as Debian or Nix. It's true that the commons needs people willing to put in time and effort on boring things, but they have to be boring in the first place. If the author were to show up and say "Hey, Nix, if you rearchitect in this massive way it may or may not bring big improvements" and sent in a pull request, it would be rightly rejected. But it's still possible that a few days of rearchitecture can deliver the same results as a year of profiling and microoptimizations. The point of distri, as I understand it, is to have something to point to and say, this architectural change will actually work, and it's worth implementing in an actually-used distro.
deleted 6y ago
[deleted]
soraminazuki 6y ago
It looks like distri have very different design goals from Nix with an emphasis on speed rather than rock solid reproducibility. Although some of the key details do seem to be inspired by Nix, that's no reason to discourage experimentation by calling it a travesty.

arendtio 6y ago

Well, in my opinion the Arch Linux package manager is fast enough, but that might be influenced the years I was using Gentoo (Portage & Paludis) which aren't even part of his benchmark... Edit: For clarification: I am referring to this benchmark https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/ https://michael.stapelberg.ch/posts/2019-08-17-linux-package... For those who are unaware, Portage and Paludis are both magnitudes slower than the other packages managers. However, the comparison is not completely fair, as they both where source based package managers in the beginning, which have to work a bit differently. Nevertheless, Paludis is still a lot slower when being used with binary packages as it doesn't take all the shortcuts the others take.