5 ms·

So ... this is more or less exactly how Haiku's package management system works. (It was designed in ~2008-2009, merged into the nightlies in 2013, and of cours

So ... this is more or less exactly how Haiku's package management system works. (It was designed in ~2008-2009, merged into the nightlies in 2013, and of course has been used by default since then.) We use different terminology for a number of these things, but all the concepts are almost identical. Is this really the case of two completely separate creations of the same idea? Or is this another instance of the Linux world NIH'ing something a different project had for a while without even mentioning it?

secure 7y ago
I learnt about the Haiku package management system when I had already developed distri for several months. I do actually reference Haiku, but in the referenced post https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/#appendix-a-related-work https://michael.stapelberg.ch/posts/2019-08-17-linux-package..., not in the distri introduction post itself. I think it’s very telling that the two approaches look so similar, so I was really happy to learn about the similarities with Haiku!
- waddlesplash 7y ago
  "HaikuDepot" is just the name for the GUI interface. The package manager itself has no real "name." The number of similarities being so high, I am still skeptical you really did not come across Haiku before, but well... On a different note, some key things we have discovered in working with this system for almost a decade now are: (1) One really needs a dedicated kernel module for this. Managing the abstraction in userland only gets too tedious after a while, especially around updates and the like. Moving mount management to a kernel "packagefs" makes things so much simpler (and of course more performant). Not to mention that you can then write a dedicated file format which supports random access better. (2) Perhaps the most powerful feature of this system, the ability to boot into a previous state, is virtually impossible without kernel support. (3) You will be surprised at how many users want to change things inside packages; and how many Linux users refuse to switch to a system that does not allow this. On Haiku we have mechanisms for overriding packaged files and "blacklisting" files; but on Linux you may find people are far too wired into the "old way" to get a radical change like this off the ground. (4) The second most powerful feature, the ability to install packages in ~ (or, eventually, anywhere else for that matter) sounds great, but it requires a massive amount of software patching, and for applications to use APIs (that Linux does not have) to iterate through paths rather than hard-coding them. So if you are going to make such a big, systemic change, why not just ditch Linux for Haiku instead of re-creating what we have spent so long making work already?
  - secure 7y ago
    > Managing the abstraction in userland only gets too tedious after a while Can you elaborate on what you found tedious? Thus far, I feel like that part works reasonably well. > Perhaps the most powerful feature of this system, the ability to boot into a previous state, is virtually impossible without kernel support Interesting. Why is resetting the package store contents to an earlier state not sufficient? What does the kernel need to do? Or do you mean to avoid a reboot? > You will be surprised at how many users want to change things inside packages Hah, yeah. I’m one of these users myself. In distri, you can just rebuild any package and forcefully install the result onto your system, if you really want to and can live with the consequences. Often, this means you need a reboot right after. I usually iterate on a package by starting its programs from the outside, or by using qemu which has pretty quick boot times. > The second most powerful feature, the ability to install packages in ~ Yeah, distri doesn’t attempt to do this, for the reasons you outline :) > why not just ditch Linux for Haiku Copy&pasting my reply to you on twitter for the others: I tried Haiku multiple times, but it’s just too different than what I’m used to. Honestly, Linux is niche enough for my preference. If I was switching OS, I would probably look for more mainstream, not less :)
    - waddlesplash 7y ago
      > Can you elaborate on what you found tedious? Thus far, I feel like that part works reasonably well. Most of it revolves around update states, i.e. if I update 100 packages at once, the number of kernel calls to de-activate the previous 100 and activate the new 100 is probably in the tens of thousands, and there is a lot of inefficiency here because the kernel does not really know "what" you are doing (just unmounting, moving files, remounting, adding links, etc. etc. etc.) Whereas on Haiku with packagefs, the package_daemon gives the packagefs the new state description, and it can update all of its internal states and re-bind everything internally with a lot less overhead, because both ends of the system know exactly what is going on. > Why is resetting the package store contents to an earlier state not sufficient? What does the kernel need to do? Or do you mean to avoid a reboot? I mean if you e.g. take a bad update, and now your system does not boot, how are you going to revert to an older one? All distros today store older kernels so if that was the problem it's easy to revert that, but if the problem was some package, you have to pull up a recovery shell and mess around in it. If you manage all of the package mounting in userland, how are you going to handle booting from an older "state" if the package management tools themselves, which must be outside the packaged area in your model, got broken? So then you still have to drop into a recovery shell. On Haiku, you just get into the bootloader menu, and then you choose to boot from the previous "state" (which includes the kernel, init system, etc. etc.) because the bootloader itself speaks "packagefs", and can load the kernel out of a package. > In distri, you can just rebuild any package and forcefully install the result onto your system, if you really want to and can live with the consequences. But you miss the point: Of course you can do the same on Haiku, or use the "non-packaged" directories, etc. We have convinced the Haiku "power users" to go this route. But it is very ingrained into a lot of people, and getting them to switch to it is a large cultural problem. Especially if the package manager itself feels "bolted on" and not part of the system, which it virtually always will be in Linux. > Yeah, distri doesn’t attempt to do this, for the reasons you outline :) Well, then the "Linux desktop usability" people who complain that Linux is unsuitable for $average_user will remain correct...

secure 7y ago

I learnt about the Haiku package management system when I had already developed distri for several months. I do actually reference Haiku, but in the referenced post https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/#appendix-a-related-work https://michael.stapelberg.ch/posts/2019-08-17-linux-package..., not in the distri introduction post itself. I think it’s very telling that the two approaches look so similar, so I was really happy to learn about the similarities with Haiku!

waddlesplash 7y ago
"HaikuDepot" is just the name for the GUI interface. The package manager itself has no real "name." The number of similarities being so high, I am still skeptical you really did not come across Haiku before, but well... On a different note, some key things we have discovered in working with this system for almost a decade now are: (1) One really needs a dedicated kernel module for this. Managing the abstraction in userland only gets too tedious after a while, especially around updates and the like. Moving mount management to a kernel "packagefs" makes things so much simpler (and of course more performant). Not to mention that you can then write a dedicated file format which supports random access better. (2) Perhaps the most powerful feature of this system, the ability to boot into a previous state, is virtually impossible without kernel support. (3) You will be surprised at how many users want to change things inside packages; and how many Linux users refuse to switch to a system that does not allow this. On Haiku we have mechanisms for overriding packaged files and "blacklisting" files; but on Linux you may find people are far too wired into the "old way" to get a radical change like this off the ground. (4) The second most powerful feature, the ability to install packages in ~ (or, eventually, anywhere else for that matter) sounds great, but it requires a massive amount of software patching, and for applications to use APIs (that Linux does not have) to iterate through paths rather than hard-coding them. So if you are going to make such a big, systemic change, why not just ditch Linux for Haiku instead of re-creating what we have spent so long making work already?
- secure 7y ago
  > Managing the abstraction in userland only gets too tedious after a while Can you elaborate on what you found tedious? Thus far, I feel like that part works reasonably well. > Perhaps the most powerful feature of this system, the ability to boot into a previous state, is virtually impossible without kernel support Interesting. Why is resetting the package store contents to an earlier state not sufficient? What does the kernel need to do? Or do you mean to avoid a reboot? > You will be surprised at how many users want to change things inside packages Hah, yeah. I’m one of these users myself. In distri, you can just rebuild any package and forcefully install the result onto your system, if you really want to and can live with the consequences. Often, this means you need a reboot right after. I usually iterate on a package by starting its programs from the outside, or by using qemu which has pretty quick boot times. > The second most powerful feature, the ability to install packages in ~ Yeah, distri doesn’t attempt to do this, for the reasons you outline :) > why not just ditch Linux for Haiku Copy&pasting my reply to you on twitter for the others: I tried Haiku multiple times, but it’s just too different than what I’m used to. Honestly, Linux is niche enough for my preference. If I was switching OS, I would probably look for more mainstream, not less :)
  - waddlesplash 7y ago
    > Can you elaborate on what you found tedious? Thus far, I feel like that part works reasonably well. Most of it revolves around update states, i.e. if I update 100 packages at once, the number of kernel calls to de-activate the previous 100 and activate the new 100 is probably in the tens of thousands, and there is a lot of inefficiency here because the kernel does not really know "what" you are doing (just unmounting, moving files, remounting, adding links, etc. etc. etc.) Whereas on Haiku with packagefs, the package_daemon gives the packagefs the new state description, and it can update all of its internal states and re-bind everything internally with a lot less overhead, because both ends of the system know exactly what is going on. > Why is resetting the package store contents to an earlier state not sufficient? What does the kernel need to do? Or do you mean to avoid a reboot? I mean if you e.g. take a bad update, and now your system does not boot, how are you going to revert to an older one? All distros today store older kernels so if that was the problem it's easy to revert that, but if the problem was some package, you have to pull up a recovery shell and mess around in it. If you manage all of the package mounting in userland, how are you going to handle booting from an older "state" if the package management tools themselves, which must be outside the packaged area in your model, got broken? So then you still have to drop into a recovery shell. On Haiku, you just get into the bootloader menu, and then you choose to boot from the previous "state" (which includes the kernel, init system, etc. etc.) because the bootloader itself speaks "packagefs", and can load the kernel out of a package. > In distri, you can just rebuild any package and forcefully install the result onto your system, if you really want to and can live with the consequences. But you miss the point: Of course you can do the same on Haiku, or use the "non-packaged" directories, etc. We have convinced the Haiku "power users" to go this route. But it is very ingrained into a lot of people, and getting them to switch to it is a large cultural problem. Especially if the package manager itself feels "bolted on" and not part of the system, which it virtually always will be in Linux. > Yeah, distri doesn’t attempt to do this, for the reasons you outline :) Well, then the "Linux desktop usability" people who complain that Linux is unsuitable for $average_user will remain correct...