4 ms·

I would love to see a Linux distribution picking up on the concept of NixOS, with its content-addressed-package store and then use overlayfs and bubblewrap to c

by chme 2y ago

I would love to see a Linux distribution picking up on the concept of NixOS, with its content-addressed-package store and then use overlayfs and bubblewrap to create process local FHS compatible root file systems with only the dependencies of each process inside.

bjackman 2y ago
They don't do the special package manager thing but aside from that I think the "immutable distros" like Fedora Silverblue are in this vein no?
- chme 2y ago
  I would consider Fedora Silverblue/CoreOS, NixOS and even flatpak as a sort of hacky way to implement that. I don't know of any PoC for this. I would guess that there might be some research budget available for that approach.
sham1 2y ago
Well, both Guix System and NixOS can do this, where they construct a containerised environment with FHS layout. IIRC neither uses overlayfs and they do the namespaces by themselves instead of relying on Bubblewrap, but inside the container you get exactly what you'd expect. At least in Guix it's `guix shell -CF`, where `-C` is of course the `--container` flag, and `-F` for `--emulate-fhs`. I'd imagine that `nix-shell` works the same way, but I'm more familiar with guix than nix, so I can only speak for that.
- chme 2y ago
  I only ever tried NixOS, but the FHS stuff there seems to only be about getting Steam and other proprietary software to work. I would suggest to have this per default, so that the only application that actually sees the real file system would be PID1.
  - sham1 2y ago
    That is an interesting idea, but I don't really see what the purpose would really be. As you said, having these kinds of FHS containers really is only actually useful for proprietary software, with free software being adaptable for a "non-standard" filesystem. Of course, it's sometimes helpful to do this even for free software, which is of course why this is an option, but I do struggle to see what would be the idea with making it the default for all non-PID1 apps in such a system. All I could really reckon is that it might be easier for user familiarity, but at least in my experience, one gets used to the new layout pretty quickly. You'd ideally be configuring the whole system in either Nix language or Scheme anyway, so things like `/etc` are a bit superfluous, and since you can get `/bin/sh` and `/usr/bin/env` symlinked for things that expect it, most things should work anyway. Well, unless done poorly, but nothing one couldn't patch.
    - chme 2y ago
      In NixOS you have only one central package store `/nix/store`, so all packages need to be installed there. This is necessary, because the software is patched to use that path. IMO patching it this way is a bit hacky, when there a better options. If overlayfs is used, the software would not require patching, so you could have multiple package stores if you like. Maybe a package store in `~/.nix/store` as well. This would allow with a immutable rootfs (e.g. immutable `/nix/store`) to still install packages per user, or even per project a different package store.
INTPenis 2y ago
I understood some of those words. But I've been using Atomic Fedora for the last 2 years and my whole work flow has shifted to using containers for everything CLI, I can have specific container images for work on qemu images, or terraform for example. And everything else like Steam games, VLC, Firefox all run in Flatpaks. So the end goal is to not make any modifications to the host image.
- chme 2y ago
  > So the end goal is to not make any modifications to the host image. IMO the goal should not be to have a immutable host image, but to have a immutable rootfs of each process. Immutable host base images might contain libraries or software that is not required by the application, they are more difficult to update, because they need to be updated as a unit, and probably require a reboot. To make this stable, you need to hack around with A/B partitions for a fallback. IMO this would not be necessary if we had every package just installed under its checksum under `/pkgs` or what ever, an then use overlayfs to create a file system customized for every process. All that is required here to revert back is the old 'profile' and then start the old packages from the `/pkgs`. Like NixOS does it. However NixOS doesn't use overlayfs, they use symlinks and patching of the paths in the sources to archive it. Using overlayfs would also allow changing the package path (`/pkgs` here, or nixos package store `/nix/store`) at runtime, because the software would not require patches, so people could install an additional store in their home directory for instance.
  - INTPenis 2y ago
    Sounds like you're leaning more towards the Qubes OS model, even though I'm sure you don't want to go that extreme. Either way I'm very happy with Atomic Linux, it's a huge improvement. I'm sure there will be more improvements that will offer separate rootfs for each process. Isn't that already handled by running flatpaks in their own container root?
    - chme 2y ago
      > Sounds like you're leaning more towards the Qubes OS model, even though I'm sure you don't want to go that extreme. Qubes OS is full virtualization. I don't want to boot a separate kernel for every process I want to start. That is not very useful. > Either way I'm very happy with Atomic Linux, it's a huge improvement. I'm sure there will be more improvements that will offer separate rootfs for each process. Isn't that already handled by running flatpaks in their own container root? As the article here says, not all apps can be used as flatpak. Currently flatpak is mostly for GUI desktop apps, most common cli tools are missing. For instance there is no gcc flatpak. Also the layers under a flatpak are not separated by individual packages but by broader runtime environments, thus they contain more stuff than each individual application requires. I want a system that could even be deployed on a small embedded device, because it is so generic.

by chme 2y ago

bjackman 2y ago

They don't do the special package manager thing but aside from that I think the "immutable distros" like Fedora Silverblue are in this vein no?

chme 2y ago
I would consider Fedora Silverblue/CoreOS, NixOS and even flatpak as a sort of hacky way to implement that. I don't know of any PoC for this. I would guess that there might be some research budget available for that approach.

sham1 2y ago

Well, both Guix System and NixOS can do this, where they construct a containerised environment with FHS layout. IIRC neither uses overlayfs and they do the namespaces by themselves instead of relying on Bubblewrap, but inside the container you get exactly what you'd expect. At least in Guix it's `guix shell -CF`, where `-C` is of course the `--container` flag, and `-F` for `--emulate-fhs`. I'd imagine that `nix-shell` works the same way, but I'm more familiar with guix than nix, so I can only speak for that.

chme 2y ago
I only ever tried NixOS, but the FHS stuff there seems to only be about getting Steam and other proprietary software to work. I would suggest to have this per default, so that the only application that actually sees the real file system would be PID1.
- sham1 2y ago
  That is an interesting idea, but I don't really see what the purpose would really be. As you said, having these kinds of FHS containers really is only actually useful for proprietary software, with free software being adaptable for a "non-standard" filesystem. Of course, it's sometimes helpful to do this even for free software, which is of course why this is an option, but I do struggle to see what would be the idea with making it the default for all non-PID1 apps in such a system. All I could really reckon is that it might be easier for user familiarity, but at least in my experience, one gets used to the new layout pretty quickly. You'd ideally be configuring the whole system in either Nix language or Scheme anyway, so things like `/etc` are a bit superfluous, and since you can get `/bin/sh` and `/usr/bin/env` symlinked for things that expect it, most things should work anyway. Well, unless done poorly, but nothing one couldn't patch.
  - chme 2y ago
    In NixOS you have only one central package store `/nix/store`, so all packages need to be installed there. This is necessary, because the software is patched to use that path. IMO patching it this way is a bit hacky, when there a better options. If overlayfs is used, the software would not require patching, so you could have multiple package stores if you like. Maybe a package store in `~/.nix/store` as well. This would allow with a immutable rootfs (e.g. immutable `/nix/store`) to still install packages per user, or even per project a different package store.

INTPenis 2y ago

I understood some of those words. But I've been using Atomic Fedora for the last 2 years and my whole work flow has shifted to using containers for everything CLI, I can have specific container images for work on qemu images, or terraform for example. And everything else like Steam games, VLC, Firefox all run in Flatpaks. So the end goal is to not make any modifications to the host image.

chme 2y ago
> So the end goal is to not make any modifications to the host image. IMO the goal should not be to have a immutable host image, but to have a immutable rootfs of each process. Immutable host base images might contain libraries or software that is not required by the application, they are more difficult to update, because they need to be updated as a unit, and probably require a reboot. To make this stable, you need to hack around with A/B partitions for a fallback. IMO this would not be necessary if we had every package just installed under its checksum under `/pkgs` or what ever, an then use overlayfs to create a file system customized for every process. All that is required here to revert back is the old 'profile' and then start the old packages from the `/pkgs`. Like NixOS does it. However NixOS doesn't use overlayfs, they use symlinks and patching of the paths in the sources to archive it. Using overlayfs would also allow changing the package path (`/pkgs` here, or nixos package store `/nix/store`) at runtime, because the software would not require patches, so people could install an additional store in their home directory for instance.
- INTPenis 2y ago
  Sounds like you're leaning more towards the Qubes OS model, even though I'm sure you don't want to go that extreme. Either way I'm very happy with Atomic Linux, it's a huge improvement. I'm sure there will be more improvements that will offer separate rootfs for each process. Isn't that already handled by running flatpaks in their own container root?
  - chme 2y ago
    > Sounds like you're leaning more towards the Qubes OS model, even though I'm sure you don't want to go that extreme. Qubes OS is full virtualization. I don't want to boot a separate kernel for every process I want to start. That is not very useful. > Either way I'm very happy with Atomic Linux, it's a huge improvement. I'm sure there will be more improvements that will offer separate rootfs for each process. Isn't that already handled by running flatpaks in their own container root? As the article here says, not all apps can be used as flatpak. Currently flatpak is mostly for GUI desktop apps, most common cli tools are missing. For instance there is no gcc flatpak. Also the layers under a flatpak are not separated by individual packages but by broader runtime environments, thus they contain more stuff than each individual application requires. I want a system that could even be deployed on a small embedded device, because it is so generic.