3 ms·

I like the principle of Nix that one can simultaneously install different versions of the same software and make layered choices of what version to use with wha

I like the principle of Nix that one can simultaneously install different versions of the same software and make layered choices of what version to use with what or depending on the use case. Nix has spearheaded that principle and that's great. That being said, that fine-grained layering selection is done via symlinks in Nix afaik, whereas a couple newer packaging systems (e.g. OCI containers or flatpak) can do such layering with newer stuff like bind mounts and namespaces+sandboxing (and I don't just mean sandbox for build time but for run time) and thus increase the security by selectively choosing what a package is supposed to have access to. I wonder how fast Nix will adapt to such new possibilities. I think it should do so quickly (e.g. switch to OCI as the underlying layering system; I hear that the Tvix project is experimenting with that?), as that could establish Nix as the dominant system/distribution in that field whereas otherwise it would be overtaken and left behind by whatever OCI-container-based distribution manages to come out as the dominant one. There is currently (temporarily) a unique window of opportunity in that: * Docker is totally ruining their position in the OCI world, and had never really put effort into building a comprehensive quality curated distribution. That is: their registry may be "comprehensive" as in large choice, but apart from a small set of base images, it's mostly a hotchpotch of low-quality uncurated images with uncertain security… and often found to be of severely lacking in the security domain. * Redhat has a much too closed policy for their OCI registries and has made the mistake of restricting their OCI stuff to the server side while fedora pushes flatpak/flathub which is too restricted to the desktop. That artificial chasm between a server-only and a desktop-only system sucks. * Ubuntu has completely borked their attempts at new sandboxed/layered package formats, snap sucks. And Debian and the other remaining big distros have nothing in that category Nix has the advantage of already having a large, comprehensive and curated set of packages. All it needs is to adopt OCI as its underlying layering system (instead of symlinks), make its large package base trivially accessible to OCI, and make an effort on UX (a little more accessible and easier) and it could come out as the dominant distribution.

anon291 4y ago
No. The nix store is a graph. Oci images are a tree. This is terrible for code ruse. Nix is superior. Oci should adopt a nix based approach.
- EntrePrescott 4y ago
  The choice of topology of the package and layering system - be it a tree as in OCI or a more general graph topology as in Nix - is only a very small part of either of these systems. I agree that the general graph topology is superior in some points. My point though was that at the implementational level, the old symlink-based way of implementing it in Nix is severely lacking the isolation and more general security capabilities of the bind-mount, namespaces and cgroup-based approach of OCI and other newer packaging systems. And Nix needs to implement that. My impression is that the OCI package spec isn't what would be in the way of implementing a system that would combine the isolation and security of a system based on bind-mount, namespaces and cgroups like OCI with a graph topology like Nix and that there would thus be an opportunity to combine the two, which would help Nix take over the dominating position in that space. If OCI turned out to be impossible to use with the more graph-based approach of Nix, then that would mean a much higher implementational work - not that it couldn't be done, but it still would need to be done then. But either way, Nix cannot stay with its old symlink-based layering: Failing to implement the security features that are now expected from modern packaging systems (isolation, bind mounts, cgroups, namespaces etc) is a surefire way to progressively maneuver into irrelevance. Nix has a window of opportunity here due to the current weakness of the big players in the field. But it can't afford to let that slip.
  - tadfisher 4y ago
    Bind-mount, namespaces and cgroups are runtime properties, completely orthogonal to the problem Nix (the build system/package manager) is meant to solve. NixOS is the configuration layer on top that encodes those properties via systemd units, and that is where you'll find the concept of a "service" and where you can isolate the runtime parts of the system.
    - vilunov 4y ago
      It's not orthogonal, Nix goes far out of its way to fix compositionality issues with hacks and dubious conventions, when instead namespaces (bind-mounts) could be used to deliver expected environments.
vilunov 4y ago
OCI is based on Linux namespaces, which provide a way to create isolated filesystem trees for processes. You probably want that, and I agree with you, but probably forcing this into OCI itself is a lost cause, since its tooling is too based on layers.
midchildan 4y ago
Treating packaging boundaries and runtime isolation as the same thing is exactly the problem with Docker and similar solutions. Just because some package didn't require another package at build time doesn't mean we don't ever want to use them together at runtime. Yet Docker conflates the two, introducing all sorts of unnecessary friction all over the place. This is why something as simple as getting more than one process to work with each other on Docker is such an overcomplicated mess. The runtime isolation boundary set by Docker doesn't represent any sort of logical component or security boundary in your system. It merely reflects how the underlying image was built. This is a classic anti-pattern of mixing up policy with implementation. Runtime isolation policy should be independent of build time implementation. Nix gets this right with better design and composable packages. It's trivial to create a container that includes only the packages you want, with dependencies handled automatically by Nix. Docker, on the other hand, leaves you with a binary blob (i.e., Docker image) that's neither composable nor customizable.
- vilunov 4y ago
  > Just because some package didn't require another package at build time doesn't mean we don't ever want to use them together at runtime. Yet Docker conflates the two, introducing all sorts of unnecessary friction all over the place. But Docker in fact doesn't have compiletime dependencies, it needs you to specify runtime deps only. If you want to build something in Docker, you use two-stage builds and the runtime deps of the first stage become your compiletime deps. > This is why something as simple as getting more than one process to work with each other on Docker is such an overcomplicated mess I don't get this, why is it considered an overcomplicated mess? If you want to run several processes in one container, you just launch it with a lightweight process manager, if you want to run it in separate containers -- well, that's even easier, just launch separate containers and configure the communication between them with a network. > Nix gets this right with better design and composable packages Nix actually implements it worse than Docker in some sense. Particularly, the exact problem that you described: > Just because some package didn't require another package at build time doesn't mean we don't ever want to use them together at runtime is not solved in Nix, runtime deps must be a subset of compiletime deps. > that's neither composable nor customizable Compositionality is a completely different issue on which I do have problems with Docker. DAG-oriented environment building is strictly better than inheritance-oriented, but that's all orthogonal with compiletime-runtime separation.

anon291 4y ago

No. The nix store is a graph. Oci images are a tree. This is terrible for code ruse. Nix is superior. Oci should adopt a nix based approach.

EntrePrescott 4y ago
The choice of topology of the package and layering system - be it a tree as in OCI or a more general graph topology as in Nix - is only a very small part of either of these systems. I agree that the general graph topology is superior in some points. My point though was that at the implementational level, the old symlink-based way of implementing it in Nix is severely lacking the isolation and more general security capabilities of the bind-mount, namespaces and cgroup-based approach of OCI and other newer packaging systems. And Nix needs to implement that. My impression is that the OCI package spec isn't what would be in the way of implementing a system that would combine the isolation and security of a system based on bind-mount, namespaces and cgroups like OCI with a graph topology like Nix and that there would thus be an opportunity to combine the two, which would help Nix take over the dominating position in that space. If OCI turned out to be impossible to use with the more graph-based approach of Nix, then that would mean a much higher implementational work - not that it couldn't be done, but it still would need to be done then. But either way, Nix cannot stay with its old symlink-based layering: Failing to implement the security features that are now expected from modern packaging systems (isolation, bind mounts, cgroups, namespaces etc) is a surefire way to progressively maneuver into irrelevance. Nix has a window of opportunity here due to the current weakness of the big players in the field. But it can't afford to let that slip.
- tadfisher 4y ago
  Bind-mount, namespaces and cgroups are runtime properties, completely orthogonal to the problem Nix (the build system/package manager) is meant to solve. NixOS is the configuration layer on top that encodes those properties via systemd units, and that is where you'll find the concept of a "service" and where you can isolate the runtime parts of the system.
  - vilunov 4y ago
    It's not orthogonal, Nix goes far out of its way to fix compositionality issues with hacks and dubious conventions, when instead namespaces (bind-mounts) could be used to deliver expected environments.

vilunov 4y ago

OCI is based on Linux namespaces, which provide a way to create isolated filesystem trees for processes. You probably want that, and I agree with you, but probably forcing this into OCI itself is a lost cause, since its tooling is too based on layers.

midchildan 4y ago

Treating packaging boundaries and runtime isolation as the same thing is exactly the problem with Docker and similar solutions. Just because some package didn't require another package at build time doesn't mean we don't ever want to use them together at runtime. Yet Docker conflates the two, introducing all sorts of unnecessary friction all over the place. This is why something as simple as getting more than one process to work with each other on Docker is such an overcomplicated mess. The runtime isolation boundary set by Docker doesn't represent any sort of logical component or security boundary in your system. It merely reflects how the underlying image was built. This is a classic anti-pattern of mixing up policy with implementation. Runtime isolation policy should be independent of build time implementation. Nix gets this right with better design and composable packages. It's trivial to create a container that includes only the packages you want, with dependencies handled automatically by Nix. Docker, on the other hand, leaves you with a binary blob (i.e., Docker image) that's neither composable nor customizable.

vilunov 4y ago
> Just because some package didn't require another package at build time doesn't mean we don't ever want to use them together at runtime. Yet Docker conflates the two, introducing all sorts of unnecessary friction all over the place. But Docker in fact doesn't have compiletime dependencies, it needs you to specify runtime deps only. If you want to build something in Docker, you use two-stage builds and the runtime deps of the first stage become your compiletime deps. > This is why something as simple as getting more than one process to work with each other on Docker is such an overcomplicated mess I don't get this, why is it considered an overcomplicated mess? If you want to run several processes in one container, you just launch it with a lightweight process manager, if you want to run it in separate containers -- well, that's even easier, just launch separate containers and configure the communication between them with a network. > Nix gets this right with better design and composable packages Nix actually implements it worse than Docker in some sense. Particularly, the exact problem that you described: > Just because some package didn't require another package at build time doesn't mean we don't ever want to use them together at runtime is not solved in Nix, runtime deps must be a subset of compiletime deps. > that's neither composable nor customizable Compositionality is a completely different issue on which I do have problems with Docker. DAG-oriented environment building is strictly better than inheritance-oriented, but that's all orthogonal with compiletime-runtime separation.