4 ms·

I am frustrated that you ignored my main argument about Docker sufficiently serving the same business function as Nix, with lower maintenance. Instead you focus

I am frustrated that you ignored my main argument about Docker sufficiently serving the same business function as Nix, with lower maintenance. Instead you focused on a semantic argument about one word in my post - "reproducible". You have the wrong idea about reproducibility, where you say basically everything that is not Nix or Guix is not reproducible. This ignores things like conda and techniques like package version pinning that allow researchers and businesspeople to get the same results from the same code. Here's a definition of "reproducible" from Wikipedia: Any results should be documented by making all data and code available in such a way that the computations can be executed again with identical results. https://en.wikipedia.org/wiki/Reproducibility https://en.wikipedia.org/wiki/Reproducibility ----- > To be clear, are you suggesting that > RUN sudo apt-get update && sudo apt-get -y install ... No, you need to pin the dependency versions. With Python this practice is already normalized with requirements.txt or conda yml files. So you would take an existing project and do: RUN conda env create -f environment.yml which would likely be copy-pasted from the project README. The yml file specifies version numbers for dependencies. The SAT solver is deterministic. For other languages like C maybe the project didn't specify dependency version. So you need to figure them out when you first get a successful build, then specify their versions in apt. You can specify version numbers in the apt-get install line. Yes, this is reproducible. Definitely good enough for most business use cases. When I say reproducible I do not mean ivory tower math proof reproducible. I just mean that the code will run on the relevant machines they are targeting. As I wrote in my initial comment. And as I defined at the top of this comment. Also Nix provides a worse experience for pinning dependency versions since it does not have a native concept of version numbers [0]. Instead people have to grep through the Nixpkgs repo to find the correct hash of their dependency version. > This is even more true for Nix, which has the largest and most up-to-date package repositories out there No, Docker has the closure (to borrow Nix's terminology) of all of the package managers in that graph. If you add the height of the Debian packages with Pypi and Hackage you already have Nix beat. You can keep adding - cargo, ruby gems, etc all in their native package managers. If Nix were better off then people would be adapting Nix packages to other ecosystems. But the reality is the other way around. > Plus, with Nix, you can easily make a new package based on existing packages with a mere few lines of code if the existing packages doesn't fit your needs. Other package managers besides Guix doesn't offer you that flexibility so you'd have to compile from scratch With Nix, you are forced to make new packages based on existing packages. That is not a benefit. Regarding "if the existing packages doesn't fit your needs", compiling from source is not a big deal since Docker caches output artifacts. [0]: https://github.com/NixOS/nixpkgs/issues/93327 https://github.com/NixOS/nixpkgs/issues/93327

soraminazuki 5y ago
> I am frustrated that you ignored my main argument Your main argument was that Docker "sufficiently" serves the same goal of reproducibility. I just pointed out how it doesn't come anywhere close. Addressing the core of an argument is far from a "semantic" argument. > where you say basically everything that is not Nix or Guix is not reproducible My definition of reproducibility is that you get identical build results every time, which should meet the definition you quoted from Wikipedia. "docker build," which runs arbitrary shell commands with network access, is the farthest thing possible from any sane definition of "reproducible." > Also Nix provides a worse experience for pinning dependency versions The exact opposite is true. No other system-level package managers like apt or yum truly supports pinning packages. With apt or yum, packages in a repository snapshot are tightly coupled together since they're all installed into a single shared location. It's not possible to swap out or pin a subset of packages without the risk of breakage. Nix provides a truly working way to pin packages. Packages are installed into its own isolated location to avoid collisions and dependencies are explicitly specified. This makes it possible to mix packages from stable channels, unstable channels, and even specific git commits of those channels. This can't be done with apt or yum. Language-level package managers are somewhat more flexible regarding pinning, but still has problems. More on that next. > No, you need to pin the dependency versions. With Python this practice is already normalized with requirements.txt or conda yml files. Yet CI builds constantly break because Python dependencies are a moving target. And no, the SAT solver doesn't make the builds deterministic. The fact that you even need a SAT solver just makes it clear that dependency management is getting out of hand and we need better tools. > If you add the height of the Debian packages with Pypi and Hackage you already have Nix beat. ... If Nix were better off then people would be adapting Nix packages to other ecosystems. I don't know why you believe Nix and only Nix has to compete with all other package managers combined. You must really dislike it if you can convince yourself that is fair comparison. But Nix can be used along with other package managers, so I don't see the point here. The only anomaly here are Docker images, that monolith binary blob that doesn't compose well like packages in other package managers. And speaking of fairness > or a maintainer does it with a much smaller code review Where did you get this idea from? Nix has a growing community and Nixpkgs is one of the most active repositories on GitHub. Other package repositories with the possible exception of Homebrew and AUR has a much higher barrier to entry, which would most definitely result in, "smaller code review." > Nix project people commit directly to master frequently and do self-merges of PRs Self-merges are nowhere near being unique to Nixpkgs so it's unfair to only call Nixpkgs out for it. And if you count language-specific package repositories like NPM or PyPI, you should assume there is zero code review for most packages. While regrettably there are self-merges in Nixpkgs, it is definitely in the minority and a lot of those changes are especially trivial stuff. Since Nixpkgs has a vibrant community, things like this tend to get attention and some community members are keeping an eye on it and is quick to bring these instances up. It's also worth noting that the Nix community is especially invested in automated testing compared to other package managers and these are run on PRs that ends up being self-merged. > With Nix, you are forced to make new packages based on existing packages. That is 100% FUD. > "if the existing packages doesn't fit your needs", compiling from source is not a big deal since Docker caches output artifacts. It is a big deal that you can't reuse code with non-Nix package managers. Docker caching isn't relevant and does nothing to deal with maintainability or reproducibility issues. Our company maintains custom OpenSSL RPMs, and it has been a constant source of pain due to RPM's lack of code reusability. Now we also have to maintain our own version of every single package that relies on our build of OpenSSL, which is a nightmare. This wouldn't have been a problem with Nix.

soraminazuki 5y ago

> I am frustrated that you ignored my main argument Your main argument was that Docker "sufficiently" serves the same goal of reproducibility. I just pointed out how it doesn't come anywhere close. Addressing the core of an argument is far from a "semantic" argument. > where you say basically everything that is not Nix or Guix is not reproducible My definition of reproducibility is that you get identical build results every time, which should meet the definition you quoted from Wikipedia. "docker build," which runs arbitrary shell commands with network access, is the farthest thing possible from any sane definition of "reproducible." > Also Nix provides a worse experience for pinning dependency versions The exact opposite is true. No other system-level package managers like apt or yum truly supports pinning packages. With apt or yum, packages in a repository snapshot are tightly coupled together since they're all installed into a single shared location. It's not possible to swap out or pin a subset of packages without the risk of breakage. Nix provides a truly working way to pin packages. Packages are installed into its own isolated location to avoid collisions and dependencies are explicitly specified. This makes it possible to mix packages from stable channels, unstable channels, and even specific git commits of those channels. This can't be done with apt or yum. Language-level package managers are somewhat more flexible regarding pinning, but still has problems. More on that next. > No, you need to pin the dependency versions. With Python this practice is already normalized with requirements.txt or conda yml files. Yet CI builds constantly break because Python dependencies are a moving target. And no, the SAT solver doesn't make the builds deterministic. The fact that you even need a SAT solver just makes it clear that dependency management is getting out of hand and we need better tools. > If you add the height of the Debian packages with Pypi and Hackage you already have Nix beat. ... If Nix were better off then people would be adapting Nix packages to other ecosystems. I don't know why you believe Nix and only Nix has to compete with all other package managers combined. You must really dislike it if you can convince yourself that is fair comparison. But Nix can be used along with other package managers, so I don't see the point here. The only anomaly here are Docker images, that monolith binary blob that doesn't compose well like packages in other package managers. And speaking of fairness > or a maintainer does it with a much smaller code review Where did you get this idea from? Nix has a growing community and Nixpkgs is one of the most active repositories on GitHub. Other package repositories with the possible exception of Homebrew and AUR has a much higher barrier to entry, which would most definitely result in, "smaller code review." > Nix project people commit directly to master frequently and do self-merges of PRs Self-merges are nowhere near being unique to Nixpkgs so it's unfair to only call Nixpkgs out for it. And if you count language-specific package repositories like NPM or PyPI, you should assume there is zero code review for most packages. While regrettably there are self-merges in Nixpkgs, it is definitely in the minority and a lot of those changes are especially trivial stuff. Since Nixpkgs has a vibrant community, things like this tend to get attention and some community members are keeping an eye on it and is quick to bring these instances up. It's also worth noting that the Nix community is especially invested in automated testing compared to other package managers and these are run on PRs that ends up being self-merged. > With Nix, you are forced to make new packages based on existing packages. That is 100% FUD. > "if the existing packages doesn't fit your needs", compiling from source is not a big deal since Docker caches output artifacts. It is a big deal that you can't reuse code with non-Nix package managers. Docker caching isn't relevant and does nothing to deal with maintainability or reproducibility issues. Our company maintains custom OpenSSL RPMs, and it has been a constant source of pain due to RPM's lack of code reusability. Now we also have to maintain our own version of every single package that relies on our build of OpenSSL, which is a nightmare. This wouldn't have been a problem with Nix.