5 ms·

I've been using Nix OS since this year and am mostly happy with it, but if I had to name my largest issue with it then it's that updates are so traffic intensiv

by est31 6y ago

I've been using Nix OS since this year and am mostly happy with it, but if I had to name my largest issue with it then it's that updates are so traffic intensive. Basically, glibc is linked dynamically on most distros, including Nix OS. But if there is a glibc update that only changes the .so file, it still updates all packages depending on glibc. A more expansive list of my issues with nix (with some curse words :p): https://gist.github.com/est31/005f17628d55fda27b401d3c994233a7 https://gist.github.com/est31/005f17628d55fda27b401d3c994233...

arianvanp 6y ago
I'm confused with your first point? > No concept of source packages. There is only a binary cache (hydra) and the derivations in nixpkgs. NixOS _Only_ has a concept of source packages. The cache is a (sound) optimisation om top of it. just disable the binary cache and you can build all packages from source.
- the_duke 6y ago
  Nix packages don't contain any source code. The package definition describes how to fetch the source code from a source (like a Git repo or a hosted archive) and build it. The built result only contains what is necessary at runtime. A sizeable amount of packages don't even fetch source code but a prebuilt binary which is then fixed up to work with Nix. There is a source cache, but it is optional. As an example, check out ripgrep [1]. It uses `fetchFromGithub` to retrieve the code. [1] https://github.com/NixOS/nixpkgs/blob/3bb54189b0c8132752fff320a3bf441118119a5e/pkgs/tools/text/ripgrep-all/default.nix#L9 https://github.com/NixOS/nixpkgs/blob/3bb54189b0c8132752fff3...
  - catern 6y ago
    Note that the source code is referenced by a hash, so it can't change without changing the package. Also, the source code of all packages built by Hydra is on cache.nixos.org alongside the resulting binaries.
    - the_duke 6y ago
      True, which is absolutely sufficient for most use cases. I'm currently doing some work for ML and data science companies where full reproducibility and introspection is very much desired. So you need to run your own source cache to provide that guarantee, because you can't count on cache.nixos.org still providing the source code from a package built 4 years ago. But that's why I love the IPFS cache efforts. [1] Running your own node to pin all required sources should then be relatively easy. [1] https://blog.ipfs.io/2020-09-08-nix-ipfs-milestone-1/ https://blog.ipfs.io/2020-09-08-nix-ipfs-milestone-1/
      - ryukafalz 6y ago
        Software Heritage is also helpful here, and Guix is integrating with it - see e.g. https://guix.gnu.org/blog/2019/connecting-reproducible-deployment-to-a-long-term-source-code-archive/ https://guix.gnu.org/blog/2019/connecting-reproducible-deplo...
        danieldk 6y ago
        Nix is also integrating with the Software Heritage: https://www.tweag.io/blog/2020-06-18-software-heritage/ https://www.tweag.io/blog/2020-06-18-software-heritage/
      - est31 6y ago
        Debian's archive reaches back 15 years now: https://snapshot.debian.org/ https://snapshot.debian.org/ It also contains source code.
    - est31 6y ago
      Are those sources target independent or specific for each new build of the package? That is, is there a new source code package on hydra when one of the dependencies changes? Or does it only change if the package itself changes? Also, they are only available when hydra builds the package anyways, right? So if some package is not built by hydra (like how it used to be for the texlive packages), it'll still download the sources from the various places they are hosted. As for the hash, it's good that the source code is hashed, but my main concern was that it was downloading from external sources in the first place. This is bad for privacy, as those hosts know I'm downloading from them, as well as for reliability, because the hosts might not have as good uptime as a debian package mirror.
  - ris 6y ago
    Binary packages make me sad too and I wish there were a way to mark them as such using a `meta` key. It starts to become a bit of a grey area in some cases though. For instance - java packages. Is a .jar a binary? Probably. But so many java applications rely on pulling loads of .jars down from maven. Are we going to sit down and figure out how to build all those jars from source? It's not uncommon for there to be literally hundreds.
    - catern 6y ago
      I find that binary packages are quite rare for free/open source software. So avoiding proprietary software (with, e.g., allowUnfree = false) gets most of the way there. I agree that it would be nice to tag (with meta) FOSS packages that aren't built from source, though. Every instance of that is a bug, IMO...
      - soraminazuki 6y ago
        > I find that binary packages are quite rare for free/open source software. Unfortunately this isn’t the case with some languages that have their own package managers, the prime example being Java as the parent commenter mentioned. It’s near impossible to build Java applications without fetching tons of binary jars from maven that Debian just gives up on providing their own package in many cases[1]. While Nix does build Java applications from source, the dependencies are fetched from maven in binary form. [1]: https://wiki.debian.org/Hadoop https://wiki.debian.org/Hadoop

by est31 6y ago

arianvanp 6y ago

I'm confused with your first point? > No concept of source packages. There is only a binary cache (hydra) and the derivations in nixpkgs. NixOS _Only_ has a concept of source packages. The cache is a (sound) optimisation om top of it. just disable the binary cache and you can build all packages from source.

the_duke 6y ago
Nix packages don't contain any source code. The package definition describes how to fetch the source code from a source (like a Git repo or a hosted archive) and build it. The built result only contains what is necessary at runtime. A sizeable amount of packages don't even fetch source code but a prebuilt binary which is then fixed up to work with Nix. There is a source cache, but it is optional. As an example, check out ripgrep [1]. It uses `fetchFromGithub` to retrieve the code. [1] https://github.com/NixOS/nixpkgs/blob/3bb54189b0c8132752fff320a3bf441118119a5e/pkgs/tools/text/ripgrep-all/default.nix#L9 https://github.com/NixOS/nixpkgs/blob/3bb54189b0c8132752fff3...
- catern 6y ago
  Note that the source code is referenced by a hash, so it can't change without changing the package. Also, the source code of all packages built by Hydra is on cache.nixos.org alongside the resulting binaries.
  - the_duke 6y ago
    True, which is absolutely sufficient for most use cases. I'm currently doing some work for ML and data science companies where full reproducibility and introspection is very much desired. So you need to run your own source cache to provide that guarantee, because you can't count on cache.nixos.org still providing the source code from a package built 4 years ago. But that's why I love the IPFS cache efforts. [1] Running your own node to pin all required sources should then be relatively easy. [1] https://blog.ipfs.io/2020-09-08-nix-ipfs-milestone-1/ https://blog.ipfs.io/2020-09-08-nix-ipfs-milestone-1/
    - ryukafalz 6y ago
      Software Heritage is also helpful here, and Guix is integrating with it - see e.g. https://guix.gnu.org/blog/2019/connecting-reproducible-deployment-to-a-long-term-source-code-archive/ https://guix.gnu.org/blog/2019/connecting-reproducible-deplo...
      - danieldk 6y ago
        Nix is also integrating with the Software Heritage: https://www.tweag.io/blog/2020-06-18-software-heritage/ https://www.tweag.io/blog/2020-06-18-software-heritage/
    - est31 6y ago
      Debian's archive reaches back 15 years now: https://snapshot.debian.org/ https://snapshot.debian.org/ It also contains source code.
  - est31 6y ago
    Are those sources target independent or specific for each new build of the package? That is, is there a new source code package on hydra when one of the dependencies changes? Or does it only change if the package itself changes? Also, they are only available when hydra builds the package anyways, right? So if some package is not built by hydra (like how it used to be for the texlive packages), it'll still download the sources from the various places they are hosted. As for the hash, it's good that the source code is hashed, but my main concern was that it was downloading from external sources in the first place. This is bad for privacy, as those hosts know I'm downloading from them, as well as for reliability, because the hosts might not have as good uptime as a debian package mirror.
- ris 6y ago
  Binary packages make me sad too and I wish there were a way to mark them as such using a `meta` key. It starts to become a bit of a grey area in some cases though. For instance - java packages. Is a .jar a binary? Probably. But so many java applications rely on pulling loads of .jars down from maven. Are we going to sit down and figure out how to build all those jars from source? It's not uncommon for there to be literally hundreds.
  - catern 6y ago
    I find that binary packages are quite rare for free/open source software. So avoiding proprietary software (with, e.g., allowUnfree = false) gets most of the way there. I agree that it would be nice to tag (with meta) FOSS packages that aren't built from source, though. Every instance of that is a bug, IMO...
    - soraminazuki 6y ago
      > I find that binary packages are quite rare for free/open source software. Unfortunately this isn’t the case with some languages that have their own package managers, the prime example being Java as the parent commenter mentioned. It’s near impossible to build Java applications without fetching tons of binary jars from maven that Debian just gives up on providing their own package in many cases[1]. While Nix does build Java applications from source, the dependencies are fetched from maven in binary form. [1]: https://wiki.debian.org/Hadoop https://wiki.debian.org/Hadoop