4 ms·

After a day and a half of usage it became painfully obvious to me that nix is unusable. There are at least four competing ways to use the nix language to config

After a day and a half of usage it became painfully obvious to me that nix is unusable. There are at least four competing ways to use the nix language to configure software. Each of them individually tickles my sense of very neat engineering but together they are a nightmare, with unique shims that must be used between every combination. Much worse, nix does not do any actual package management, i.e., dependency management. There is no way to derive a common-denominator configuration for a dependency shared between two dependent packages, each with their own configuration requirements (a la portage). Nix style is to install the dependency twice. It achieves that wonderfully, which is essential for development environments, but it should permit the shared dependency model for the "base system".

YoshiRulz 4y ago
> There is no way to derive a common-denominator configuration for a dependency shared between two dependent packages, each with their own configuration requirements (a la portage). Nix style is to install the dependency twice. I've never used Portage; what did you mean by this? If you recompiled a library with different flags then you'd expect it to produce a different result. To use the exact same library in multiple packages, change their `buildInputs` by overriding their argument attrsets (`aPkg.override { aLib = aLibModified; }`). To replace it in every consuming package, use an overlay `final: prev: { aLib = prev.aLib.overrideAttrs { ... }; }`.
- puffoflogic 4y ago
  Software which actually manages dependencies should be able to figure out the correct collection of configuration options for the common dependency on its own. It should also be able to detect conflicts between user-requested configuration and dependency requirements (while still allowing an expert user to ignore any stated requirements). Portage does all of this. Nix does not. Nix seems great for deploying VMs and other kinds of managed images, where the cost of manually managing dependencies is amortized. But it is not useful for a development machine. That is a shame, because the other thing that Nix ought to be good at is creating development environments. I wish that Portage's abilities (or any other comparable software; e.g. Cargo does much the same kind of resolution but on a different scale) were explicitly considered and designed into Nix (-the-package-manager), because the features would complement each other incredibly well. Portage's biggest drawback is that, although USE flags are managed declaratively, the system as a whole is, shall we say, aggressively stateful - every problem I've ever had with it is when it fails to find a path from the state it's in to the state I'm requesting.
  - roland-s 4y ago
    I dug into portage a bit and I think I see what you're saying. But the only way Portage can automatically load a dependency is if the ebuild specifies it. For example an ebuild can install dependencies in DEPEND or RDEPEND by checking what USE flags were passed ([1]). You can also have CFLAGs which modify compilation without installing any dependencies. So for example, in portage if I install vim with USE=python, and gnucash with USE=python, then they will both pull in python dependencies based on what was specified in the respective ebuilds (see [2] and [3]), which will probably install the same version of dev-python libs. But that's only because someone packaged vim and gnucash with a USE=python option and specified the required packages for that flag. You can do the same thing in Nix (e.g. you can package software with a bunch of options and pull in required dependencies based on the options selected by the user), but you're right that Nix doesn't have a standardized API for doing this like Portage does. I think it would be a nice thing to have. Maybe it takes a lot of work for maintainers to pre-package every possible build option like they do in Gentoo. Instead, Nix focuses more on providing a unified interface (the Nix language) for end users to modify anything about any package. nixpkgs is also pretty flexible, allowing maintainers to use packaging approaches that aren't "standard practice", and those approaches are all exposed to the end user if they need to make changes. This usually means reading the Nix package source to see what's going on if you want to make changes to a given package. For example some packages like vim do provide a nice interface for users to choose pre-packaged build options [4], similar to USE in Portage. Since Nix is lazy-evaluated, you just have to reference the dependency in the code somewhere (search for ${python3} in the linked example), and then that dependency will only get installed if that bit of code gets executed (e.g. if the pythonSupport option gets passed). I think that's a big part of what makes Nix a pain, the fact that end users can make almost any change they can think of to any attribute on the entire system. But that's also what makes it very powerful. Right now most people use overlays, as people described above. So you can modify the "buildInputs" attribute (the set of build-time dependencies), or the "buildPhase" to pass some different CMake flags, or any install phase you like, for any package in nixpkgs (see [5] for all bajillion phases). Nix flakes might make it easier to compose custom set of packages, but I'm not sure how yet. Oh and one more random tangent - this whole time I've been talking about build dependencies in Nix as opposed to runtime dependencies. That's because runtime dependencies are loaded automatically based on whether they get referenced in the final output of the build. Since every package has a unique hash, Nix literally greps the final build for package hashes, and if it finds any it loads them in as runtime dependencies. This is different from something like Portage where runtime dependencies are explicitly specified in RDEPEND. [1] https://devmanual.gentoo.org/general-concepts/dependencies/#use-conditional-dependencies https://devmanual.gentoo.org/general-concepts/dependencies/#... [2] https://gitweb.gentoo.org/repo/gentoo.git/tree/app-office/gnucash/gnucash-4.12-r1.ebuild#n62 https://gitweb.gentoo.org/repo/gentoo.git/tree/app-office/gn... [3] https://gitweb.gentoo.org/repo/gentoo.git/tree/app-editors/vim/vim-9.0.1000.ebuild#n54 https://gitweb.gentoo.org/repo/gentoo.git/tree/app-editors/v... [4] https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/configurable.nix#L13-L27 https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio... [5] https://nixos.org/manual/nixpkgs/stable/#ssec-controlling-phases https://nixos.org/manual/nixpkgs/stable/#ssec-controlling-ph...
roland-s 4y ago
Nix can be a huge pain to use, and with the recent redesign it somehow feels even more disjointed than before. But it's still one of my favorite pieces of software. I think a clean interface like https://devenv.sh/ https://devenv.sh/ could make all the difference (havent tried it yet but looks promising). That said, if I understand you correctly, I think nix does let you do what you're describing. You can make an overlay, which is a way to modify a set of packages by reaching in and changing build instructions or add packages to the set. So, you can make an overlay of nixpkgs and add a modified libgcc called mylibgcc to the set. In the same overlay, you can modify package A and package B that depend on libgcc, and change their build instructions, including making them use mylibgcc as a build dependency. If you do this for then both packages will use mylibgcc as a single shared dependency without duplicating it. Im on mobile but could try to send a working example if you need one, lmk. Some references: https://nixos.wiki/wiki/Nixpkgs/Modifying_Packages https://nixos.wiki/wiki/Nixpkgs/Modifying_Packages https://nixos.wiki/wiki/Overlays https://nixos.wiki/wiki/Overlays

YoshiRulz 4y ago

> There is no way to derive a common-denominator configuration for a dependency shared between two dependent packages, each with their own configuration requirements (a la portage). Nix style is to install the dependency twice. I've never used Portage; what did you mean by this? If you recompiled a library with different flags then you'd expect it to produce a different result. To use the exact same library in multiple packages, change their `buildInputs` by overriding their argument attrsets (`aPkg.override { aLib = aLibModified; }`). To replace it in every consuming package, use an overlay `final: prev: { aLib = prev.aLib.overrideAttrs { ... }; }`.

puffoflogic 4y ago
Software which actually manages dependencies should be able to figure out the correct collection of configuration options for the common dependency on its own. It should also be able to detect conflicts between user-requested configuration and dependency requirements (while still allowing an expert user to ignore any stated requirements). Portage does all of this. Nix does not. Nix seems great for deploying VMs and other kinds of managed images, where the cost of manually managing dependencies is amortized. But it is not useful for a development machine. That is a shame, because the other thing that Nix ought to be good at is creating development environments. I wish that Portage's abilities (or any other comparable software; e.g. Cargo does much the same kind of resolution but on a different scale) were explicitly considered and designed into Nix (-the-package-manager), because the features would complement each other incredibly well. Portage's biggest drawback is that, although USE flags are managed declaratively, the system as a whole is, shall we say, aggressively stateful - every problem I've ever had with it is when it fails to find a path from the state it's in to the state I'm requesting.
- roland-s 4y ago
  I dug into portage a bit and I think I see what you're saying. But the only way Portage can automatically load a dependency is if the ebuild specifies it. For example an ebuild can install dependencies in DEPEND or RDEPEND by checking what USE flags were passed ([1]). You can also have CFLAGs which modify compilation without installing any dependencies. So for example, in portage if I install vim with USE=python, and gnucash with USE=python, then they will both pull in python dependencies based on what was specified in the respective ebuilds (see [2] and [3]), which will probably install the same version of dev-python libs. But that's only because someone packaged vim and gnucash with a USE=python option and specified the required packages for that flag. You can do the same thing in Nix (e.g. you can package software with a bunch of options and pull in required dependencies based on the options selected by the user), but you're right that Nix doesn't have a standardized API for doing this like Portage does. I think it would be a nice thing to have. Maybe it takes a lot of work for maintainers to pre-package every possible build option like they do in Gentoo. Instead, Nix focuses more on providing a unified interface (the Nix language) for end users to modify anything about any package. nixpkgs is also pretty flexible, allowing maintainers to use packaging approaches that aren't "standard practice", and those approaches are all exposed to the end user if they need to make changes. This usually means reading the Nix package source to see what's going on if you want to make changes to a given package. For example some packages like vim do provide a nice interface for users to choose pre-packaged build options [4], similar to USE in Portage. Since Nix is lazy-evaluated, you just have to reference the dependency in the code somewhere (search for ${python3} in the linked example), and then that dependency will only get installed if that bit of code gets executed (e.g. if the pythonSupport option gets passed). I think that's a big part of what makes Nix a pain, the fact that end users can make almost any change they can think of to any attribute on the entire system. But that's also what makes it very powerful. Right now most people use overlays, as people described above. So you can modify the "buildInputs" attribute (the set of build-time dependencies), or the "buildPhase" to pass some different CMake flags, or any install phase you like, for any package in nixpkgs (see [5] for all bajillion phases). Nix flakes might make it easier to compose custom set of packages, but I'm not sure how yet. Oh and one more random tangent - this whole time I've been talking about build dependencies in Nix as opposed to runtime dependencies. That's because runtime dependencies are loaded automatically based on whether they get referenced in the final output of the build. Since every package has a unique hash, Nix literally greps the final build for package hashes, and if it finds any it loads them in as runtime dependencies. This is different from something like Portage where runtime dependencies are explicitly specified in RDEPEND. [1] https://devmanual.gentoo.org/general-concepts/dependencies/#use-conditional-dependencies https://devmanual.gentoo.org/general-concepts/dependencies/#... [2] https://gitweb.gentoo.org/repo/gentoo.git/tree/app-office/gnucash/gnucash-4.12-r1.ebuild#n62 https://gitweb.gentoo.org/repo/gentoo.git/tree/app-office/gn... [3] https://gitweb.gentoo.org/repo/gentoo.git/tree/app-editors/vim/vim-9.0.1000.ebuild#n54 https://gitweb.gentoo.org/repo/gentoo.git/tree/app-editors/v... [4] https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/configurable.nix#L13-L27 https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio... [5] https://nixos.org/manual/nixpkgs/stable/#ssec-controlling-phases https://nixos.org/manual/nixpkgs/stable/#ssec-controlling-ph...

roland-s 4y ago

Nix can be a huge pain to use, and with the recent redesign it somehow feels even more disjointed than before. But it's still one of my favorite pieces of software. I think a clean interface like https://devenv.sh/ https://devenv.sh/ could make all the difference (havent tried it yet but looks promising). That said, if I understand you correctly, I think nix does let you do what you're describing. You can make an overlay, which is a way to modify a set of packages by reaching in and changing build instructions or add packages to the set. So, you can make an overlay of nixpkgs and add a modified libgcc called mylibgcc to the set. In the same overlay, you can modify package A and package B that depend on libgcc, and change their build instructions, including making them use mylibgcc as a build dependency. If you do this for then both packages will use mylibgcc as a single shared dependency without duplicating it. Im on mobile but could try to send a working example if you need one, lmk. Some references: https://nixos.wiki/wiki/Nixpkgs/Modifying_Packages https://nixos.wiki/wiki/Nixpkgs/Modifying_Packages https://nixos.wiki/wiki/Overlays https://nixos.wiki/wiki/Overlays