2 ms·
> never appreciated how much package management is intertwined with individual ecosystems' cultures until I tried using a "universal" package manager like Nix.
by kaba0 5y ago
> never appreciated how much package management is intertwined with individual ecosystems' cultures until I tried using a "universal" package manager like Nix.
Unfortunately it is not a problem for nix to solve, they can’t really mandate upstream projects to not include hardcoded executables.
Also, the problem with npm/pip etc is that they are themselves package managers more or less trying to solve the same thing. So I would even wager that without some language-specific help nix can’t really work seamlessly across other dependency managers. Also, it is probably not even the best idea to replicate npm’s every dep inside nixpkgs (there are third-party nix repos).
Also, as an example Haskell has it sort of solved with a specific program that can translate a haskell specific dependency specification (analogous to package.json) to a .nix build file. There may very well be something similar for npm as well.
- earksiinni 5y ago> Unfortunately it is not a problem for nix to solve, they can’t really mandate upstream projects to not include hardcoded executables. Yet it's normal for Linux distros to patch upstream so that it works well inside of the distro. You may say "but that's crazy! Downstream patching Node packages opens up a whole can of worms!" Maybe you would be right--or maybe not. My point is that when I would raise these kinds of issues, I was met with a collective shrug or various justifications for why brokenness is actually a good thing. That's what I call a kind of academic/purist mentality that's really great for a research project and not practical for a product. If you ship a Linux distro (and note, I'm talking here about NixOS, not just Nix) that comes with software in the repo, the universal expectation is that the software works as intended, and if it doesn't that it's a bug. There can be an argument over whether the bug is upstream or downstream, but the fault is recognized. That recognition doesn't seem to happen in Nix world in cases where the package comes from an ecosystem that has assumptions that contradict Nix's (or NixOS') assumptions. > So I would even wager that without some language-specific help nix can’t really work seamlessly across other dependency managers. The thing is, Nix provides various helper functions to aid in writing derivations for packages from the different ecosystems. They actually try to adapt to the various idioms, which is admirable. My gripe is that each set of adaptations for each different ecosystem has its own entirely unique API. There's no meta-framework or set of common practices for writing makeNodePackage vs. makePythonPackage vs. makeHaskellPackage even though in many cases they share similar challenges (e.g., X and Y ecosystem's packages import dependencies from hardwired relative paths). It's a question of abstraction; makeXYZPackage exists, but there's no supra-ecosystem abstraction for writing it. That sounds like an obscure concern until you try using Nix as your daily driver for dev work. As long as you stick to one ecosystem, it's passable. Still a bad UX because nothing is properly documented and really bad anti-patterns (such as forcing package maintainers to update hundreds of thousands of packages to update a single package) are championed, but at least you only need to learn one ecosystem's quirks and idiosyncrasies. The situation changes very quickly as soon as you try to mesh NPM packages with Python with Ruby, etc. I actually raised this concern a while ago with the Nix devs, and their response was basically that it would take too much effort to harmonize all the different approaches. I thought that was a valid response, but it's unfortunate since I think it's the project's Achilles' heel.
- kaba0 5y agoThanks for expanding on it! I feel the all-platform abstraction would be very difficult and leaky but at least some minor convention would indeed be great. Though I’m not sure how perfect the nix language is for very high-level abstractions —- I think the documentation problem is also in part due to not having static types.
- squiggleblaz 5y ago>Also, the problem with npm/pip etc is that they are themselves package managers more or less trying to solve the same thing. The problem is that they're trying to solve different problems. NPM/pip etc are mostly about letting the developer of an internal application choose and acquire dependencies for their project. Nixpkgs/dpkg etc are about letting the developers of a system acquire and maintain dependencies for a whole system. These two goals pull in different directions. NPM/pip want to have available every released version of every package, so that when I can keep using legacy-package with a well-tested version of web-dependency. This way, I can focus my attention on the important matters of writing new code for contemporary-project. I can be confident that legacy-package will keep running the same way it used to, security bugs and all, until I make a decision to attend to it. Nixpkgs/dpkg want to have available as many packages as possible with a target that every api incompatible library is included, so that any two programs which use the same api use the same version of a package. This way, if they need to patch a well-tested but insecure version of web-dependency, all programs which use it will be kept up-to-date. Because nix is an agnostic build tool, it can run in both modes simultaneously. You can create a nix expression that uses nixpkgs to build a system that provides a runtime environment and a system that provides a buildtime environment and then uses internal project dependency managers to fulfil the lockfile and installs the product of that, node_modules and all, onto the runtime system. You will lose certain guarantees by doing it this way, and perhaps some parts will require awkward, but that is an engineering decision. For instance, nix won't be able to verify that you aren't depending on changeable remote systems, because you'll have chosen to do that: Nix doesn't trust that npm is add-only, but you might. (Also, some aspects of this will be poorly documented. That's a fixable problem, but the maintainers of a package distribution are probably less interested in the case that optimises for concerns they've chosen not to optimise for, so the documentation for working that way will naturally be worse.)