15 ms·

Ditch your version manager

167 points by Arkh4m 5y ago

type0 5y ago
Nice introduction to nix and niv!
kazinator 5y ago
> Nix is a tool that takes a unique approach to package management and system configuration. Nix is the basis of an OS distro: NixOS. This article is just re-articulating the idea that {the/an} OS distro should be managing this stuff, and not some fledgling language-specific programs (that go behind its back, and do a half-baked job by not controlling the packages external to their language ghetto).
- anderskaseorg 5y ago
  I think NixOS is great, but you don’t need to switch to NixOS to use Nix. You can also install Nix on other Linux distributions or even on macOS, and use Nix to manage individual tools and projects without using it to manage your entire system.
  - dboreham 5y ago
    You can also use Nix to build docker/container images, without a Dockerfile.
- rrix2 5y ago
  the article is also pointing out that Nix can solve for multiple versions, shrink-wrapped environments, cross platform, etc, in a way that most OS distros can't. Nix is more than NixOS of course
oneplane 5y ago
Ditch Your Version Manager.. to then install a different version manager.
- PikachuEXE 5y ago
  https://xkcd.com/927/ https://xkcd.com/927/
  - curiouspear 5y ago
    I came here just for this
    - smohare 5y ago
      Weird, since it’s not really relevant.
      - Vorh 5y ago
        It is, though? Just swap "standards" with "programs"
        anderskaseorg 5y ago
        The point behind the XKCD is that competing standards cause interoperability problems. I would argue, as does the original post, that Nix solves interoperability problems rather than creating them—it is a tool that helps you get stuff done, not a standard that purports to tell other people how they need to do things so you can get stuff done. But I’m not sure whom I’d be arguing this with, since a link to an XKCD does not constitute an argument to begin with.
        oneplane 5y ago
        Does it? I don't think having everyone learn a functional language to write code the manage their dependencies is interoperable with everyone... That doesn't mean it's bad, it just means that it's not going to universally cover everyone's use cases.
- swsieber 5y ago
  Though it's advocating for a system level version manager, not a language level manager.
deleted 5y ago
[deleted]
RcouF1uZ4gsC 5y ago
Does this work on Windows? For me, the version manager that provides the nicest experience cross-platform is Cargo from Rust. It can specify and pin specific versions. It is super easy to give to a fellow developer and have them reproduce the build. In addition, because Rust is suitable for low-level, high performance work, many times all the libraries you actually need are written in Rust and everything works very seemlessly (for example, there is no need to have libxml like the Ruby xml library needed, because you can probably write an XML parser in Rust that will be on par or better performance wise with libxml).
- aidenn0 5y ago
  Nix is very unixy so it probably could be made to work on WSL, but not native windows. Yes, if all of your dependencies are written in a single language, you can use the language-specific version manager. TFA was pointing out a solution that work across different languages.
  - swsieber 5y ago
    It would be very nice to have a cross language and cross platform package manager. Most single language package managers are cross platform, and many single platform managers are cross language.
    - aidenn0 5y ago
      It might be possible to port it to msys/mingw
    - tempay 5y ago
      Conda manages this pretty well and the community ran “conda-forge” channel works. It’s language, platform and architecture agnostic. It only supports distributing binaries which can be a benefit or downside when compared to Nix’s “build the world with caching” approach depending on your needs. Personally I find conda’s approach to be more pragmatic for working with unprivileged systems and how packages expect to be used. Though I do hop the Nix-store like model continues to grow in popularity as it’s much better than the classic posix install layout. In conda, packages are installed into “environments” which are just folders like Python’s venv except language agnostic so you can install specific OpenSSL/Clang/GCC/libWhatever versions and have one environment per project you work on. It’s also the only package manager I’m aware of that can provide a good experience across Windows/Linux/macOS for x86_64/arm64/ppc64le. One issue with conda is that the ecosystem of tooling is a little fragmented. The classic “conda” package has performance issues but there is a second implementation “mamba” which fixes this at the cost of minor changes in behaviour and attempts to merge mamba into conda seem to have stalled. If you want to try it out Id recommend using the “Mambaforge” distribution and using “mamba” for everything except “conda activate” (which is actually a shell function).
phendrenad2 5y ago
Managing dependencies is a big problem, and I feel like we've given up on solving it directly, and instead built workarounds. If we question our assumptions, the first question is: Why do we need multiple Ruby versions at all? Why isn't the latest version of Ruby sufficient? Well, obviously, Ruby's behavior has changed over time. But why isn't it backward-compatible? Why can't I just run Ruby 3.0 with a flag that tells it to emulate Ruby 2.6? Or 1.8 for that matter? Okay another one. "Nokogiri" is the Ruby gem (library) for parsing XML (including HTML). You always need to remember to install libxml when you install it. Why? Why doesn't Nokogiri include an XML parser? Because it would be slow? You can include a native binary in the gem which does the hard computations. Will that work on random new architectures like M1? No? But you can fall back to a Ruby implementation and show a warning message. Also, what if I just need some quick XML parsing and don't care if the parsing is 1000x slower? Can I just get Nokogiri with the Ruby-implemented parser? No? Why not? Then every single Ruby gem out there starts using newer Ruby features and thus, you must update your Ruby version. Why can't library authors gracefully handle older versions of Ruby?
- mixedCase 5y ago
  > You can include a native binary in the gem which does the hard computations. Will that work on random new architectures like M1? No? But you can fall back to a Ruby implementation and show a warning message. Promptly ignored by most people, all while doubling maintenance efforts. It's a trade-off without an easy answer.
- Master_Odin 5y ago
  This just adds burden to programming language and library authors. Sure I could almost certainly support Python 2.7 and Python < 3.6 in all my libraries, but it introduces more code and shims I've got to maintain. Given I'm not being paid for this work and do it for fun, there's no compelling reason to support the small subset of users that cannot or will not update. I imagine variations of the above apply to nokogiri and some amount of Ruby core developers.
  - fiddlerwoaroof 5y ago
jeffparsons 5y ago
Arrived expecting to vehemently disagree, given how much I rely on version managers for everything (Ruby, Node, etc.) in my work. Came away thinking, yeah, okay, Nix probably is a better solution to all of this and more.
a-dub 5y ago
woah this direnv thing is neat
- Groxx 5y ago
  I've been using it at work for a couple years now, across tons of projects and multiple languages/runtimes/etc. Very highly recommended. It's fast, safe, and effective. Much, much better than the various nvm/rvm/rbenv/etc that came before it, and it takes no effort to integrate.
IceWreck 5y ago
Nix, guix and traditional package managers that support multiple versions (like dnf) are better solutions than just containerizing the whole thing. But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions. Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem.
- terrywang 5y ago
  nix is really nice (finding it much better than homebrew cross macOS and Linux) and do good jobs at the best cost for some of my uses cases (kind of binenv / arkade, etc. that manages some binaries that the target hosts use). However, nix has its own problems that I may not qualified to comment (not savvy enough user) - a bit of learning curve (NixOS vs nix package manager) - memory hogging while running system-wide update - disk space consumption is a mystery (GC & optimisation doesn't do good jobs freeing up space) Don't get me wrong, I do like nix ;-)
  - cwp 5y ago
    The learning curve is the biggest issue. Nix is different, and that makes it hard to learn unless you're already into functional programming. I like it too, but damn, it's so hard to get someone up to speed, even if they're sold on the benefits and want to learn.
- nineteen999 5y ago
  > Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem. There a large number of developers around now who've never done traditional (RPM, DEB, SysV) packaging and don't understand it. So while the large distros like Redhat and Debian push on with it, the development community only sees Dockerfiles, Snaps, Flatpaks etc.
  - thejosh 5y ago
mongol 5y ago
Everything to install with Nix must be packaged "the Nix way", I assume? How is the breadth and quality of Nix packages? Just for fun I checked the kiwi image builder, from the openSUSE project, which I recently had some versioning problems with, and it was not there.
- cwp 5y ago
  Nixpkgs is quite good. It has the package I'm looking for most of the time. According to Repology[1], it's got more packaged projects than any other package repository, and more packages that are up to date than any other repository. It's also pretty on top of CVEs, with only 0.38% of packages having potential vulnerabilities. Also check out Repology's size/freshness graph[2] [1] https://repology.org https://repology.org [2] https://repology.org/repositories/graphs https://repology.org/repositories/graphs
zaSmilingIdiot 5y ago
Looks interesting enough... But how does one solve the issue of security level updates for some dependency language? Or when a particular version of some application reaches EOL and is no longer maintained, or theres some functionality in a newer version of Nodejs|Ruby|etc thats needed? From what I understand this would require an update to the Nix version that supports it... but that also potentially means bumping other environmental versions as well, which might not be desired. But I suppose this would amount to the user arranging the structure of their filesystem correctly so its one "system" per dir/folder... Or is there a better way to cater to this? And I suppose this still means that the node modules, gems, etc that are being used then anyway also need to be updated after this accordingly. From my limited understanding of Nix, it seems interesting, and the article was actually useful to me. But I cant seem to shake the feeling that this is another packaging abstraction like others before it, and while it seems like a better variant, its not much different to having X, Y and Z listed as requirements and then letting the dev go off and install such dependencies on their system, in the way that they best know how. Juniors or those new to a specific environment might not know the ecosystem so well so as to know to use rbenv or nvm or whatever, but I'm not sure how Nix solves this issue differently than one of the specific tools its replacing. Theres clearly more to Nix than just setting up language environments, which I'm guessing is where its usefuleness really kicks in. But purely for lang env set up, I'm not sure I see a point over other tooling...
- rgoulter 5y ago
  > But how does one solve the issue of security level updates for some dependency language? Or when a particular version of some application reaches EOL and is no longer maintained, or theres some functionality in a newer version of Nodejs|Ruby|etc thats needed? Nix supports building packages from multiple package sources. e.g. maybe you want an old version of some package which is only available in an older release of nixpkgs. It would be possible to use the older nixpkgs release to install that old package, and a newer nixpkgs release for the others. -- You even don't have to use the main nixpkgs repository. > But I suppose this would amount to the user arranging the structure of their filesystem correctly so its one "system" per dir/folder. Nix handles its filesystem arrangement for all of the "multiple different versions of some package" already. That's part of Nix's value proposition. This also allows being able have packages available in your shell without having to 'pollute' the rest of the system is a neat dynamic, particularly with direnv. > Theres clearly more to Nix than just setting up language environments, which I'm guessing is where its usefuleness really kicks in. Depends how often you (want to) jump into fresh environments, but e.g. Nix allows being able to have a consistent set of programs installed quite easily.
Kiro 5y ago
> My ideal dependency manager would allow me to specify each and every dependency Don't you do that in Docker? I've read the sentence about Ubuntu EOL but don't really understand.
thehm 5y ago
Looks interesting, but I don't see any benefits over using conda. And given the network effect benefit, conda seems like it will remain the more useful tool.
KronisLV 5y ago
> My ideal dependency manager would allow me to specify each and every dependency that is required to work on my projects. It should be easily reproducible, declarative and easy to upgrade. To me it feels like what's left out here is the fact that you need the amount of dependencies that you have to be reasonable in the first place, otherwise no single piece of software is going to help you all that much. For example, a new project that was created with "create-react-app" takes 181 MB of space on disk and has 35'894 files in it. That includes about 1467 modules, all for a relatively simple web application. It doesn't matter if you're using package.json/package.lock, or any other tool or solution out there - with that amount of dependencies you're simply not doing dependency "management" of any kind. I'd argue that if you have >100 dependencies in your project, it's probably too big, unless you have a team that's dedicated to managing and auditing all of them. Of course, no one actually audits their dependencies when faced with such large numbers, and so what's left for most developers is to just trust what's out there.
- cwp 5y ago
  You're not wrong, but that's a different problem. Nix makes sure that you get a predictable and reproducible tree of dependencies, and allows different applications to depend on different versions of the same dependencies. That is, it's a solution to DLL hell. It's solid engineering based on solid theory, and it really does let you manage configuration with a level of reliability that most other package managers only pretend to have. Now auditing dependencies, knowing that the packages you depend on aren't malicious and have no known vulnerabilities... well, that's a whole separate problem. And yeah, we don't really have a solution to that right now. The best we can do, as you say, is keep the attack surface small. But if we did try to solve the auditing problem, the solution would have to sit on top of nix or something like it. If you can't precisely specify a dependency graph and reliably install from that specification, it doesn't matter how good your auditing is or what sort of system of trust you can create. You don't know what you're getting anyway.
  - omegalulw 5y ago
    +1 a good dependency manager should 1) pull all necessary packages for you 2) build them if required. It's your job when downloading a package to use or when adding a dependency to make sure you trust the source - this is not a dependency management problem. A good dependency manager should help you better filter trusted sources or at least have a good build file layout to make it obvious what the sources are but that's it.
coldtea 5y ago
Oh, an Amway-style pitch for Nix.
yewenjie 5y ago
Nix Flakes are easier to manage than niv though.
- ricardobeat 5y ago
  The fact you need a version manager on top of your new, pristine and reproducible version manager is a bit concerning. Can’t whatever shortcomings exist be addressed in nix itself?
  - yewenjie 5y ago
    Yes, that is what Nix Flakes does. You declare your stuff with a flake.nix and it creates a flake.lock file.
  - imatpot 5y ago
    it's not on top of Nix itself but (about to be) a part of it. according to the NixOS Wiki [1] Flakes is an upcoming feature of the base package manager. it's not part of the stable channel yet, and "proper" documentation only currently exists in the unstable manual [2]. [1] https://nixos.wiki/wiki/Flakes https://nixos.wiki/wiki/Flakes [2] https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html https://nixos.org/manual/nix/unstable/command-ref/new-cli/ni...
    - e3bc54b2 5y ago
      GP is kind of right though. Flakes are effectively packages for Nix expressions. Even priginal Flakes RFC calls this out. But the difference is, as you mentioned, flakes are integrated within Nix, and provide better ergonomics.
ricardobeat 5y ago
I had the feeling this would be an ad for Nix after reading the first few paragraphs. Is there really a difference from the Docker example given (“your Ubuntu version is EOL”) though? Any of your language or tools can be EOL’ed even if installed via nix, and you have the exact same problem in your hands.
- aidenn0 5y ago
  As long as you can track down a source-file with the same sha, you can reproduce the build. I've had very old nix expressions fail to build after clearing my cache, but it was usually fixed fairly simply by changing a URL from http://example.com/release/foo-x.y.z.tgz http://example.com/release/foo-x.y.z.tgz to http://example.com/archive/foo-x.y.z.tgz http://example.com/archive/foo-x.y.z.tgz When that wasn't doable, I could google the name of the .tgz file, download a couple and find the one with the matching hash, drop the .tgz inside the directory and change the source to a relative path. Actually I do this even in the "release -> archive" case because "Move a URL once, shame on you. Move it twice, shame on me." [edit] If you really don't want to change the .nix expression, you can manually add the file to the cache, and things will Just Work.
throwaway984393 5y ago
First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager. Second, to use this package manager, you first need to install direnv. How do you install it? with brew, a different package manager. Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first. Fourth, finally, we get to install a simple Unix program, that any package manager could have provided. For the fifth trick, freezing dependencies, you first have to have all the correct versions of all the dependencies to do what you want. How do you establish the right ones? Manually. Just like with any other package manager that builds apps against a specific tree of build deps. "Reproducibility!" Because no other package manager could possibly download the same tarballs and run the same commands. Must be all that functional programming magic. And sixth, the idea that this will work forever. Right. Because any other distro and package manager could not possibly work again in the future, with a mirror of all the packages in a release. That's only for super cool futuristic package managers. No way to reproduce the same packages in the future with old package managers. Look, Nixians, this is all old hat. Every decent package manager can do all these things, just in a less complicated way. Modern systems use containers, and they don't need Nix for that. Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.
- rossmohax 5y ago
  It's not about what is possible, it is about ergonomics. We could have used email for conversations online, yet we use Slack. Other tool require lots of thought and carefull execution for what Nix gives you for free.
  - dmitriid 5y ago
    > Other tool require lots of thought and carefull execution for what Nix gives you for free. How do I pin ruby version to 2.6.3 in Nix? This is given to me for free in other tools. As are reproducible builds because all modern package/dependency managers lock versions.
blt 5y ago
This seems like a good thing for system-level packages, but what about those language-level packages? Could Nix replace both dockerfiles (or a list of apt packages) and pip/conda?
- whateveracct 5y ago
  The short answer to that is..yes :) For Docker, you just call a Nix function and specify what programs to include (both implicitly and explicitly via use) and Nix automatically includes the transitive closure of what you need
gizdan 5y ago
As an alternative, I use asdf[0] which is written in bash. It allows you to manage versions of any package through plugins. Plugins can be written in any language (as long as they can be set to chmod +x). Most plugins are written in bash. Though there is a wealth of plugins, what's nice is that it's trivial to write your plugin. So if something doesn't exist yet, or you have some internal tool you need to version, it takes less than an hour to write a plugin. [0] https://asdf-vm.com/ https://asdf-vm.com/
dmitriid 5y ago
The more I see posts praising nix, the more I am confused by the decisions made. In this post: --- start quote --- How about a different version of Ruby or Node? Let’s say that our project depends on Ruby 2.6 and Node 10. We can go and search for those specific versions --- end quote --- So, to begin with, we still need a "version manager", because we want specific package versions. But look at how this is implemented in nix: let pkgs = import <nixpkgs> { }; in pkgs.mkShell { buildInputs = [ pkgs.hello pkgs.ruby_2_6 pkgs.nodejs-10_x ]; } Why? Because unlike every sane package/dependency manager where you specify a package and a version, nix pretends each version is a separate package. And these packages aren't even correct. If you do go and search for ruby, for example [1], you get the following: ruby Version: 2.7.4 ruby_3_0 Version: 3.0.2 ruby_2_6 Version: 2.6.8 This... This is laughable. How do I install ruby 2.6.8? Oh, there's no ruby_2_6_8, because of course there isn't. And this could be difference between a secure system and all your base are belong to us. And they call this reproducible builds? And that's before getting into the ridiculous --- start quote --- All the software that we installed depends on the specific version of the nixpkgs channel that we installed on our system [whose only version is a commit hash in a git repo] --- end quote --- So you need an extra tool [2] for, quote, "painless dependencies for Nix projects." Yes, sure. I'm definitely ditching my version managers in favor of this tool, that hasn't solved these issues in 18 years of its existence. [1] https://search.nixos.org/packages?channel=21.05&from=0&size=50&sort=relevance&type=packages&query=ruby https://search.nixos.org/packages?channel=21.05&from=0&size=... [2] https://github.com/nmattia/niv https://github.com/nmattia/niv
- rvanlaar 5y ago
  Nix is not easy nor simple. It's complicated but not complex. re: ruby version In ubuntu I can only install ruby2.7 and I don't which minor version. [1] I would need to use rvm anyway. It's the same with nix. What is possible is to pin the version you need by specifying the commit. [2] shows the diff of the commit that moved ruby_2_7 from 2.7.3 to 2.7.4. Say for example ruby 2.7.4 has a regression and the project needs to stay on 2.7.3. The revision has for 2.7.3 is used. [1] > apt search ruby |egrep "^ruby2|^ruby3" WARNING: apt does not have a stable CLI interface. Use with caution in scripts. ruby2.7/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64 ruby2.7-dev/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64 ruby2.7-doc/hirsute-updates,hirsute-updates,hirsute-security,hirsute-security 2.7.2-4ubuntu1.2 all [2] https://github.com/NixOS/nixpkgs/commit/5f9f17cc11caa07e4a9faa68d308d86e16d0f930 https://github.com/NixOS/nixpkgs/commit/5f9f17cc11caa07e4a9f...
kiryin 5y ago
In my experience, Nix and Guix are nice toys, but I'm just not ready for the kind of lifestyle change they require to actually use them for anything. For me, "pinning" a package to a specific version means not downloading and building a newer version.
- topspin 5y ago
  In work-a-day devops containers have solved the bulk of the problems functional package managers aimed to solve in the first place.
jbergens 5y ago
I think webassembly may be a solution to many of the problems with installing tools. We would need an npm like manager for all the tools/libs.
- peter998 5y ago
  Completely agreed! This is one of the main goals behind https://wapm.io https://wapm.io :)
marcus_holmes 5y ago
Now can we solve the problem of installing a program by curl'ing a script from the internet to shell? (bonus points for requiring sudo too)
- moondev 5y ago
  Run it in a container or vm with cloud-init if ultra paranoid. Multipass makes it easy to cloud-init throwaway vms
shtps 5y ago
The problem I have with Nix (and Guix) is that you're shit out of luck if you need it to work together with some network based package manager or existing project that someone didn't add to it yet. It gets really complicated really fast, and things that "just work" with other packages managers and projects, because they are opinionated and linux-y, most of the time don't work well with Nix or have to be shoehorned into it. It always feels like you're fighting against everything to get things into Nix that just weren't meant to be.
- jbboehr 5y ago
  > you need it to work together with some network based package manager Well, you can disable sandboxing if you want, although it's not recommended. This lets you run a package manager that requires networking in a derivation.
hda111 5y ago
Good introduction to Nix. I wish I would have had this when I started. But what do they mean with “works forever”? What if the Git repo or Web server of the project is gone in several years? How do you reproduce it? Does niv include a local source code mirror?
- opan 5y ago
  Guix has integration with software heritage. Not sure about Nix.

167 points by Arkh4m 5y ago

Nice introduction to nix and niv!

> Nix is a tool that takes a unique approach to package management and system configuration. Nix is the basis of an OS distro: NixOS. This article is just re-articulating the idea that {the/an} OS distro should be managing this stuff, and not some fledgling language-specific programs (that go behind its back, and do a half-baked job by not controlling the packages external to their language ghetto).

anderskaseorg 5y ago
I think NixOS is great, but you don’t need to switch to NixOS to use Nix. You can also install Nix on other Linux distributions or even on macOS, and use Nix to manage individual tools and projects without using it to manage your entire system.
- dboreham 5y ago
  You can also use Nix to build docker/container images, without a Dockerfile.
rrix2 5y ago
the article is also pointing out that Nix can solve for multiple versions, shrink-wrapped environments, cross platform, etc, in a way that most OS distros can't. Nix is more than NixOS of course

oneplane 5y ago

Ditch Your Version Manager.. to then install a different version manager.

PikachuEXE 5y ago
https://xkcd.com/927/ https://xkcd.com/927/
- curiouspear 5y ago
  I came here just for this
  - smohare 5y ago
    Weird, since it’s not really relevant.
    - Vorh 5y ago
      It is, though? Just swap "standards" with "programs"
      - anderskaseorg 5y ago
        The point behind the XKCD is that competing standards cause interoperability problems. I would argue, as does the original post, that Nix solves interoperability problems rather than creating them—it is a tool that helps you get stuff done, not a standard that purports to tell other people how they need to do things so you can get stuff done. But I’m not sure whom I’d be arguing this with, since a link to an XKCD does not constitute an argument to begin with.
        oneplane 5y ago
        Does it? I don't think having everyone learn a functional language to write code the manage their dependencies is interoperable with everyone... That doesn't mean it's bad, it just means that it's not going to universally cover everyone's use cases.
swsieber 5y ago
Though it's advocating for a system level version manager, not a language level manager.

deleted 5y ago

[deleted]

RcouF1uZ4gsC 5y ago

Does this work on Windows? For me, the version manager that provides the nicest experience cross-platform is Cargo from Rust. It can specify and pin specific versions. It is super easy to give to a fellow developer and have them reproduce the build. In addition, because Rust is suitable for low-level, high performance work, many times all the libraries you actually need are written in Rust and everything works very seemlessly (for example, there is no need to have libxml like the Ruby xml library needed, because you can probably write an XML parser in Rust that will be on par or better performance wise with libxml).

aidenn0 5y ago
Nix is very unixy so it probably could be made to work on WSL, but not native windows. Yes, if all of your dependencies are written in a single language, you can use the language-specific version manager. TFA was pointing out a solution that work across different languages.
- swsieber 5y ago
  It would be very nice to have a cross language and cross platform package manager. Most single language package managers are cross platform, and many single platform managers are cross language.
  - aidenn0 5y ago
    It might be possible to port it to msys/mingw
  - tempay 5y ago
    Conda manages this pretty well and the community ran “conda-forge” channel works. It’s language, platform and architecture agnostic. It only supports distributing binaries which can be a benefit or downside when compared to Nix’s “build the world with caching” approach depending on your needs. Personally I find conda’s approach to be more pragmatic for working with unprivileged systems and how packages expect to be used. Though I do hop the Nix-store like model continues to grow in popularity as it’s much better than the classic posix install layout. In conda, packages are installed into “environments” which are just folders like Python’s venv except language agnostic so you can install specific OpenSSL/Clang/GCC/libWhatever versions and have one environment per project you work on. It’s also the only package manager I’m aware of that can provide a good experience across Windows/Linux/macOS for x86_64/arm64/ppc64le. One issue with conda is that the ecosystem of tooling is a little fragmented. The classic “conda” package has performance issues but there is a second implementation “mamba” which fixes this at the cost of minor changes in behaviour and attempts to merge mamba into conda seem to have stalled. If you want to try it out Id recommend using the “Mambaforge” distribution and using “mamba” for everything except “conda activate” (which is actually a shell function).

phendrenad2 5y ago

Managing dependencies is a big problem, and I feel like we've given up on solving it directly, and instead built workarounds. If we question our assumptions, the first question is: Why do we need multiple Ruby versions at all? Why isn't the latest version of Ruby sufficient? Well, obviously, Ruby's behavior has changed over time. But why isn't it backward-compatible? Why can't I just run Ruby 3.0 with a flag that tells it to emulate Ruby 2.6? Or 1.8 for that matter? Okay another one. "Nokogiri" is the Ruby gem (library) for parsing XML (including HTML). You always need to remember to install libxml when you install it. Why? Why doesn't Nokogiri include an XML parser? Because it would be slow? You can include a native binary in the gem which does the hard computations. Will that work on random new architectures like M1? No? But you can fall back to a Ruby implementation and show a warning message. Also, what if I just need some quick XML parsing and don't care if the parsing is 1000x slower? Can I just get Nokogiri with the Ruby-implemented parser? No? Why not? Then every single Ruby gem out there starts using newer Ruby features and thus, you must update your Ruby version. Why can't library authors gracefully handle older versions of Ruby?

mixedCase 5y ago
> You can include a native binary in the gem which does the hard computations. Will that work on random new architectures like M1? No? But you can fall back to a Ruby implementation and show a warning message. Promptly ignored by most people, all while doubling maintenance efforts. It's a trade-off without an easy answer.
Master_Odin 5y ago
This just adds burden to programming language and library authors. Sure I could almost certainly support Python 2.7 and Python < 3.6 in all my libraries, but it introduces more code and shims I've got to maintain. Given I'm not being paid for this work and do it for fun, there's no compelling reason to support the small subset of users that cannot or will not update. I imagine variations of the above apply to nokogiri and some amount of Ruby core developers.
- fiddlerwoaroof 5y ago

jeffparsons 5y ago

Arrived expecting to vehemently disagree, given how much I rely on version managers for everything (Ruby, Node, etc.) in my work. Came away thinking, yeah, okay, Nix probably is a better solution to all of this and more.

a-dub 5y ago

woah this direnv thing is neat

Groxx 5y ago
I've been using it at work for a couple years now, across tons of projects and multiple languages/runtimes/etc. Very highly recommended. It's fast, safe, and effective. Much, much better than the various nvm/rvm/rbenv/etc that came before it, and it takes no effort to integrate.

IceWreck 5y ago

Nix, guix and traditional package managers that support multiple versions (like dnf) are better solutions than just containerizing the whole thing. But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions. Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem.

terrywang 5y ago
nix is really nice (finding it much better than homebrew cross macOS and Linux) and do good jobs at the best cost for some of my uses cases (kind of binenv / arkade, etc. that manages some binaries that the target hosts use). However, nix has its own problems that I may not qualified to comment (not savvy enough user) - a bit of learning curve (NixOS vs nix package manager) - memory hogging while running system-wide update - disk space consumption is a mystery (GC & optimisation doesn't do good jobs freeing up space) Don't get me wrong, I do like nix ;-)
- cwp 5y ago
  The learning curve is the biggest issue. Nix is different, and that makes it hard to learn unless you're already into functional programming. I like it too, but damn, it's so hard to get someone up to speed, even if they're sold on the benefits and want to learn.
nineteen999 5y ago
> Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem. There a large number of developers around now who've never done traditional (RPM, DEB, SysV) packaging and don't understand it. So while the large distros like Redhat and Debian push on with it, the development community only sees Dockerfiles, Snaps, Flatpaks etc.
- thejosh 5y ago

mongol 5y ago

Everything to install with Nix must be packaged "the Nix way", I assume? How is the breadth and quality of Nix packages? Just for fun I checked the kiwi image builder, from the openSUSE project, which I recently had some versioning problems with, and it was not there.

cwp 5y ago
Nixpkgs is quite good. It has the package I'm looking for most of the time. According to Repology[1], it's got more packaged projects than any other package repository, and more packages that are up to date than any other repository. It's also pretty on top of CVEs, with only 0.38% of packages having potential vulnerabilities. Also check out Repology's size/freshness graph[2] [1] https://repology.org https://repology.org [2] https://repology.org/repositories/graphs https://repology.org/repositories/graphs

zaSmilingIdiot 5y ago

Looks interesting enough... But how does one solve the issue of security level updates for some dependency language? Or when a particular version of some application reaches EOL and is no longer maintained, or theres some functionality in a newer version of Nodejs|Ruby|etc thats needed? From what I understand this would require an update to the Nix version that supports it... but that also potentially means bumping other environmental versions as well, which might not be desired. But I suppose this would amount to the user arranging the structure of their filesystem correctly so its one "system" per dir/folder... Or is there a better way to cater to this? And I suppose this still means that the node modules, gems, etc that are being used then anyway also need to be updated after this accordingly. From my limited understanding of Nix, it seems interesting, and the article was actually useful to me. But I cant seem to shake the feeling that this is another packaging abstraction like others before it, and while it seems like a better variant, its not much different to having X, Y and Z listed as requirements and then letting the dev go off and install such dependencies on their system, in the way that they best know how. Juniors or those new to a specific environment might not know the ecosystem so well so as to know to use rbenv or nvm or whatever, but I'm not sure how Nix solves this issue differently than one of the specific tools its replacing. Theres clearly more to Nix than just setting up language environments, which I'm guessing is where its usefuleness really kicks in. But purely for lang env set up, I'm not sure I see a point over other tooling...

rgoulter 5y ago
> But how does one solve the issue of security level updates for some dependency language? Or when a particular version of some application reaches EOL and is no longer maintained, or theres some functionality in a newer version of Nodejs|Ruby|etc thats needed? Nix supports building packages from multiple package sources. e.g. maybe you want an old version of some package which is only available in an older release of nixpkgs. It would be possible to use the older nixpkgs release to install that old package, and a newer nixpkgs release for the others. -- You even don't have to use the main nixpkgs repository. > But I suppose this would amount to the user arranging the structure of their filesystem correctly so its one "system" per dir/folder. Nix handles its filesystem arrangement for all of the "multiple different versions of some package" already. That's part of Nix's value proposition. This also allows being able have packages available in your shell without having to 'pollute' the rest of the system is a neat dynamic, particularly with direnv. > Theres clearly more to Nix than just setting up language environments, which I'm guessing is where its usefuleness really kicks in. Depends how often you (want to) jump into fresh environments, but e.g. Nix allows being able to have a consistent set of programs installed quite easily.

Kiro 5y ago

> My ideal dependency manager would allow me to specify each and every dependency Don't you do that in Docker? I've read the sentence about Ubuntu EOL but don't really understand.

thehm 5y ago

Looks interesting, but I don't see any benefits over using conda. And given the network effect benefit, conda seems like it will remain the more useful tool.

KronisLV 5y ago

> My ideal dependency manager would allow me to specify each and every dependency that is required to work on my projects. It should be easily reproducible, declarative and easy to upgrade. To me it feels like what's left out here is the fact that you need the amount of dependencies that you have to be reasonable in the first place, otherwise no single piece of software is going to help you all that much. For example, a new project that was created with "create-react-app" takes 181 MB of space on disk and has 35'894 files in it. That includes about 1467 modules, all for a relatively simple web application. It doesn't matter if you're using package.json/package.lock, or any other tool or solution out there - with that amount of dependencies you're simply not doing dependency "management" of any kind. I'd argue that if you have >100 dependencies in your project, it's probably too big, unless you have a team that's dedicated to managing and auditing all of them. Of course, no one actually audits their dependencies when faced with such large numbers, and so what's left for most developers is to just trust what's out there.

cwp 5y ago
You're not wrong, but that's a different problem. Nix makes sure that you get a predictable and reproducible tree of dependencies, and allows different applications to depend on different versions of the same dependencies. That is, it's a solution to DLL hell. It's solid engineering based on solid theory, and it really does let you manage configuration with a level of reliability that most other package managers only pretend to have. Now auditing dependencies, knowing that the packages you depend on aren't malicious and have no known vulnerabilities... well, that's a whole separate problem. And yeah, we don't really have a solution to that right now. The best we can do, as you say, is keep the attack surface small. But if we did try to solve the auditing problem, the solution would have to sit on top of nix or something like it. If you can't precisely specify a dependency graph and reliably install from that specification, it doesn't matter how good your auditing is or what sort of system of trust you can create. You don't know what you're getting anyway.
- omegalulw 5y ago
  +1 a good dependency manager should 1) pull all necessary packages for you 2) build them if required. It's your job when downloading a package to use or when adding a dependency to make sure you trust the source - this is not a dependency management problem. A good dependency manager should help you better filter trusted sources or at least have a good build file layout to make it obvious what the sources are but that's it.

coldtea 5y ago

Oh, an Amway-style pitch for Nix.

yewenjie 5y ago

Nix Flakes are easier to manage than niv though.

ricardobeat 5y ago
The fact you need a version manager on top of your new, pristine and reproducible version manager is a bit concerning. Can’t whatever shortcomings exist be addressed in nix itself?
- yewenjie 5y ago
  Yes, that is what Nix Flakes does. You declare your stuff with a flake.nix and it creates a flake.lock file.
- imatpot 5y ago
  it's not on top of Nix itself but (about to be) a part of it. according to the NixOS Wiki [1] Flakes is an upcoming feature of the base package manager. it's not part of the stable channel yet, and "proper" documentation only currently exists in the unstable manual [2]. [1] https://nixos.wiki/wiki/Flakes https://nixos.wiki/wiki/Flakes [2] https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html https://nixos.org/manual/nix/unstable/command-ref/new-cli/ni...
  - e3bc54b2 5y ago
    GP is kind of right though. Flakes are effectively packages for Nix expressions. Even priginal Flakes RFC calls this out. But the difference is, as you mentioned, flakes are integrated within Nix, and provide better ergonomics.

ricardobeat 5y ago

I had the feeling this would be an ad for Nix after reading the first few paragraphs. Is there really a difference from the Docker example given (“your Ubuntu version is EOL”) though? Any of your language or tools can be EOL’ed even if installed via nix, and you have the exact same problem in your hands.

aidenn0 5y ago
As long as you can track down a source-file with the same sha, you can reproduce the build. I've had very old nix expressions fail to build after clearing my cache, but it was usually fixed fairly simply by changing a URL from http://example.com/release/foo-x.y.z.tgz http://example.com/release/foo-x.y.z.tgz to http://example.com/archive/foo-x.y.z.tgz http://example.com/archive/foo-x.y.z.tgz When that wasn't doable, I could google the name of the .tgz file, download a couple and find the one with the matching hash, drop the .tgz inside the directory and change the source to a relative path. Actually I do this even in the "release -> archive" case because "Move a URL once, shame on you. Move it twice, shame on me." [edit] If you really don't want to change the .nix expression, you can manually add the file to the cache, and things will Just Work.

throwaway984393 5y ago

First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager. Second, to use this package manager, you first need to install direnv. How do you install it? with brew, a different package manager. Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first. Fourth, finally, we get to install a simple Unix program, that any package manager could have provided. For the fifth trick, freezing dependencies, you first have to have all the correct versions of all the dependencies to do what you want. How do you establish the right ones? Manually. Just like with any other package manager that builds apps against a specific tree of build deps. "Reproducibility!" Because no other package manager could possibly download the same tarballs and run the same commands. Must be all that functional programming magic. And sixth, the idea that this will work forever. Right. Because any other distro and package manager could not possibly work again in the future, with a mirror of all the packages in a release. That's only for super cool futuristic package managers. No way to reproduce the same packages in the future with old package managers. Look, Nixians, this is all old hat. Every decent package manager can do all these things, just in a less complicated way. Modern systems use containers, and they don't need Nix for that. Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.

rossmohax 5y ago
It's not about what is possible, it is about ergonomics. We could have used email for conversations online, yet we use Slack. Other tool require lots of thought and carefull execution for what Nix gives you for free.
- dmitriid 5y ago
  > Other tool require lots of thought and carefull execution for what Nix gives you for free. How do I pin ruby version to 2.6.3 in Nix? This is given to me for free in other tools. As are reproducible builds because all modern package/dependency managers lock versions.

blt 5y ago

This seems like a good thing for system-level packages, but what about those language-level packages? Could Nix replace both dockerfiles (or a list of apt packages) and pip/conda?

whateveracct 5y ago
The short answer to that is..yes :) For Docker, you just call a Nix function and specify what programs to include (both implicitly and explicitly via use) and Nix automatically includes the transitive closure of what you need

gizdan 5y ago

As an alternative, I use asdf[0] which is written in bash. It allows you to manage versions of any package through plugins. Plugins can be written in any language (as long as they can be set to chmod +x). Most plugins are written in bash. Though there is a wealth of plugins, what's nice is that it's trivial to write your plugin. So if something doesn't exist yet, or you have some internal tool you need to version, it takes less than an hour to write a plugin. [0] https://asdf-vm.com/ https://asdf-vm.com/

dmitriid 5y ago

The more I see posts praising nix, the more I am confused by the decisions made. In this post: --- start quote --- How about a different version of Ruby or Node? Let’s say that our project depends on Ruby 2.6 and Node 10. We can go and search for those specific versions --- end quote --- So, to begin with, we still need a "version manager", because we want specific package versions. But look at how this is implemented in nix: let pkgs = import <nixpkgs> { }; in pkgs.mkShell { buildInputs = [ pkgs.hello pkgs.ruby_2_6 pkgs.nodejs-10_x ]; } Why? Because unlike every sane package/dependency manager where you specify a package and a version, nix pretends each version is a separate package. And these packages aren't even correct. If you do go and search for ruby, for example [1], you get the following: ruby Version: 2.7.4 ruby_3_0 Version: 3.0.2 ruby_2_6 Version: 2.6.8 This... This is laughable. How do I install ruby 2.6.8? Oh, there's no ruby_2_6_8, because of course there isn't. And this could be difference between a secure system and all your base are belong to us. And they call this reproducible builds? And that's before getting into the ridiculous --- start quote --- All the software that we installed depends on the specific version of the nixpkgs channel that we installed on our system [whose only version is a commit hash in a git repo] --- end quote --- So you need an extra tool [2] for, quote, "painless dependencies for Nix projects." Yes, sure. I'm definitely ditching my version managers in favor of this tool, that hasn't solved these issues in 18 years of its existence. [1] https://search.nixos.org/packages?channel=21.05&from=0&size=50&sort=relevance&type=packages&query=ruby https://search.nixos.org/packages?channel=21.05&from=0&size=... [2] https://github.com/nmattia/niv https://github.com/nmattia/niv

rvanlaar 5y ago
Nix is not easy nor simple. It's complicated but not complex. re: ruby version In ubuntu I can only install ruby2.7 and I don't which minor version. [1] I would need to use rvm anyway. It's the same with nix. What is possible is to pin the version you need by specifying the commit. [2] shows the diff of the commit that moved ruby_2_7 from 2.7.3 to 2.7.4. Say for example ruby 2.7.4 has a regression and the project needs to stay on 2.7.3. The revision has for 2.7.3 is used. [1] > apt search ruby |egrep "^ruby2|^ruby3" WARNING: apt does not have a stable CLI interface. Use with caution in scripts. ruby2.7/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64 ruby2.7-dev/hirsute-updates,hirsute-security 2.7.2-4ubuntu1.2 amd64 ruby2.7-doc/hirsute-updates,hirsute-updates,hirsute-security,hirsute-security 2.7.2-4ubuntu1.2 all [2] https://github.com/NixOS/nixpkgs/commit/5f9f17cc11caa07e4a9faa68d308d86e16d0f930 https://github.com/NixOS/nixpkgs/commit/5f9f17cc11caa07e4a9f...

kiryin 5y ago

In my experience, Nix and Guix are nice toys, but I'm just not ready for the kind of lifestyle change they require to actually use them for anything. For me, "pinning" a package to a specific version means not downloading and building a newer version.

topspin 5y ago
In work-a-day devops containers have solved the bulk of the problems functional package managers aimed to solve in the first place.

jbergens 5y ago

I think webassembly may be a solution to many of the problems with installing tools. We would need an npm like manager for all the tools/libs.

peter998 5y ago
Completely agreed! This is one of the main goals behind https://wapm.io https://wapm.io :)

marcus_holmes 5y ago

Now can we solve the problem of installing a program by curl'ing a script from the internet to shell? (bonus points for requiring sudo too)

moondev 5y ago
Run it in a container or vm with cloud-init if ultra paranoid. Multipass makes it easy to cloud-init throwaway vms

shtps 5y ago

The problem I have with Nix (and Guix) is that you're shit out of luck if you need it to work together with some network based package manager or existing project that someone didn't add to it yet. It gets really complicated really fast, and things that "just work" with other packages managers and projects, because they are opinionated and linux-y, most of the time don't work well with Nix or have to be shoehorned into it. It always feels like you're fighting against everything to get things into Nix that just weren't meant to be.

jbboehr 5y ago
> you need it to work together with some network based package manager Well, you can disable sandboxing if you want, although it's not recommended. This lets you run a package manager that requires networking in a derivation.

hda111 5y ago

Good introduction to Nix. I wish I would have had this when I started. But what do they mean with “works forever”? What if the Git repo or Web server of the project is gone in several years? How do you reproduce it? Does niv include a local source code mirror?

opan 5y ago
Guix has integration with software heritage. Not sure about Nix.