15 ms·

How to Learn Nix

322 points by allenleee 5y ago

fnord123 5y ago
This is extremely well written documentation. Kudos to the author!
harvie 5y ago
NixOS claims to be "reproducible", "declarative" and "reliable". Yet there is no way of "declaring" which version of given package you want to install, which might be seen as hard requirement for some build to be "reproducible". You might need to stick to some old (but tested) version when building "reliable" system.
- seniorivn 5y ago
  it's not as harsh as you describe it, but it's a well know issue in nix community which is why flakes were introduced
- jskrablin 5y ago
  You can use nix-shell to create complete runtime environments with pinned software versions. Not sure re nix flakes (it's been a while since I last used Nix) however I found nix-shell pretty good for exactly this type of stuff.
- q3k 5y ago
  > Yet there is no way of "declaring" which version of given package you want to install But there is? First, by pinning nixpkgs you pin to whatever set of versions was within nixpkgs at the time. Then, if you want to change the version of a package, you can use overlays to poke at what source version is being used to build a package (or what flags it's being built with, etc). There's also nothing preventing your from mixing derivations from multiple nixpkgs pins within a single nixpkgs/nixos fixpoint for your system. Stuff that in your shell.nix or /etc/nixos/configuration.nix or just nix-build it straight from a small .nix file.
  - YuukiRey 5y ago
    Overlays are very hit and miss for this. It's important for non-nix people to understand that with an overlay you typically only change the version of the source of the program. Essentially think of it like downloading an older git commit. But quite often the Nix build instructions won't work with arbitrary commits. A file that's being patched in the current Nixpkgs checkout might not exist in the version of the source you check out. Then you start playing whack a mole where you override more and more of the derivation. Using an older Nixpkgs checkout might have a higher rate of success but if Nixpkgs never included the version you need in the first place that also doesn't work. So the mechanisms you describe are in my experience very good in theory but often lacking in practice.
    - asymmetric 5y ago
      > Using an older Nixpkgs checkout might have a higher rate of success but if Nixpkgs never included the version you need in the first place that also doesn't work. Then you could get the package from a fork of nixpkgs where it is present.
    - microtonal 5y ago
      Using an older Nixpkgs checkout might have a higher rate of success but if Nixpkgs never included the version you need in the first place that also doesn't work. Yeah, but also keep in mind that uses the older version, plus older versions of all its transitive dependencies (from the same nixpkgs checkout). So, you could end up with multiple versions of glibc and other commonly-used libraries, but worse -- many known security vulnerabilities as well. Another practical annoyance is that overriding attributes often does not work across a derivation. E.g. overriding the version attribute does not override the same version in the src attribute (which is used to download the sources) [1]. So, you may have to override the attribute in several places. [1] Unless the derivation is defined in a non-canonical way: https://discourse.nixos.org/t/avoid-rec-expresions-in-nixpkgs/8293 https://discourse.nixos.org/t/avoid-rec-expresions-in-nixpkg...
  - asymmetric 5y ago
    Here is an example: https://nixos.wiki/wiki/Overlays#Overriding_a_version https://nixos.wiki/wiki/Overlays#Overriding_a_version
- zwiek 5y ago
  Indeed, when you depend on a package from a Nix channel, you implicitly depend always on the latest definition of that package in that channel. When the version matters, Nix has separate packages for separate versions. There are separate Nix packages for Python 2.7, 3.7, 3.8, 3.9 or 3.10, and there's OpenJDK 8, 11 and 17, etc. You probably don't want to depend on an overly specific version anyway. If a security fix for one of your dependencies is released, you want it. Furthermore, CI ensures that everything within a channel is maximally consistent.
- ducktective 5y ago
  Follow this thread. We had a through discussion about this. https://news.ycombinator.com/item?id=28591661 https://news.ycombinator.com/item?id=28591661
skinkestek 5y ago
From the start this looks very promising from more than one angle, including as an ux review done by an actual user: > If someone who contributes to Nix ever actually sees these blog posts, I hope that they take them for what they are: a weird kind of user acceptance test. Not demands or complaints, just the story of one person’s experience.
- pxc 5y ago
  It's such a weirdly specific genre that I never knew I could enjoy, but the writing is colorful and a little charming as well as extremely thorough
l0b0 5y ago
At part 3 now, and it's great. I feel like I might finally learn what all the arcane stuff in the manuals actually mean. Most of the Nix documentation is like reading ASCII EBNF or research papers. This, on the other hand, has just the right amount of snark: >> A store path Q is reachable from another store path P if Q is in the closure of the references relation. > > Good definition. Intuitive term.
- f1refly 5y ago
  I wasn't sure if this is supposed to be snarky, given the author said he has a background in functional programming and thus probably has a good understanding how closures are defined. The statement works both ways I suppose.
  - lxpz 5y ago
    It's "closure" in the mathematical sense, not in the functionnal programming sense. It would have been clearer with the full term: "transitive closure" instead of just "closure".
    - piaste 5y ago
      > It's "closure" in the mathematical sense, not in the functionnal programming sense. The latter being of course an instance of the former :) Your lambda uses a reference to a local variable `x`, so go follow it and transitively capture all its references, repeat until you've closed over all such transitions. let a = // whatever let b x = a + x; let f x = x + b x; // closes over b and transitively over a
eulenteufel 5y ago
I've tried to learn nix and tried to read various documentation. The problem was that there was always a piece of information missing. That was until I've found the nix pills which are a pretty good and structured introduction on people interested in building packages with nix.
- domenkozar 5y ago
  Can you try https://nix.dev https://nix.dev and let me know what's missing?
  - nvarsj 5y ago
    Nice guide! One thing I noticed missing is how to use nix flakes (e.g. in place of niv). That is my next thing to understand, and I want to convert my NixOS / shell.nix files to use them.
  - Macha 5y ago
    I'm still in the learning process, but one thing I've had a hard time is finding out what functions are available. e.g. there's stdenv.mkDerivation for a generic target, and as your docs mention pythonXY.withPackages for a python virtualenv [1]. But what if I wanted, e.g. Ruby? How do I find out what's available? What options they take? https://nix.dev/tutorials/ad-hoc-developer-environments#beyond-tooling-python-libraries https://nix.dev/tutorials/ad-hoc-developer-environments#beyo...
    - 42jd 5y ago
      No experience with ruby but nixpkgs has sections for many languages. See ruby: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/ruby.section.md https://github.com/NixOS/nixpkgs/blob/master/doc/languages-f.... Also a quick search on the NixOS wiki gives this: https://nixos.wiki/wiki/Packaging/Ruby https://nixos.wiki/wiki/Packaging/Ruby. This is less packaging related (more nix lang related) but here is my go to resource for functions: https://teu5us.github.io/nix-lib.html https://teu5us.github.io/nix-lib.html. teu5us extracted the builtin/lib function docs and put them on one place saving you from having it split up into multiple manuals. One issue with nix is the content has been written down it’s very inaccessible. It is split up between blog posts, discourse, nixpkgs code comments, the wiki, Reddit, etc. When working on something in nix I’ll frequently have 10+ sources of information. The google-fu needed is strong. My biggest recommendation is to dive into the nixpkgs repo and other public dotfiles as most likely what you are doing has been done.
eternauta3k 5y ago
I think Nix's complexity is a failure. I love the concept of Nix, but I'll wait until there's something simpler to learn.
- whateveracct 5y ago
  Most of Nix's complexity is intrinsic and not incidental. It's just that it makes it explicit instead of implicit like every other OS & package manager (many of which don't actually solve a lot of important problems and instead rely on the user to keep things consistent and working.)
- zuj 5y ago
  Yes, this is definitely a complex problem to solve. Imaging we are talking about providing a pipEnv style abstraction for all the programming languages, tools and operating systems.
- diggan 5y ago
  I thought so at first too, but then I realized, no matter what distribution comes up in the future, learning a new distribution from the ground up is going to be hard no matter what. I dove head first into Nix and after a understanding the language, the OS and package manager was actually not that hard to understand. Now in after hand, I don't regret any of the the time I spent learning it.
  - res0nat0r 5y ago
    I recently installed GUIX and just out of the gate I like the syntax much more than I ever did with Nix. The Nix syntax is just to random and hard to just guess what is going on if you're not sure what you're looking at. At least GUIX is more traditional looking Scheme type code.
    - viraptor 5y ago
      I've recently learned of Spack and really like its package descriptions. https://spack-tutorial.readthedocs.io/en/latest/tutorial_packaging.html#id7 https://spack-tutorial.readthedocs.io/en/latest/tutorial_pac... I don't have time to play with it, but really hope that one day someone will write something that effectively "compiles to nix". As far as I can tell, that could be done using the Spack's python descriptions.
enricozb 5y ago
Does anyone else not use any of the ad hoc package management with Nix? I exclusively use a declarative `config.nix`, with `sudo nixos-rebuild switch` every time I want to install a package.
- Throwaway194902 5y ago
  I extensively use nix-shell if that falls under ad-hoc package management. Together with lorri[0], I can place a `shell.nix` in my project repository and when cd'ing into the folder, I will be dropped in a shell with all project specific dependencies available to me. [0]: https://github.com/nix-community/lorri https://github.com/nix-community/lorri
- 3np 5y ago
  Mostly yes. I’ve had a couple of repo-specific shells and some utils installed via nix-env -iA. Do you find it works well for you? I think that a lot of my pain points with Nix comes from not “doing things the Nix way”.
- spindle 5y ago
  me
- kenada 5y ago
  This is what I do. If I need an ad hoc environment, I use `nix shell`, or I set up a flake for the project and have `direnv` activate it when I `cd` into the project directory or open the folder in VS Code.
- mason55 5y ago
  nix-env -iA is pretty much deprecated at this point. It was seen as an easy on ramp but in reality it caused confusion and created more issues.
indigo747 5y ago
I've bounced off Nix a couple of times and figured that the problem is that it was too different. A lot of Linux distros (especially post-systemd) have a fair number of similarities and it's pretty easy to translate knowledge between them. NixOS _probably_ works very similarly, but it's also got this additional declarative configuration layer on top, which is the point of using NixOS, and it's got its own special way of doing things. You can't translate Linux knowledge to a NixOS config directly, you need NixOS' documentation to understand the "NixOS Way" of doing things, and the NixOS way of doing things does not translate into other distros. And the documentation is difficult to understand and at times seemingly arbitrary. You have to commit to learning NixOS and understand that you are only learning NixOS, it won't help you use anything else. Other Linux distros help you learn Linux in general and inform your understanding of Linux system construction, but NixOS is its own thing. Plenty of people learn NixOS anyway, and more power to them, but it's definitely pushed me away.
- ivank 5y ago
  The NixOS way of doing things (a layer on top of plain config files) is inevitable once configuration management is taken seriously: one needs to be able to merge configuration values from multiple source files; for example, a generic role plus some more machine-specific role, working together to produce one systemd unit or file in /etc. The modules in nixpkgs effectively make it possible to combine configuration values to produce most of the config files that Linux applications want. If one weren't using NixOS to do it, it would be done with your own templates and configuration management, and probably without the huge advantage of being able to rebuild a whole immutable system without leftovers from prior configurations.
  - temptemptemp111 5y ago
    Where is the evidence for this? Where is the evidence that you can and should mix config management all of the way down the OS stack? How many containerization concepts do we need? Docker, lxc, VMs, and now NixOS? If it were a legitimate abstraction layer, then wouldn't it have caused fewer problems in implementation? And wouldn't it have seemed more intuitive to Unix experts? Yes... Reinventing the wheel again. I'm open to nicer restructuring of the Linux fileystem, but this is really re-inventing the wheel trying to polish over ugly parts that are ugly for a reason. Keep useful abstractions separate!
domenkozar 5y ago
We know documentation is too scattered and incomplete. I've started https://nix.dev https://nix.dev about a year ago to provide a resource for newcomers. I'm working on comprehensive language tutorial next, but I'm mostly interested what you'd like to learn when getting started.
- ratorx 5y ago
  Recently started using NixOS, and nix.dev has been very useful thanks! Some more NixOS-specific stuff would be really nice, if it’s in scope. I think a lot of the power of NixOS comes from the composition aspect, but I haven’t managed to figure out how mkOverride, mkIf, mkDefault and everything else actually work (rather than at a superficial level). I’m not sure if something as simple as adding an item to an option list from 2 separate bits of config is possible for example. Along the same line, another thing I’ve found hard to discover is patterns used in the standard library. For example, I recently learned about makeBinPath from some random Nix file, and I’d really like some way of discovering quality of life stuff like that without needing to randomly come across it in some Nix code.
- tarkin2 5y ago
  I’ve only read the bit on ad hoc shells but so far this is the best nix resource I’ve seen. Thanks for the hard work.
- pxc 5y ago
  I hope you take the time to read the whole series, Domen! It's like a perfect test case to drive the kind of docs you've put so much effort into— an extremely detailed experience report
- thibran 5y ago
  - How to update a single package to a git checkout - How to update an in package information that has no setting for it in the function signature - How to setup the night-light feature in sway
- _huayra_ 5y ago
  A couple of things come to mind that I'm struggling with now: 1. how to install outside of NixOS, mostly to use home-manager, in a flake-y way using the new `nix profile` thing that supercedes nix-env. The few folks I've spoken to seem to prefer to use NixOS as a shell, then to manage their dotfiles with home-manager, and trying to figure out how to manage this global state in profiles with flakes (e.g. I've heard ~/.config/nixpkgs/flake.nix is the location, but hits on a web search for this file name are almost non-existent). 2. how does one _develop_ with flakes. I get confused about how to use the repl to add something into my config (thankfully there's :lf now!). Some simple walkthrough that goes from packaging a simple application and troubleshooting in the repl, using shell to check the build stages, etc, would be extremely useful. Once one gets over this hurdle, Nix becomes an extremely attractive tool to bundle existing code and manageg packages, and new users start to see the power of it.
teekert 5y ago
Ok so I have 0 experience with Nix however, I been eyeballing it for some time now. I like the setup of this post, but I run my personal infra completely from code (docker compose, traefic). I now run this on Ubuntu and I’ve been wondering for some time why my infra can be declarative but my OS can’t. And then I started eyeballing NixOS… I think from my perspective it wil make more sense i.e. the first thing that OP runs into is: “ nix-env -i git, right? Wrong. I mean, you could do that. But you actually want to say nix-env -iA nixpkgs.git. Why? Don’t worry about it.” My Nix-noob but declarative mindset would assume the "A" is to add the install somewhere to a file needed to recreate the server at some point. Am I wrong? No idea… Actually, I already find it strand that you can call the package manager like that, shouldn't there be a declaration of the packages I want somewhere, should I not add the package that list (maybe a Yaml, like my docker-compose.yaml?) then ask Nix to make the installation consistent with the declaration?
- rgoulter 5y ago
  The "-i" shortflag is for "--install". The "-A" shortflag indicates that the argument given is an "attribute path". Roughly: without the "-A", the command searches for packages named "git" and installs that. With the "-A", it will install the "git" package from nixpkgs.
  - teekert 5y ago
    Ah, guessed wrong, well I should have known, on a declarative system there should not be a way to add packages without declaring them anyway :) (I'm glad I'm wrong)
    - rgoulter 5y ago
      > on a declarative system there should not be a way to add packages without declaring them anyway :) (I'm glad I'm wrong) home-manager is one popular way of declaring what packages should be installed. The nixpkgs manual explains a different way of declaratively managing packages: https://nixos.org/manual/nixpkgs/stable/#sec-declarative-package-management https://nixos.org/manual/nixpkgs/stable/#sec-declarative-pac...
salamat3333 5y ago
hello world!
mgaunard 5y ago
I've been looking for a simple package management solution that doesn't interfere with my OS and simply puts all packages in /some_location/package/version, and that allows me to install packages on remote machines together with the other packages they depend on. I thought Nix could be this; unfortunately it looks like it's an overengineered thing that requires that everything you do follows the Nix philosophy.
- AnIdiotOnTheNet 5y ago
  A less overengineered form of what you're talking about would be GoboLinux.
  - microtonal 5y ago
    Or if you want something that works across distributions, Linuxbrew is an option: https://docs.brew.sh/Homebrew-on-Linux https://docs.brew.sh/Homebrew-on-Linux
- pxc 5y ago
  That was my original use case for Nix! Computer lab admins wouldn't install something I wanted, so I went about seeing how to install it myself. Linuxbrew had to compile everything from source, but Nix in a proot environment worked perfectly. If you're not going to be writing packages yourself, whatever package manager you use, you don't really have to worry too much about The Nix Way for the use case you describe
- ris 5y ago
  > I've been looking for a simple package management solution that doesn't interfere with my OS and simply puts all packages in /some_location/package/version, and that allows me to install packages on remote machines together with the other packages they depend on. Almost anything that achieves this will necessarily be as "overengineered" as Nix. What you're asking as it turns out is not a simple problem at all, and a "simple" approach to it will end up rather fragile.
  - mgaunard 5y ago
riofoxx 5y ago
I can help you earn $ 6.0,0.0.0 with $ 5.0.0.0 in a w.e.e.k t.h.r.o.u.g.h M.R h.a r.l.a.n.d. w.h.o h.e.l.p.e.d m.e g.e.t the b.e.s.t in c.r.y.p.t.o t.r.a.d.i.n.g.. Contact information via WhatsApp +13052395906)
MichaelBurge 5y ago
I learned Nix a couple days ago because I needed a build system for my project that could handle many different tools: Java+Gradle, Thrift, Rust, Python, CUDA, etc. It's been kind of a pain: Docker is easier to get started with. But the payoff seems higher because it does handle all these tools in a sensible way: For development I can switch between subprojects written in many different languages and get the exact Python, Java, etc. and all necessary libraries are in scope. I've gotten it working on Linux, and I expect a little more work porting to Mac OS for teammates(mostly for things like switching from CUDA to CPU based on system), but using it should be similar to Linux. Mac doesn't support cgroups or Linux containers so sandboxing is weaker than Linux, so it's better to develop Nix projects on Linux and add other platforms afterward. I would recommend it for polyglot or Python projects. But for single-language projects with a sensible dependency manager(Rust Cargo), there's not as much benefit.
- pxc 5y ago
  Nix is really nice for polyglot development environments on macOS, because the fact that Docker requires full machine virtualization makes it very heavyweight. Nix gets to be a lot faster and simpler. (And you can use it to generate Docker images and full fat VMs if you still want that option.)
shp0ngle 5y ago
Reading through this blog makes me happy that I am using homebrew
- pxc 5y ago
  That made me laugh because I understand the feeling. Homebrew is very, very good at the things Nix is worst at. But Nix is also excellent at the things Homebrew is worst at. The new CLI that comes (disabled by default because it's still 'experimental') with the most recent major Nix release (2.4) goes a long way toward learning from the UX of tools like Homebrew. Hopefully changes like that will help more people get to the good parts before the UX warts turn them off.
k__ 5y ago
When I looked into Nix years ago, I started with a few blog posts that explained the Nix language. It's a bit obscure if you never used Haskell. Next, I read about the concepts and tried to map them to Git in my mind. In the end I had the impression that Nix is a cleaner, more general PyEnv.
Abimelex 5y ago
useless fun fact: "Nix" basically is German slang for "nothing". So I read "how to learn nothing" and asked myself. Indeed, how to do so? :D
- lom 5y ago
  The name is actually referencing nix in latin, which means snow. That’s also why the logo. But even better is “nix flakes”, which just takes the name to a completely different level.
jimthom 5y ago
A definitely great way to learn and at the same time have someone's insights on the subject. Props to you, my friend.
pxc 5y ago
Excellent blog series! Very good reading for people who've been using Nix for a long time, because it gives you a clear sense of things you've likely forgotten by now
deleted 5y ago
[deleted]
fowlie 5y ago
> Okay! Wow. The script is 1338 lines long. That’s a lotta shell – I will not reproduce it here. But also: come on, Nix maintainers. Trim one line. You’re so close. Made me laugh! :D This material is gold by the way - I am learning NixOS and HomeManager myself, and a lot of questions I also have are addressed here. Thanks a lot!
nixpulvis 5y ago
Part 41 is basically what made me realize Nix wasn't ready to use yet. In addition to a number of smaller things. Overall, I'm still on the fence about having to rewrite all the configuration files and documentation in Nix, but having a unified setup is enticing. Maybe there's a way documentation can we prevesred and translated automatically, but it seems difficult to me.
mastax 5y ago
One of the things I've hated about being a part-time systems administrator is the fear of the unknown. All the little changes to config files, the different installed programs, everything required to get the system working is scattered around the disk. It creates fear around making changes, because who knows which little configuration change was required to get everything working? I understand that better organizations try to combat this with rigorous processes and documentation, with great effort. I recently evaluated NixOS for a new server deployment. Being able to describe the entire state of the system in a single file is magical - it is so clearly The Right Way to do things. I am not a Functional Programmer but using the Nix language as just a config file is fairly straightforward. I left convinced that a simple functional language is the best way to make more complex config files, way better than Jinja2 templated YAML. (Though, functionistas, come down out of the ivory tower and make a language that looks C-family. Yeah, "it's just syntax" so why not make it easy for the proles like me to learn.) However, I ended up giving up on NixOS for two reasons. First, while learning the basics of Nix was easy I wasn't confident about the more complex things. Packages were missing features and I was going to have to start investing in reading and writing lots of Nix code. I couldn't continue ignoring the many intricacies of the Nix system if I wanted to be confident that I could quickly fix an issue at 2AM on a Saturday. As others have noted, there is a lack of good intermediate-level documentation apart from this great "How to Learn Nix" blog. My whole interest in Nix was being able to completely document the system so that future me or someone else could quickly understand how the system is configured. That doesn't work if you can't count on that person knowing Nix very well. Even if I invested in learning Nix it would feel like professional malpractice to dump some heavily customized Nix system on the next guy to come along. Second, there's no LTS. That's fine if it's a dev machine, or if you have a professional full-time ops team watching package changelogs and testing upgrades. My machines need to survive with bare-minimum supervision.
ianthehenry 5y ago
Oh hi! Author here. Fun thing to wake up to. I wanted to point something out that the series doesn't do a good job of highlighting: You can use Nix alongside your existing package manager. They won't conflict. You can set up dev environments with Nix without drinking the kool-aid. I still have Homebrew installed, and I still occasionally reach for it in the rare cases where Nix is missing some package I want to try. But I am sad every time I do: I never really realized how unbelievably slow Homebrew is until I tried Nix. So if you're Nix-curious, you can try it out without committing to anything upfront. I didn't make the switch until part 19: https://ianthehenry.com/posts/how-to-learn-nix/switching-from-homebrew-to-nix/ https://ianthehenry.com/posts/how-to-learn-nix/switching-fro... Nix is weird and hard to learn, but it's a lot like git in that respect. Yes, the UI is terrible. You have to learn the difference between `reset --hard` and `reset --mixed` and it's weird and unintuitive but you eventually get over it. And people are using it despite how unintuitive it is, because of how valuable it is to them. No one is arguing that `nix-env -qaP package` is a reasonable command to ask a fellow human being to type, but people are willing to type it anyway because the benefits are worth it. As a fun example, a few weeks ago I wrote up a little hack that I would not have even attempted before spending the time to learn Nix: https://ianthehenry.com/posts/janet-game/how-to-patch-emacs/ https://ianthehenry.com/posts/janet-game/how-to-patch-emacs/
- pxc 5y ago
  from that newer blog post: > So even if something bothers you about the software you use every day – even if you find a bug in tmux, or wish that you had some additional jq function or whatever – you probably aren’t going to do anything about it. It’s not worth it; it’s too hard. > I am obviously projecting here: that is the way that I thought before I started using Nix. I never would have tried to change this Emacs behavior back then. > But now? > Now I can get a working build environment for any package I want, just by running nix-shell. I can write a patch, and I can ask Nix to automatically apply that patch to the latest version of the package. I don’t have to worry about maintaining my own fork or manually updating it – I can keep using all the same commands to list and upgrade and uninstall packages, regardless of what I’ve done to them. I think this is what all the old greybeards have always been getting at when they talk about software freedom, this blurring of the line between user and contributor that is possible with F/OSS. I know for a fact that there are some HN posters who do use apt-get and related tools this way (that Debian developer who taught his kids to use Xmonad, for example). But I do think it's something Nix is especially good at, as a flexible, source-based package manager. It's really nice to be able to treat your customized builds in a first-class way with a mature system for managing software. Btw, if your main use case for this is customizing F/OSS, there's a SaaS offering you can use for free to recover binary caching without even having to set up a build server. Pretty cool if you have multiple computers! https://www.cachix.org/ https://www.cachix.org/
zuj 5y ago
Thanks for writing this. I am just dabbling in Nix and am glad I found this. I wish more documentation is written with your level of inner thinking.

322 points by allenleee 5y ago

This is extremely well written documentation. Kudos to the author!

NixOS claims to be "reproducible", "declarative" and "reliable". Yet there is no way of "declaring" which version of given package you want to install, which might be seen as hard requirement for some build to be "reproducible". You might need to stick to some old (but tested) version when building "reliable" system.

seniorivn 5y ago
it's not as harsh as you describe it, but it's a well know issue in nix community which is why flakes were introduced
jskrablin 5y ago
You can use nix-shell to create complete runtime environments with pinned software versions. Not sure re nix flakes (it's been a while since I last used Nix) however I found nix-shell pretty good for exactly this type of stuff.
q3k 5y ago
> Yet there is no way of "declaring" which version of given package you want to install But there is? First, by pinning nixpkgs you pin to whatever set of versions was within nixpkgs at the time. Then, if you want to change the version of a package, you can use overlays to poke at what source version is being used to build a package (or what flags it's being built with, etc). There's also nothing preventing your from mixing derivations from multiple nixpkgs pins within a single nixpkgs/nixos fixpoint for your system. Stuff that in your shell.nix or /etc/nixos/configuration.nix or just nix-build it straight from a small .nix file.
- YuukiRey 5y ago
  Overlays are very hit and miss for this. It's important for non-nix people to understand that with an overlay you typically only change the version of the source of the program. Essentially think of it like downloading an older git commit. But quite often the Nix build instructions won't work with arbitrary commits. A file that's being patched in the current Nixpkgs checkout might not exist in the version of the source you check out. Then you start playing whack a mole where you override more and more of the derivation. Using an older Nixpkgs checkout might have a higher rate of success but if Nixpkgs never included the version you need in the first place that also doesn't work. So the mechanisms you describe are in my experience very good in theory but often lacking in practice.
  - asymmetric 5y ago
    > Using an older Nixpkgs checkout might have a higher rate of success but if Nixpkgs never included the version you need in the first place that also doesn't work. Then you could get the package from a fork of nixpkgs where it is present.
  - microtonal 5y ago
    Using an older Nixpkgs checkout might have a higher rate of success but if Nixpkgs never included the version you need in the first place that also doesn't work. Yeah, but also keep in mind that uses the older version, plus older versions of all its transitive dependencies (from the same nixpkgs checkout). So, you could end up with multiple versions of glibc and other commonly-used libraries, but worse -- many known security vulnerabilities as well. Another practical annoyance is that overriding attributes often does not work across a derivation. E.g. overriding the version attribute does not override the same version in the src attribute (which is used to download the sources) [1]. So, you may have to override the attribute in several places. [1] Unless the derivation is defined in a non-canonical way: https://discourse.nixos.org/t/avoid-rec-expresions-in-nixpkgs/8293 https://discourse.nixos.org/t/avoid-rec-expresions-in-nixpkg...
- asymmetric 5y ago
  Here is an example: https://nixos.wiki/wiki/Overlays#Overriding_a_version https://nixos.wiki/wiki/Overlays#Overriding_a_version
zwiek 5y ago
Indeed, when you depend on a package from a Nix channel, you implicitly depend always on the latest definition of that package in that channel. When the version matters, Nix has separate packages for separate versions. There are separate Nix packages for Python 2.7, 3.7, 3.8, 3.9 or 3.10, and there's OpenJDK 8, 11 and 17, etc. You probably don't want to depend on an overly specific version anyway. If a security fix for one of your dependencies is released, you want it. Furthermore, CI ensures that everything within a channel is maximally consistent.
ducktective 5y ago
Follow this thread. We had a through discussion about this. https://news.ycombinator.com/item?id=28591661 https://news.ycombinator.com/item?id=28591661

skinkestek 5y ago

From the start this looks very promising from more than one angle, including as an ux review done by an actual user: > If someone who contributes to Nix ever actually sees these blog posts, I hope that they take them for what they are: a weird kind of user acceptance test. Not demands or complaints, just the story of one person’s experience.

pxc 5y ago
It's such a weirdly specific genre that I never knew I could enjoy, but the writing is colorful and a little charming as well as extremely thorough

l0b0 5y ago

At part 3 now, and it's great. I feel like I might finally learn what all the arcane stuff in the manuals actually mean. Most of the Nix documentation is like reading ASCII EBNF or research papers. This, on the other hand, has just the right amount of snark: >> A store path Q is reachable from another store path P if Q is in the closure of the references relation. > > Good definition. Intuitive term.

f1refly 5y ago
I wasn't sure if this is supposed to be snarky, given the author said he has a background in functional programming and thus probably has a good understanding how closures are defined. The statement works both ways I suppose.
- lxpz 5y ago
  It's "closure" in the mathematical sense, not in the functionnal programming sense. It would have been clearer with the full term: "transitive closure" instead of just "closure".
  - piaste 5y ago
    > It's "closure" in the mathematical sense, not in the functionnal programming sense. The latter being of course an instance of the former :) Your lambda uses a reference to a local variable `x`, so go follow it and transitively capture all its references, repeat until you've closed over all such transitions. let a = // whatever let b x = a + x; let f x = x + b x; // closes over b and transitively over a

eulenteufel 5y ago

I've tried to learn nix and tried to read various documentation. The problem was that there was always a piece of information missing. That was until I've found the nix pills which are a pretty good and structured introduction on people interested in building packages with nix.

domenkozar 5y ago
Can you try https://nix.dev https://nix.dev and let me know what's missing?
- nvarsj 5y ago
  Nice guide! One thing I noticed missing is how to use nix flakes (e.g. in place of niv). That is my next thing to understand, and I want to convert my NixOS / shell.nix files to use them.
- Macha 5y ago
  I'm still in the learning process, but one thing I've had a hard time is finding out what functions are available. e.g. there's stdenv.mkDerivation for a generic target, and as your docs mention pythonXY.withPackages for a python virtualenv [1]. But what if I wanted, e.g. Ruby? How do I find out what's available? What options they take? https://nix.dev/tutorials/ad-hoc-developer-environments#beyond-tooling-python-libraries https://nix.dev/tutorials/ad-hoc-developer-environments#beyo...
  - 42jd 5y ago
    No experience with ruby but nixpkgs has sections for many languages. See ruby: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/ruby.section.md https://github.com/NixOS/nixpkgs/blob/master/doc/languages-f.... Also a quick search on the NixOS wiki gives this: https://nixos.wiki/wiki/Packaging/Ruby https://nixos.wiki/wiki/Packaging/Ruby. This is less packaging related (more nix lang related) but here is my go to resource for functions: https://teu5us.github.io/nix-lib.html https://teu5us.github.io/nix-lib.html. teu5us extracted the builtin/lib function docs and put them on one place saving you from having it split up into multiple manuals. One issue with nix is the content has been written down it’s very inaccessible. It is split up between blog posts, discourse, nixpkgs code comments, the wiki, Reddit, etc. When working on something in nix I’ll frequently have 10+ sources of information. The google-fu needed is strong. My biggest recommendation is to dive into the nixpkgs repo and other public dotfiles as most likely what you are doing has been done.

eternauta3k 5y ago

I think Nix's complexity is a failure. I love the concept of Nix, but I'll wait until there's something simpler to learn.

whateveracct 5y ago
Most of Nix's complexity is intrinsic and not incidental. It's just that it makes it explicit instead of implicit like every other OS & package manager (many of which don't actually solve a lot of important problems and instead rely on the user to keep things consistent and working.)
zuj 5y ago
Yes, this is definitely a complex problem to solve. Imaging we are talking about providing a pipEnv style abstraction for all the programming languages, tools and operating systems.
diggan 5y ago
I thought so at first too, but then I realized, no matter what distribution comes up in the future, learning a new distribution from the ground up is going to be hard no matter what. I dove head first into Nix and after a understanding the language, the OS and package manager was actually not that hard to understand. Now in after hand, I don't regret any of the the time I spent learning it.
- res0nat0r 5y ago
  I recently installed GUIX and just out of the gate I like the syntax much more than I ever did with Nix. The Nix syntax is just to random and hard to just guess what is going on if you're not sure what you're looking at. At least GUIX is more traditional looking Scheme type code.
  - viraptor 5y ago
    I've recently learned of Spack and really like its package descriptions. https://spack-tutorial.readthedocs.io/en/latest/tutorial_packaging.html#id7 https://spack-tutorial.readthedocs.io/en/latest/tutorial_pac... I don't have time to play with it, but really hope that one day someone will write something that effectively "compiles to nix". As far as I can tell, that could be done using the Spack's python descriptions.

enricozb 5y ago

Does anyone else not use any of the ad hoc package management with Nix? I exclusively use a declarative `config.nix`, with `sudo nixos-rebuild switch` every time I want to install a package.

Throwaway194902 5y ago
I extensively use nix-shell if that falls under ad-hoc package management. Together with lorri[0], I can place a `shell.nix` in my project repository and when cd'ing into the folder, I will be dropped in a shell with all project specific dependencies available to me. [0]: https://github.com/nix-community/lorri https://github.com/nix-community/lorri
3np 5y ago
Mostly yes. I’ve had a couple of repo-specific shells and some utils installed via nix-env -iA. Do you find it works well for you? I think that a lot of my pain points with Nix comes from not “doing things the Nix way”.
spindle 5y ago
me
kenada 5y ago
This is what I do. If I need an ad hoc environment, I use `nix shell`, or I set up a flake for the project and have `direnv` activate it when I `cd` into the project directory or open the folder in VS Code.
mason55 5y ago
nix-env -iA is pretty much deprecated at this point. It was seen as an easy on ramp but in reality it caused confusion and created more issues.

indigo747 5y ago

I've bounced off Nix a couple of times and figured that the problem is that it was too different. A lot of Linux distros (especially post-systemd) have a fair number of similarities and it's pretty easy to translate knowledge between them. NixOS _probably_ works very similarly, but it's also got this additional declarative configuration layer on top, which is the point of using NixOS, and it's got its own special way of doing things. You can't translate Linux knowledge to a NixOS config directly, you need NixOS' documentation to understand the "NixOS Way" of doing things, and the NixOS way of doing things does not translate into other distros. And the documentation is difficult to understand and at times seemingly arbitrary. You have to commit to learning NixOS and understand that you are only learning NixOS, it won't help you use anything else. Other Linux distros help you learn Linux in general and inform your understanding of Linux system construction, but NixOS is its own thing. Plenty of people learn NixOS anyway, and more power to them, but it's definitely pushed me away.

ivank 5y ago
The NixOS way of doing things (a layer on top of plain config files) is inevitable once configuration management is taken seriously: one needs to be able to merge configuration values from multiple source files; for example, a generic role plus some more machine-specific role, working together to produce one systemd unit or file in /etc. The modules in nixpkgs effectively make it possible to combine configuration values to produce most of the config files that Linux applications want. If one weren't using NixOS to do it, it would be done with your own templates and configuration management, and probably without the huge advantage of being able to rebuild a whole immutable system without leftovers from prior configurations.
- temptemptemp111 5y ago
  Where is the evidence for this? Where is the evidence that you can and should mix config management all of the way down the OS stack? How many containerization concepts do we need? Docker, lxc, VMs, and now NixOS? If it were a legitimate abstraction layer, then wouldn't it have caused fewer problems in implementation? And wouldn't it have seemed more intuitive to Unix experts? Yes... Reinventing the wheel again. I'm open to nicer restructuring of the Linux fileystem, but this is really re-inventing the wheel trying to polish over ugly parts that are ugly for a reason. Keep useful abstractions separate!

domenkozar 5y ago

We know documentation is too scattered and incomplete. I've started https://nix.dev https://nix.dev about a year ago to provide a resource for newcomers. I'm working on comprehensive language tutorial next, but I'm mostly interested what you'd like to learn when getting started.

ratorx 5y ago
Recently started using NixOS, and nix.dev has been very useful thanks! Some more NixOS-specific stuff would be really nice, if it’s in scope. I think a lot of the power of NixOS comes from the composition aspect, but I haven’t managed to figure out how mkOverride, mkIf, mkDefault and everything else actually work (rather than at a superficial level). I’m not sure if something as simple as adding an item to an option list from 2 separate bits of config is possible for example. Along the same line, another thing I’ve found hard to discover is patterns used in the standard library. For example, I recently learned about makeBinPath from some random Nix file, and I’d really like some way of discovering quality of life stuff like that without needing to randomly come across it in some Nix code.
tarkin2 5y ago
I’ve only read the bit on ad hoc shells but so far this is the best nix resource I’ve seen. Thanks for the hard work.
pxc 5y ago
I hope you take the time to read the whole series, Domen! It's like a perfect test case to drive the kind of docs you've put so much effort into— an extremely detailed experience report
thibran 5y ago
- How to update a single package to a git checkout - How to update an in package information that has no setting for it in the function signature - How to setup the night-light feature in sway
_huayra_ 5y ago
A couple of things come to mind that I'm struggling with now: 1. how to install outside of NixOS, mostly to use home-manager, in a flake-y way using the new `nix profile` thing that supercedes nix-env. The few folks I've spoken to seem to prefer to use NixOS as a shell, then to manage their dotfiles with home-manager, and trying to figure out how to manage this global state in profiles with flakes (e.g. I've heard ~/.config/nixpkgs/flake.nix is the location, but hits on a web search for this file name are almost non-existent). 2. how does one _develop_ with flakes. I get confused about how to use the repl to add something into my config (thankfully there's :lf now!). Some simple walkthrough that goes from packaging a simple application and troubleshooting in the repl, using shell to check the build stages, etc, would be extremely useful. Once one gets over this hurdle, Nix becomes an extremely attractive tool to bundle existing code and manageg packages, and new users start to see the power of it.

teekert 5y ago

Ok so I have 0 experience with Nix however, I been eyeballing it for some time now. I like the setup of this post, but I run my personal infra completely from code (docker compose, traefic). I now run this on Ubuntu and I’ve been wondering for some time why my infra can be declarative but my OS can’t. And then I started eyeballing NixOS… I think from my perspective it wil make more sense i.e. the first thing that OP runs into is: “ nix-env -i git, right? Wrong. I mean, you could do that. But you actually want to say nix-env -iA nixpkgs.git. Why? Don’t worry about it.” My Nix-noob but declarative mindset would assume the "A" is to add the install somewhere to a file needed to recreate the server at some point. Am I wrong? No idea… Actually, I already find it strand that you can call the package manager like that, shouldn't there be a declaration of the packages I want somewhere, should I not add the package that list (maybe a Yaml, like my docker-compose.yaml?) then ask Nix to make the installation consistent with the declaration?

rgoulter 5y ago
The "-i" shortflag is for "--install". The "-A" shortflag indicates that the argument given is an "attribute path". Roughly: without the "-A", the command searches for packages named "git" and installs that. With the "-A", it will install the "git" package from nixpkgs.
- teekert 5y ago
  Ah, guessed wrong, well I should have known, on a declarative system there should not be a way to add packages without declaring them anyway :) (I'm glad I'm wrong)
  - rgoulter 5y ago
    > on a declarative system there should not be a way to add packages without declaring them anyway :) (I'm glad I'm wrong) home-manager is one popular way of declaring what packages should be installed. The nixpkgs manual explains a different way of declaratively managing packages: https://nixos.org/manual/nixpkgs/stable/#sec-declarative-package-management https://nixos.org/manual/nixpkgs/stable/#sec-declarative-pac...

salamat3333 5y ago

hello world!

mgaunard 5y ago

I've been looking for a simple package management solution that doesn't interfere with my OS and simply puts all packages in /some_location/package/version, and that allows me to install packages on remote machines together with the other packages they depend on. I thought Nix could be this; unfortunately it looks like it's an overengineered thing that requires that everything you do follows the Nix philosophy.

AnIdiotOnTheNet 5y ago
A less overengineered form of what you're talking about would be GoboLinux.
- microtonal 5y ago
  Or if you want something that works across distributions, Linuxbrew is an option: https://docs.brew.sh/Homebrew-on-Linux https://docs.brew.sh/Homebrew-on-Linux
pxc 5y ago
That was my original use case for Nix! Computer lab admins wouldn't install something I wanted, so I went about seeing how to install it myself. Linuxbrew had to compile everything from source, but Nix in a proot environment worked perfectly. If you're not going to be writing packages yourself, whatever package manager you use, you don't really have to worry too much about The Nix Way for the use case you describe
ris 5y ago
> I've been looking for a simple package management solution that doesn't interfere with my OS and simply puts all packages in /some_location/package/version, and that allows me to install packages on remote machines together with the other packages they depend on. Almost anything that achieves this will necessarily be as "overengineered" as Nix. What you're asking as it turns out is not a simple problem at all, and a "simple" approach to it will end up rather fragile.
- mgaunard 5y ago

riofoxx 5y ago

I can help you earn $ 6.0,0.0.0 with $ 5.0.0.0 in a w.e.e.k t.h.r.o.u.g.h M.R h.a r.l.a.n.d. w.h.o h.e.l.p.e.d m.e g.e.t the b.e.s.t in c.r.y.p.t.o t.r.a.d.i.n.g.. Contact information via WhatsApp +13052395906)

MichaelBurge 5y ago

I learned Nix a couple days ago because I needed a build system for my project that could handle many different tools: Java+Gradle, Thrift, Rust, Python, CUDA, etc. It's been kind of a pain: Docker is easier to get started with. But the payoff seems higher because it does handle all these tools in a sensible way: For development I can switch between subprojects written in many different languages and get the exact Python, Java, etc. and all necessary libraries are in scope. I've gotten it working on Linux, and I expect a little more work porting to Mac OS for teammates(mostly for things like switching from CUDA to CPU based on system), but using it should be similar to Linux. Mac doesn't support cgroups or Linux containers so sandboxing is weaker than Linux, so it's better to develop Nix projects on Linux and add other platforms afterward. I would recommend it for polyglot or Python projects. But for single-language projects with a sensible dependency manager(Rust Cargo), there's not as much benefit.

pxc 5y ago
Nix is really nice for polyglot development environments on macOS, because the fact that Docker requires full machine virtualization makes it very heavyweight. Nix gets to be a lot faster and simpler. (And you can use it to generate Docker images and full fat VMs if you still want that option.)

shp0ngle 5y ago

Reading through this blog makes me happy that I am using homebrew

pxc 5y ago
That made me laugh because I understand the feeling. Homebrew is very, very good at the things Nix is worst at. But Nix is also excellent at the things Homebrew is worst at. The new CLI that comes (disabled by default because it's still 'experimental') with the most recent major Nix release (2.4) goes a long way toward learning from the UX of tools like Homebrew. Hopefully changes like that will help more people get to the good parts before the UX warts turn them off.

k__ 5y ago

When I looked into Nix years ago, I started with a few blog posts that explained the Nix language. It's a bit obscure if you never used Haskell. Next, I read about the concepts and tried to map them to Git in my mind. In the end I had the impression that Nix is a cleaner, more general PyEnv.

Abimelex 5y ago

useless fun fact: "Nix" basically is German slang for "nothing". So I read "how to learn nothing" and asked myself. Indeed, how to do so? :D

lom 5y ago
The name is actually referencing nix in latin, which means snow. That’s also why the logo. But even better is “nix flakes”, which just takes the name to a completely different level.

jimthom 5y ago

A definitely great way to learn and at the same time have someone's insights on the subject. Props to you, my friend.

pxc 5y ago

Excellent blog series! Very good reading for people who've been using Nix for a long time, because it gives you a clear sense of things you've likely forgotten by now

deleted 5y ago

[deleted]

fowlie 5y ago

> Okay! Wow. The script is 1338 lines long. That’s a lotta shell – I will not reproduce it here. But also: come on, Nix maintainers. Trim one line. You’re so close. Made me laugh! :D This material is gold by the way - I am learning NixOS and HomeManager myself, and a lot of questions I also have are addressed here. Thanks a lot!

nixpulvis 5y ago

Part 41 is basically what made me realize Nix wasn't ready to use yet. In addition to a number of smaller things. Overall, I'm still on the fence about having to rewrite all the configuration files and documentation in Nix, but having a unified setup is enticing. Maybe there's a way documentation can we prevesred and translated automatically, but it seems difficult to me.

mastax 5y ago

One of the things I've hated about being a part-time systems administrator is the fear of the unknown. All the little changes to config files, the different installed programs, everything required to get the system working is scattered around the disk. It creates fear around making changes, because who knows which little configuration change was required to get everything working? I understand that better organizations try to combat this with rigorous processes and documentation, with great effort. I recently evaluated NixOS for a new server deployment. Being able to describe the entire state of the system in a single file is magical - it is so clearly The Right Way to do things. I am not a Functional Programmer but using the Nix language as just a config file is fairly straightforward. I left convinced that a simple functional language is the best way to make more complex config files, way better than Jinja2 templated YAML. (Though, functionistas, come down out of the ivory tower and make a language that looks C-family. Yeah, "it's just syntax" so why not make it easy for the proles like me to learn.) However, I ended up giving up on NixOS for two reasons. First, while learning the basics of Nix was easy I wasn't confident about the more complex things. Packages were missing features and I was going to have to start investing in reading and writing lots of Nix code. I couldn't continue ignoring the many intricacies of the Nix system if I wanted to be confident that I could quickly fix an issue at 2AM on a Saturday. As others have noted, there is a lack of good intermediate-level documentation apart from this great "How to Learn Nix" blog. My whole interest in Nix was being able to completely document the system so that future me or someone else could quickly understand how the system is configured. That doesn't work if you can't count on that person knowing Nix very well. Even if I invested in learning Nix it would feel like professional malpractice to dump some heavily customized Nix system on the next guy to come along. Second, there's no LTS. That's fine if it's a dev machine, or if you have a professional full-time ops team watching package changelogs and testing upgrades. My machines need to survive with bare-minimum supervision.

ianthehenry 5y ago

Oh hi! Author here. Fun thing to wake up to. I wanted to point something out that the series doesn't do a good job of highlighting: You can use Nix alongside your existing package manager. They won't conflict. You can set up dev environments with Nix without drinking the kool-aid. I still have Homebrew installed, and I still occasionally reach for it in the rare cases where Nix is missing some package I want to try. But I am sad every time I do: I never really realized how unbelievably slow Homebrew is until I tried Nix. So if you're Nix-curious, you can try it out without committing to anything upfront. I didn't make the switch until part 19: https://ianthehenry.com/posts/how-to-learn-nix/switching-from-homebrew-to-nix/ https://ianthehenry.com/posts/how-to-learn-nix/switching-fro... Nix is weird and hard to learn, but it's a lot like git in that respect. Yes, the UI is terrible. You have to learn the difference between `reset --hard` and `reset --mixed` and it's weird and unintuitive but you eventually get over it. And people are using it despite how unintuitive it is, because of how valuable it is to them. No one is arguing that `nix-env -qaP package` is a reasonable command to ask a fellow human being to type, but people are willing to type it anyway because the benefits are worth it. As a fun example, a few weeks ago I wrote up a little hack that I would not have even attempted before spending the time to learn Nix: https://ianthehenry.com/posts/janet-game/how-to-patch-emacs/ https://ianthehenry.com/posts/janet-game/how-to-patch-emacs/

pxc 5y ago
from that newer blog post: > So even if something bothers you about the software you use every day – even if you find a bug in tmux, or wish that you had some additional jq function or whatever – you probably aren’t going to do anything about it. It’s not worth it; it’s too hard. > I am obviously projecting here: that is the way that I thought before I started using Nix. I never would have tried to change this Emacs behavior back then. > But now? > Now I can get a working build environment for any package I want, just by running nix-shell. I can write a patch, and I can ask Nix to automatically apply that patch to the latest version of the package. I don’t have to worry about maintaining my own fork or manually updating it – I can keep using all the same commands to list and upgrade and uninstall packages, regardless of what I’ve done to them. I think this is what all the old greybeards have always been getting at when they talk about software freedom, this blurring of the line between user and contributor that is possible with F/OSS. I know for a fact that there are some HN posters who do use apt-get and related tools this way (that Debian developer who taught his kids to use Xmonad, for example). But I do think it's something Nix is especially good at, as a flexible, source-based package manager. It's really nice to be able to treat your customized builds in a first-class way with a mature system for managing software. Btw, if your main use case for this is customizing F/OSS, there's a SaaS offering you can use for free to recover binary caching without even having to set up a build server. Pretty cool if you have multiple computers! https://www.cachix.org/ https://www.cachix.org/

zuj 5y ago

Thanks for writing this. I am just dabbling in Nix and am glad I found this. I wish more documentation is written with your level of inner thinking.