13 ms·

This article is a good explanation of how nix works at a high level, and I'm excited to see nix getting some really prominent support, but for some reason it ne

by nilkn 6y ago

This article is a good explanation of how nix works at a high level, and I'm excited to see nix getting some really prominent support, but for some reason it never tells you what the point of all of this is, so I think many folks might feel turned off by it. In other words, I don't believe it ever compellingly answers the question that constitutes its title. The word "package" doesn't even appear until near the end of the article! This is probably a second or third article to read about nix rather than the very first thing that exposes you to it. The point of nix is just to create completely reproducible builds and package management, including support for multiple versions of packages side-by-size with no issues. It's sort of a next-generation package management system that tries to avoid most of the pitfalls that OS package managers have fumbled with up to this point. It's really that simple. "nix" as a term refers to a system of multiple components that work together to achieve that goal. There's the package manager itself (called Nix), the language that build instructions are written in (also called Nix), an existing ecosystem of predefined packages (called nixpkgs), and an optional Linux distro that uses Nix the package manager as its package manager (called NixOS).

inetknght 6y ago
So... statically compiled executables? Wasn't that tried long ago and it was determined that the user should be able to choose when to upgrade dependencies, such as if a dependency needs an out-of-band update to work on the localhost OS?
- smilliken 6y ago
  Your criticism misses the mark because nix users have the ability to update a dependency and rebuild all of the dependees. With nix, I can update openssl in one place and be sure that everything that depends on it gets re-evaluated. How can I be confident that everything is linking the patched openssl I want when I'm using aptitude, pip, npm, docker, etc?
  - ComodoHacker 6y ago
    But what if they're using different versions of OpenSSL?
    - ris 6y ago
      It'll only rebuild the packages that depended on that particular openssl. This is an area where Nix shines, because all packages are explicitly bound to their dependencies, it means it's no longer relevant what one file happens to be occupying `/usr/lib/libssl.so`, or even `/lib/x86_64-linux-gnu/libc.so.6`, so you can be running different apps that rely on totally different glibc versions alongside each other with no problems.
      - ComodoHacker 6y ago
        I mean if there's a bug, it's not enough to patch one particular OpenSSL. You have to audit manually, so Nix won't make much of a difference.
        takeda 6y ago
        Well, by default it uses the same library everywhere, but it gives you an option that you can have two apps that rely on e.g. different version of openssl. If you do that it is on you to make sure both dependencies are updated. There is a bit of benefit as well. For example instead of creating a new derivation, you can override existing one (kind of like you extend a class in OO) if for example you make another version of openssl based off the existing one, but maybe changed compile flags, then nix will be smart enough to recompile both.
        bb010g 6y ago
        Except the auditing doesn't have to be manual: https://github.com/flyingcircusio/vulnix https://github.com/flyingcircusio/vulnix
      - zerd 6y ago
        Oh, that sounds like it could solve my issue. E.g. I compiled my program against my glibc which is 2.24, but target was running 2.23. I didn't even use any "new" 2.24 features, but it was missing a symbol, so it wouldn't run. But then to compile against 2.23 locally, I had to get an older version of GCC (not sure why, but), and everything needed to build the older GCC, and so on. But then I got some old versions of libraries on my system, so I had to delete them. I ended up compiling it all inside a docker container because I didn't want to pollute my env with older versions.
- ben0x539 6y ago
  The user should use nix to do that.
- ex_amazon_sde 6y ago
  Good point: statically compiled binaries are a big problem for security updates. This applies to Linux distributions as well as large organizations that have their internal distribution (like Amazon).
nerdponx 6y ago
What I'd really like to see is a realistic, end-to-end tutorial for either 1) deploying a relatively straightforward web application (like Dokuwiki or ZNC), or 2) setting up a basic desktop for day-to-day use. I feel like I've seen a lot of "snippets", I feel like I understand how Nix works and what it's supposed to be good for, but I don't have a coherent sense of the steps involved in actually using it for mundane things.
- Ericson2314 6y ago
  Look for nix users' "this is not a dotfiles repo" repo on GitHub.
  - deleted 6y ago
    [deleted]
  - nerdponx 6y ago
    Care to clarify and/or add a link? I'm not sure what this means. (Edit: I did not downvote you)
    - Ericson2314 6y ago
      https://github.com/ihebchagra/dotfiles https://github.com/ihebchagra/dotfiles I guess sometimes they are called dotfiles haha. https://www.google.com/search?q=personal+config+github+nix https://www.google.com/search?q=personal+config+github+nix seem to be a good search Perhaps I misremembered how many of them make throw shade on dotfiles repos, or perhaps it is just that google isn't good at finding such things ("not dotfiles" won't work).
- yakshaving_jgt 6y ago
  It's a bit old now, but not that much has changed. All my commercial projects use essentially this method of deployment. https://jezenthomas.com/deploying-a-haskell-web-service-with-nix/ https://jezenthomas.com/deploying-a-haskell-web-service-with...
- edynoid 6y ago
cldellow 6y ago
The PhD thesis that goes with Nix is a really great intro to it. It's very accessible and well-structured. It motivates the need for a new take on package management by analogy to memory management in programming languages. It contrasts how software deployment works today (“Fuck it, I’m sure /usr/bin/local/python is a reference to a python 2.7.12 install with PIL available”) to how we used to write software (“Fuck it, I’m sure 0x7FEA6072 is the address of the array of files; so clearly, assuming a 32-bit address space, the seventh file is at 0x7FEA608A.”) In both cases, as long as your assumption was correct, things go swimmingly. But it could be easier and rely less on hoping your assumptions are correct, and more on things that are verifiably true. And that's what Nix offers: a way to build software that is insulated against assumptions and "it works on my machine". Thesis: https://edolstra.github.io/pubs/phd-thesis.pdf https://edolstra.github.io/pubs/phd-thesis.pdf
- amelius 6y ago
  The downside is that the approach works best when packages are aware of the Nix approach. But the packages have not been written with Nix in mind, and some work is needed per package to adapt to the approach.
  - Ericson2314 6y ago
    Nixpkgs has already done much of that work for you, thankfully.
    - amelius 6y ago
      Yes that's true. But some things remain difficult, for example installing the latest version of CUDA. https://discourse.nixos.org/t/cuda-setup-on-nixos/1118/9 https://discourse.nixos.org/t/cuda-setup-on-nixos/1118/9
      - ninjin 6y ago
        Yes, absolutely. Although to be fair, CUDA is a royal pain (although way easier) even on more standard distributions (supporting multiple versions seamlessly is hours and hours of fun and breaks all too easily) and is behind the great majority of time “wasted”. NVIDIA deserves a lot more flak for this than they receive.
ironmagma 6y ago
Isn’t that the same goal as Docker? I’m surprised there’s still no Docker base image for NixOS...
- rgoulter 6y ago
  "Same execution environment everywhere" is one way which developers use Docker. Docker gets this by copying the layers of a built image. Unlike nix, the image building itself doesn't need to be reproducible. So you can have a Dockerfile which works now but will fail to build in however many months. "Reproducible builds" do get you "same execution environment everywhere". But they have the stronger guarantee that for the same inputs, outputs will be the same. IMO/IME, I don't think that aspect of nix is a strong selling point for use of nix on developer workstations. Probably thanks to less-elegant solutions like "<language> Version Manager" etc.. But I think the nix language makes for a nicer way of describing a package of software you're developing in terms of dependencies and outputs than Dockerfile.
  - tomberek 6y ago
    I do have to commend Docker for providing and managing an agreed upon VM for non-Linux users to host all their containers. It's the "killer-feature" that has made it as successful as it is. But underneath it requires a VM (libcontainer,LXC,virtualbox,hyperkit,etc) on non-Linux machines. This helps developers work together and quickly get small projects up and running. I'd contend that after a while, a mess of containers/sidecars ends up becoming just as difficult to manage as a mess of native binaries. Hence the growth of so many container management systems. Now, because they are re-inventions of service managers we get the benefit of designing them from scratch for modern needs, but also loose many of the benefits of well understood semantics of native processes. Looking for feedback: I've been playing with an idea (and have a system in production using it to try out the concept) where the Dockerfile only contains busybox+nix and you when you run it you specify an environment as a Nix path. Specify a binary cache via env vars. Using "nix run" this will download all deps and run your program, with bind mounts all containers can share the host cache. Put a RUN into the Dockerfile and you can prefetch all the deps. Basically it's a Docker container that uses Nix at build or run time for all the heavy lifting, instead of the docker layers mechanism.
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
lloeki 6y ago
> for some reason it never tells you what the point of all of this is, so I think many folks might feel turned off by it I’m attempting to write some hands-on pragmatic no-bullshit articles tackling that. Why? I’ve been hearing a lot about Nix over the years, how it’s an experiment yet so good it’s actually usable. But every article I found obscure the good parts, either because of too much details and theory or because they’re apt->nix-env but without the whys. And the good parts are really good. So good that as soon as I was enlightened I simply dropped my lonesome 13 years ArchMac project on the floor. But it needs a pragmatic approach to explaining what it is and how it can be of value to you day to day. Everything is actually in the manual, but it lacks some ties with existing knowledge for people to make the jump, unless they’re really really curious enough to piece things together. That’s what I’m writing.
- takeda 6y ago
  That sounds great. I remeber when I started, it was very difficult, I don't think I would be able to start without nix-pills articles. Nix needs more tutorials, especially in areas that could showcase it (which you planning to do). Nix is often referred as a package manager, but I believe that characterization does a bit of disservice to it, because it can do much more than that. IMO the areas where it excels the most for me is the build system. Ability to have a language define the exact environment that developer has is IMO awesome.
Breza 6y ago
Thank you for the explanation! I'd never heard of nix and I was confused by the article.

by nilkn 6y ago

inetknght 6y ago

So... statically compiled executables? Wasn't that tried long ago and it was determined that the user should be able to choose when to upgrade dependencies, such as if a dependency needs an out-of-band update to work on the localhost OS?

smilliken 6y ago
Your criticism misses the mark because nix users have the ability to update a dependency and rebuild all of the dependees. With nix, I can update openssl in one place and be sure that everything that depends on it gets re-evaluated. How can I be confident that everything is linking the patched openssl I want when I'm using aptitude, pip, npm, docker, etc?
- ComodoHacker 6y ago
  But what if they're using different versions of OpenSSL?
  - ris 6y ago
    It'll only rebuild the packages that depended on that particular openssl. This is an area where Nix shines, because all packages are explicitly bound to their dependencies, it means it's no longer relevant what one file happens to be occupying `/usr/lib/libssl.so`, or even `/lib/x86_64-linux-gnu/libc.so.6`, so you can be running different apps that rely on totally different glibc versions alongside each other with no problems.
    - ComodoHacker 6y ago
      I mean if there's a bug, it's not enough to patch one particular OpenSSL. You have to audit manually, so Nix won't make much of a difference.
      - takeda 6y ago
        Well, by default it uses the same library everywhere, but it gives you an option that you can have two apps that rely on e.g. different version of openssl. If you do that it is on you to make sure both dependencies are updated. There is a bit of benefit as well. For example instead of creating a new derivation, you can override existing one (kind of like you extend a class in OO) if for example you make another version of openssl based off the existing one, but maybe changed compile flags, then nix will be smart enough to recompile both.
      - bb010g 6y ago
        Except the auditing doesn't have to be manual: https://github.com/flyingcircusio/vulnix https://github.com/flyingcircusio/vulnix
    - zerd 6y ago
      Oh, that sounds like it could solve my issue. E.g. I compiled my program against my glibc which is 2.24, but target was running 2.23. I didn't even use any "new" 2.24 features, but it was missing a symbol, so it wouldn't run. But then to compile against 2.23 locally, I had to get an older version of GCC (not sure why, but), and everything needed to build the older GCC, and so on. But then I got some old versions of libraries on my system, so I had to delete them. I ended up compiling it all inside a docker container because I didn't want to pollute my env with older versions.
ben0x539 6y ago
The user should use nix to do that.
ex_amazon_sde 6y ago
Good point: statically compiled binaries are a big problem for security updates. This applies to Linux distributions as well as large organizations that have their internal distribution (like Amazon).

nerdponx 6y ago

What I'd really like to see is a realistic, end-to-end tutorial for either 1) deploying a relatively straightforward web application (like Dokuwiki or ZNC), or 2) setting up a basic desktop for day-to-day use. I feel like I've seen a lot of "snippets", I feel like I understand how Nix works and what it's supposed to be good for, but I don't have a coherent sense of the steps involved in actually using it for mundane things.

Ericson2314 6y ago
Look for nix users' "this is not a dotfiles repo" repo on GitHub.
- deleted 6y ago
  [deleted]
- nerdponx 6y ago
  Care to clarify and/or add a link? I'm not sure what this means. (Edit: I did not downvote you)
  - Ericson2314 6y ago
    https://github.com/ihebchagra/dotfiles https://github.com/ihebchagra/dotfiles I guess sometimes they are called dotfiles haha. https://www.google.com/search?q=personal+config+github+nix https://www.google.com/search?q=personal+config+github+nix seem to be a good search Perhaps I misremembered how many of them make throw shade on dotfiles repos, or perhaps it is just that google isn't good at finding such things ("not dotfiles" won't work).
yakshaving_jgt 6y ago
It's a bit old now, but not that much has changed. All my commercial projects use essentially this method of deployment. https://jezenthomas.com/deploying-a-haskell-web-service-with-nix/ https://jezenthomas.com/deploying-a-haskell-web-service-with...
edynoid 6y ago

cldellow 6y ago

The PhD thesis that goes with Nix is a really great intro to it. It's very accessible and well-structured. It motivates the need for a new take on package management by analogy to memory management in programming languages. It contrasts how software deployment works today (“Fuck it, I’m sure /usr/bin/local/python is a reference to a python 2.7.12 install with PIL available”) to how we used to write software (“Fuck it, I’m sure 0x7FEA6072 is the address of the array of files; so clearly, assuming a 32-bit address space, the seventh file is at 0x7FEA608A.”) In both cases, as long as your assumption was correct, things go swimmingly. But it could be easier and rely less on hoping your assumptions are correct, and more on things that are verifiably true. And that's what Nix offers: a way to build software that is insulated against assumptions and "it works on my machine". Thesis: https://edolstra.github.io/pubs/phd-thesis.pdf https://edolstra.github.io/pubs/phd-thesis.pdf

amelius 6y ago
The downside is that the approach works best when packages are aware of the Nix approach. But the packages have not been written with Nix in mind, and some work is needed per package to adapt to the approach.
- Ericson2314 6y ago
  Nixpkgs has already done much of that work for you, thankfully.
  - amelius 6y ago
    Yes that's true. But some things remain difficult, for example installing the latest version of CUDA. https://discourse.nixos.org/t/cuda-setup-on-nixos/1118/9 https://discourse.nixos.org/t/cuda-setup-on-nixos/1118/9
    - ninjin 6y ago
      Yes, absolutely. Although to be fair, CUDA is a royal pain (although way easier) even on more standard distributions (supporting multiple versions seamlessly is hours and hours of fun and breaks all too easily) and is behind the great majority of time “wasted”. NVIDIA deserves a lot more flak for this than they receive.

ironmagma 6y ago

Isn’t that the same goal as Docker? I’m surprised there’s still no Docker base image for NixOS...

rgoulter 6y ago
"Same execution environment everywhere" is one way which developers use Docker. Docker gets this by copying the layers of a built image. Unlike nix, the image building itself doesn't need to be reproducible. So you can have a Dockerfile which works now but will fail to build in however many months. "Reproducible builds" do get you "same execution environment everywhere". But they have the stronger guarantee that for the same inputs, outputs will be the same. IMO/IME, I don't think that aspect of nix is a strong selling point for use of nix on developer workstations. Probably thanks to less-elegant solutions like "<language> Version Manager" etc.. But I think the nix language makes for a nicer way of describing a package of software you're developing in terms of dependencies and outputs than Dockerfile.
- tomberek 6y ago
  I do have to commend Docker for providing and managing an agreed upon VM for non-Linux users to host all their containers. It's the "killer-feature" that has made it as successful as it is. But underneath it requires a VM (libcontainer,LXC,virtualbox,hyperkit,etc) on non-Linux machines. This helps developers work together and quickly get small projects up and running. I'd contend that after a while, a mess of containers/sidecars ends up becoming just as difficult to manage as a mess of native binaries. Hence the growth of so many container management systems. Now, because they are re-inventions of service managers we get the benefit of designing them from scratch for modern needs, but also loose many of the benefits of well understood semantics of native processes. Looking for feedback: I've been playing with an idea (and have a system in production using it to try out the concept) where the Dockerfile only contains busybox+nix and you when you run it you specify an environment as a Nix path. Specify a binary cache via env vars. Using "nix run" this will download all deps and run your program, with bind mounts all containers can share the host cache. Put a RUN into the Dockerfile and you can prefetch all the deps. Basically it's a Docker container that uses Nix at build or run time for all the heavy lifting, instead of the docker layers mechanism.

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

lloeki 6y ago

> for some reason it never tells you what the point of all of this is, so I think many folks might feel turned off by it I’m attempting to write some hands-on pragmatic no-bullshit articles tackling that. Why? I’ve been hearing a lot about Nix over the years, how it’s an experiment yet so good it’s actually usable. But every article I found obscure the good parts, either because of too much details and theory or because they’re apt->nix-env but without the whys. And the good parts are really good. So good that as soon as I was enlightened I simply dropped my lonesome 13 years ArchMac project on the floor. But it needs a pragmatic approach to explaining what it is and how it can be of value to you day to day. Everything is actually in the manual, but it lacks some ties with existing knowledge for people to make the jump, unless they’re really really curious enough to piece things together. That’s what I’m writing.

takeda 6y ago
That sounds great. I remeber when I started, it was very difficult, I don't think I would be able to start without nix-pills articles. Nix needs more tutorials, especially in areas that could showcase it (which you planning to do). Nix is often referred as a package manager, but I believe that characterization does a bit of disservice to it, because it can do much more than that. IMO the areas where it excels the most for me is the build system. Ability to have a language define the exact environment that developer has is IMO awesome.

Breza 6y ago

Thank you for the explanation! I'd never heard of nix and I was confused by the article.