4 ms·

i replaced cloud autoscaled docker runners with fixed set of machines with nix store cache pee machine. 10x imrovement.

grhmc 3y ago
Yep, that works great for cases where you trust all your contributors. We do the same for some projects internally, too. However, for public projects where you don't it gets a bit ... dicier. Glad to hear that's working for you!
- dlahoda 3y ago
  we have public oss project. i guess we were not attacked yet. prs from forks run only main workflows, and these run nix which is kind of isolated enough. i guess one could attack with some infinite nix store bomb.
esafak 3y ago
Are you saying you replaced Docker with Nix? I am still trying to understand the use case for Nix. Can someone explain? Is it a complete replacement for Docker?
- rgoulter 3y ago
  > Is it a complete replacement for Docker? Docker mainly does two things: there's Docker images as a way of sharing some container-image, and the container runtime to allow running those images for tasks or services in isolated/fresh ways. Docker images make it easy to distribute software which runs the same everywhere. Nix is a package manager which tackles that problem, but without using container images: Nix is for distributing software so it has the same behaviour everywhere. Nix users are often very enthusiastic about Nix because it also enables all sorts of neat developer experiences. e.g. Nix is great for setting up development environments. In terms of Nix-vs-Docker, Nix is also capable of building Docker/OCI images. So, you could use Nix instead of writing a Dockerfile.
  - dusanh 3y ago
    Does it make sense to use nix during the image build process to precisely define dependencies I want to have in the resulting container? Or perhaps in other words, can I use nix and docker together, to have a precisely defined environment I can share with others devs as a result?
    - dlahoda 3y ago
      yes. nix builds linux filesystem with all dependencies and puts it into oci image. so locally i use process-compose to avoiding waiting docker builds.
      - dusanh 3y ago
        Hm, I skimmed the github page for process-compose and I don't understand how it is different from building the container image directly with nix.
    - rgoulter 3y ago
      > Does it make sense to use nix during the image build process to precisely define dependencies I want to have in the resulting container? Using a dockerfile with a step like "RUN nix ..."? You can, but it strikes me as a cumbersome way of doing things. Mitchell H describes this way of doing things here: https://mitchellh.com/writing/nix-with-dockerfiles https://mitchellh.com/writing/nix-with-dockerfiles Whereas, I'd reckon the more idiomatic thing to do is to build the Docker image with Nix code. -- You're going to get a precisely defined environment to share across workstation/CI/etc. Some example Nix code for building Docker images is here https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/docker/examples.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...
      - dusanh 3y ago
        Thanks for the links, that is very useful! It seems like I was thinking about this process the other way around.
- takeda 3y ago
  It's really hard to explain it because Nix is a paradigm shift (I think that's why it is hard initially grasp it). But you can use it for things like: - declare your application with all dependencies explicitly, so when someone else wants to build it they can (I would argue this is the primary purpose and rest is just built on top of that) - common dev environment (so other developers can get the same dev environment as you with all exact same build tools) - build toolchain (for example if you do embedded environment) - you could use it as a replacement for homebrew/mac ports - a configuration file holding your .dot files (home manager) - if you use NixOS (OS that was built around Nix) then you have OS with a built-in configuration management (i.e. salt/puppet/ansible/chef) that is truly declarative I think this[1] also shows some crazy stuff you can do with it. Regarding question around Docker, the great thing is that if you define your application as a nix derivation, then you can easily generate docker that just contains your application and dependencies (the docker in that case is just a deployment unit). The reproducible environment is what docker promised, but practically failed to deliver. Instead of deliver reproducibility, it actually delivered repeatability. [1] https://youtu.be/6Le0IbPRzOE?t=109 https://youtu.be/6Le0IbPRzOE?t=109
  - esafak 3y ago
    Thank you for the explanation. Let me see if I understood. The basic draw is that it offers a programming language-agnostic specification for dependency management, replacing language-specific specifications like requirements.txt. Then you pair it with a containerization technology like Docker or Podman. And if you use NixOS, you can skip the last part?
    - takeda 3y ago
      > Thank you for the explanation. Let me see if I understood. The basic draw is that it offers a programming language-agnostic specification for dependency management, replacing language-specific specifications like requirements.txt Yes, actually I got interested in Nix because with requirements.txt I could only define python dependencies, and I had no control over for example installing postgresql C library that psycopg2 depends on. > Then you pair it with a containerization technology like Docker or Podman You don't have to, but you can, given that in most places containers are being used a lot of people use nix that way. > And if you use NixOS, you can skip the last part? Yes, although keep in mind that for example the requirement is to use Kubernetes then you would have to use Kubernetes. But if you need to create for example an EC2 instance. You can use Ubuntu + ansible or you could use NixOS. Honestly I don't have much experience with NixOS as at my workplace we are mandated to use specific distro for everything. Edit: I forgot to add additional benefits with using NixOS compared to Ubuntu + ansible for example. All updates are atomic. You either end up with the new configuration or the old configuration, there's no in-between as it would happen with ansible. Second big benefit is easy way to rollback.

Yep, that works great for cases where you trust all your contributors. We do the same for some projects internally, too. However, for public projects where you don't it gets a bit ... dicier. Glad to hear that's working for you!

dlahoda 3y ago
we have public oss project. i guess we were not attacked yet. prs from forks run only main workflows, and these run nix which is kind of isolated enough. i guess one could attack with some infinite nix store bomb.

esafak 3y ago

Are you saying you replaced Docker with Nix? I am still trying to understand the use case for Nix. Can someone explain? Is it a complete replacement for Docker?

rgoulter 3y ago
> Is it a complete replacement for Docker? Docker mainly does two things: there's Docker images as a way of sharing some container-image, and the container runtime to allow running those images for tasks or services in isolated/fresh ways. Docker images make it easy to distribute software which runs the same everywhere. Nix is a package manager which tackles that problem, but without using container images: Nix is for distributing software so it has the same behaviour everywhere. Nix users are often very enthusiastic about Nix because it also enables all sorts of neat developer experiences. e.g. Nix is great for setting up development environments. In terms of Nix-vs-Docker, Nix is also capable of building Docker/OCI images. So, you could use Nix instead of writing a Dockerfile.
- dusanh 3y ago
  Does it make sense to use nix during the image build process to precisely define dependencies I want to have in the resulting container? Or perhaps in other words, can I use nix and docker together, to have a precisely defined environment I can share with others devs as a result?
  - dlahoda 3y ago
    yes. nix builds linux filesystem with all dependencies and puts it into oci image. so locally i use process-compose to avoiding waiting docker builds.
    - dusanh 3y ago
      Hm, I skimmed the github page for process-compose and I don't understand how it is different from building the container image directly with nix.
  - rgoulter 3y ago
    > Does it make sense to use nix during the image build process to precisely define dependencies I want to have in the resulting container? Using a dockerfile with a step like "RUN nix ..."? You can, but it strikes me as a cumbersome way of doing things. Mitchell H describes this way of doing things here: https://mitchellh.com/writing/nix-with-dockerfiles https://mitchellh.com/writing/nix-with-dockerfiles Whereas, I'd reckon the more idiomatic thing to do is to build the Docker image with Nix code. -- You're going to get a precisely defined environment to share across workstation/CI/etc. Some example Nix code for building Docker images is here https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/docker/examples.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...
    - dusanh 3y ago
      Thanks for the links, that is very useful! It seems like I was thinking about this process the other way around.
takeda 3y ago
It's really hard to explain it because Nix is a paradigm shift (I think that's why it is hard initially grasp it). But you can use it for things like: - declare your application with all dependencies explicitly, so when someone else wants to build it they can (I would argue this is the primary purpose and rest is just built on top of that) - common dev environment (so other developers can get the same dev environment as you with all exact same build tools) - build toolchain (for example if you do embedded environment) - you could use it as a replacement for homebrew/mac ports - a configuration file holding your .dot files (home manager) - if you use NixOS (OS that was built around Nix) then you have OS with a built-in configuration management (i.e. salt/puppet/ansible/chef) that is truly declarative I think this[1] also shows some crazy stuff you can do with it. Regarding question around Docker, the great thing is that if you define your application as a nix derivation, then you can easily generate docker that just contains your application and dependencies (the docker in that case is just a deployment unit). The reproducible environment is what docker promised, but practically failed to deliver. Instead of deliver reproducibility, it actually delivered repeatability. [1] https://youtu.be/6Le0IbPRzOE?t=109 https://youtu.be/6Le0IbPRzOE?t=109
- esafak 3y ago
  Thank you for the explanation. Let me see if I understood. The basic draw is that it offers a programming language-agnostic specification for dependency management, replacing language-specific specifications like requirements.txt. Then you pair it with a containerization technology like Docker or Podman. And if you use NixOS, you can skip the last part?
  - takeda 3y ago
    > Thank you for the explanation. Let me see if I understood. The basic draw is that it offers a programming language-agnostic specification for dependency management, replacing language-specific specifications like requirements.txt Yes, actually I got interested in Nix because with requirements.txt I could only define python dependencies, and I had no control over for example installing postgresql C library that psycopg2 depends on. > Then you pair it with a containerization technology like Docker or Podman You don't have to, but you can, given that in most places containers are being used a lot of people use nix that way. > And if you use NixOS, you can skip the last part? Yes, although keep in mind that for example the requirement is to use Kubernetes then you would have to use Kubernetes. But if you need to create for example an EC2 instance. You can use Ubuntu + ansible or you could use NixOS. Honestly I don't have much experience with NixOS as at my workplace we are mandated to use specific distro for everything. Edit: I forgot to add additional benefits with using NixOS compared to Ubuntu + ansible for example. All updates are atomic. You either end up with the new configuration or the old configuration, there's no in-between as it would happen with ansible. Second big benefit is easy way to rollback.