6 ms·

I build container images using Nix (actually OCI containers; I don't use Docker at all, although I think it can run OCI containers). Going through these points

I build container images using Nix (actually OCI containers; I don't use Docker at all, although I think it can run OCI containers). Going through these points one at a time: > Use a smaller base image or libc By default, Nix will build container images from scratch. No base image (although I guess you can add one as a layer to your manifest JSON, if you like). > Make the image context smaller Nix tracks dependencies very precisely. Our container images will only contain the things we asked for, and their dependencies (transitively). > Fully minimize and tidy up image layers This only seems relevant for imperative images, built "from the inside" (e.g. the talk about contents being "overwritten"?) > Splitting up processes and services for breaking up images The helper-functions in Nixpkgs contain tricks to automatically figure out which content is better kept in separate layers, based on how many times they're referenced (as a proxy for how often they'll be shared across containers). Docker only supports 128 layers, so the remaining content gets combined into the final layer; e.g. see https://grahamc.com/blog/nix-and-layered-docker-images https://grahamc.com/blog/nix-and-layered-docker-images

secondcoming 5y ago
What has this got to do with making Docker images smaller?
- dwohnitmok 5y ago
  Presumably parent's point is "making Docker images with Nix makes them very small [smaller even than the tips mentioned in the article]."
  - yjftsjthsd-h 5y ago
    And it does so more or less by default, for free, without having to make any special effort. Edit: Any special effort beyond using nix, I should say; that is a significant cost if you're not already paying it.
- chriswarbo 5y ago
  > What has this got to do with making Docker images smaller? "Docker images" are .tar.gz files containing some binary executable. The article starts by putting an entire Linux distribution into the .tar.gz (Ubuntu, Alpine, etc.), then goes through several ways to try and make the result smaller. Nix does things the other way: the .tar.gz will only contain the specified binary, plus its dependencies. It's minimal by default (although there are still tricks to making things smaller, e.g. taking dependencies from `pkgsMusl` instead of `pkgs`, to avoid glibc)
astockwell 5y ago
Blogpost or it didn't happen! (Seriously though, I would love to read a post detailing this process start-to-finish for the uninitiated)
- emptysongglass 5y ago
  Here's [1] my favorite thus far using Nix's new flakes to build a Rust project into a container image with Nix. [1] https://community.fly.io/t/running-reproducible-rust-a-fly-and-nix-love-story/3781 https://community.fly.io/t/running-reproducible-rust-a-fly-a...
emptysongglass 5y ago
I've started building my personal projects like this but the end result is an image running as root, which is not best practice. Have you been able to build images with Nix that run as an arbitrary user?
- flurie 5y ago
  There is an example of doing this in the docker-tools examples! https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/docker/examples.nix#L449 https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...
Cloudef 5y ago
Nixos website has very nice example of minimal docker images: https://nixos.org/#asciinema-demo-example_4 https://nixos.org/#asciinema-demo-example_4
- Lutger 5y ago
  I get why nix is attractive, really, and its cool you can build docker images. But for people who don't know nix its a bit strange to say that Dockerfiles are tedious, and then proceed to show a much more arcane and verbose way of building them as a solution to that tediousness. The one thing that makes Dockerfiles attractive (and Docker in general), is that they are so simple and intuitive. You can immediately leverage your basic linux knowledge. That is why a lot of people like it whose main job is not about packaging.
  - licebmi__at__ 5y ago
    I think the keyword for that phrase is "reliable". Overtime, I had to go back to fix "working" base images that had become stale, and when dealing with layers and layers of base images this really gets tedious. BTW, I haven't dealt with nix generated images in production, I'm not aware of possible drawbacks (other than people not being familiar with nix), so I'm not really endorsing it, although I would like to try.
wereHamster 5y ago
I recently tried to use nix to build images (docker images though, not sure if k8s can run OCI, or if I can push OCI to a docker registry (artifactory)). What I found cumbersome though is to «cross-compile» images, I'm on mac/arm but the images need to run on linux/amd64. Ok my code is JavaScript (Node.js), I can compile the sources into plain JS on any machine, and then pack into an image together with nodejs from `pkgsLinux = import <nixpkgs> { system = "x86_64-linux"; };`, and all will work. I can even install node_modules on my build machine since the node_modules folder is (often) just a bunch of .js files. But some dependencies have native code, so I'd have to cross-compile those. And that's where I gave up.
- lilyball 5y ago
  I haven't attempted this yet but I have wondered about the feasibility of using the nixos/nix docker image just to build a linux/amd64 docker image using Nix. Or setting up a nixos/nix docker image as a remote builder for Nix.
- chriswarbo 5y ago
  > not sure if k8s can run OCI, or if I can push OCI to a docker registry (artifactory) I push OCI images to AWS ECR, so it would probably work. > I'm on mac/arm but the images need to run on linux/amd64 I've never used the cross-compilation support in Nixpkgs. My work machine runs macOS (x86_64), so I have a Linux VM as a remote builder. I happen to use LimaVM ( https://github.com/lima-vm/lima https://github.com/lima-vm/lima ), but anything would work (Qemu, VirtualBox, the VM bundled with Docker Desktop, etc.). PS: Since containers only work on Linux, and I'm often working on macOS, I use the following assertion to check that I'm not accidentally including Mac builds in a layer: assert all (d: ((d.type or null) == "derivation") -> (d.system == "x86_64-linux")) drvs;
didip 5y ago
But I have to learn nix script, which is not easy to learn.
- rgoulter 5y ago
  I'd say the common cases, (especially if someone has already done something similar that you can copy-paste from), are reasonably straightforward. Generally, getting a good understanding of Nix takes some time. It's hard because you need to understand what Nix does, and understand what you're trying to do, and how it constrains what you're trying to do.
  - exdsq 5y ago
    I used Nix at work but eventually realized, managing dependencies, compiling cross-platform, and deploying wasn’t actually bad enough to warrant that level of complexity

secondcoming 5y ago

What has this got to do with making Docker images smaller?

dwohnitmok 5y ago
Presumably parent's point is "making Docker images with Nix makes them very small [smaller even than the tips mentioned in the article]."
- yjftsjthsd-h 5y ago
  And it does so more or less by default, for free, without having to make any special effort. Edit: Any special effort beyond using nix, I should say; that is a significant cost if you're not already paying it.
chriswarbo 5y ago
> What has this got to do with making Docker images smaller? "Docker images" are .tar.gz files containing some binary executable. The article starts by putting an entire Linux distribution into the .tar.gz (Ubuntu, Alpine, etc.), then goes through several ways to try and make the result smaller. Nix does things the other way: the .tar.gz will only contain the specified binary, plus its dependencies. It's minimal by default (although there are still tricks to making things smaller, e.g. taking dependencies from `pkgsMusl` instead of `pkgs`, to avoid glibc)

astockwell 5y ago

Blogpost or it didn't happen! (Seriously though, I would love to read a post detailing this process start-to-finish for the uninitiated)

emptysongglass 5y ago
Here's [1] my favorite thus far using Nix's new flakes to build a Rust project into a container image with Nix. [1] https://community.fly.io/t/running-reproducible-rust-a-fly-and-nix-love-story/3781 https://community.fly.io/t/running-reproducible-rust-a-fly-a...

emptysongglass 5y ago

I've started building my personal projects like this but the end result is an image running as root, which is not best practice. Have you been able to build images with Nix that run as an arbitrary user?

flurie 5y ago
There is an example of doing this in the docker-tools examples! https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/docker/examples.nix#L449 https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...

Cloudef 5y ago

Nixos website has very nice example of minimal docker images: https://nixos.org/#asciinema-demo-example_4 https://nixos.org/#asciinema-demo-example_4

Lutger 5y ago
I get why nix is attractive, really, and its cool you can build docker images. But for people who don't know nix its a bit strange to say that Dockerfiles are tedious, and then proceed to show a much more arcane and verbose way of building them as a solution to that tediousness. The one thing that makes Dockerfiles attractive (and Docker in general), is that they are so simple and intuitive. You can immediately leverage your basic linux knowledge. That is why a lot of people like it whose main job is not about packaging.
- licebmi__at__ 5y ago
  I think the keyword for that phrase is "reliable". Overtime, I had to go back to fix "working" base images that had become stale, and when dealing with layers and layers of base images this really gets tedious. BTW, I haven't dealt with nix generated images in production, I'm not aware of possible drawbacks (other than people not being familiar with nix), so I'm not really endorsing it, although I would like to try.

wereHamster 5y ago

I recently tried to use nix to build images (docker images though, not sure if k8s can run OCI, or if I can push OCI to a docker registry (artifactory)). What I found cumbersome though is to «cross-compile» images, I'm on mac/arm but the images need to run on linux/amd64. Ok my code is JavaScript (Node.js), I can compile the sources into plain JS on any machine, and then pack into an image together with nodejs from `pkgsLinux = import <nixpkgs> { system = "x86_64-linux"; };`, and all will work. I can even install node_modules on my build machine since the node_modules folder is (often) just a bunch of .js files. But some dependencies have native code, so I'd have to cross-compile those. And that's where I gave up.

lilyball 5y ago
I haven't attempted this yet but I have wondered about the feasibility of using the nixos/nix docker image just to build a linux/amd64 docker image using Nix. Or setting up a nixos/nix docker image as a remote builder for Nix.
chriswarbo 5y ago
> not sure if k8s can run OCI, or if I can push OCI to a docker registry (artifactory) I push OCI images to AWS ECR, so it would probably work. > I'm on mac/arm but the images need to run on linux/amd64 I've never used the cross-compilation support in Nixpkgs. My work machine runs macOS (x86_64), so I have a Linux VM as a remote builder. I happen to use LimaVM ( https://github.com/lima-vm/lima https://github.com/lima-vm/lima ), but anything would work (Qemu, VirtualBox, the VM bundled with Docker Desktop, etc.). PS: Since containers only work on Linux, and I'm often working on macOS, I use the following assertion to check that I'm not accidentally including Mac builds in a layer: assert all (d: ((d.type or null) == "derivation") -> (d.system == "x86_64-linux")) drvs;

didip 5y ago

But I have to learn nix script, which is not easy to learn.

rgoulter 5y ago
I'd say the common cases, (especially if someone has already done something similar that you can copy-paste from), are reasonably straightforward. Generally, getting a good understanding of Nix takes some time. It's hard because you need to understand what Nix does, and understand what you're trying to do, and how it constrains what you're trying to do.
- exdsq 5y ago
  I used Nix at work but eventually realized, managing dependencies, compiling cross-platform, and deploying wasn’t actually bad enough to warrant that level of complexity