5 ms·

Optimising Docker Layers for Better Caching with Nix

137 points by grhmc 8y ago

tazjin 8y ago
Great post on the issues of Docker's linear layering method (which a tweet[1] by the author of the post also explains well independently of computers) and how to improve on it with Nix. The "old" way of building Docker images with Nix also provided some of these benefits, but for quick builds often ended up being slower due to the dependency on qemu. This is an awesome improvement on that! I'd recommend people building & deploying a lot of Docker images (especially in languages with many dependencies on native libraries and such, which can be tricky to get right but which Nix mostly just handles for you) to see if building images with Nix could improve your workflow! [1]: https://twitter.com/grhmc/status/1046529919897886720 https://twitter.com/grhmc/status/1046529919897886720
manveru 8y ago
Tried this out on the day it was released, and it's been awesome so far. Usually structuring images would take a lot of manual tweaking to get the layers just right, but with this I just adjust the number of layers and watch magic happen :) Average push size down from over 300MB to 15MB for an image of over 1.4GB, and takes like 15 lines of code to write.
zimbatm 8y ago
Nix and Docker integration is almost optimal now, no other tooling provides that type of functionality
- jarvuschris 8y ago
  [Habitat](http://habitat.sh/ http://habitat.sh/) builds on the concepts in Nix, and here is a relatively old experiment I found on optimizing Docker layering with it: https://github.com/chetan/habitat-dockerize-layers https://github.com/chetan/habitat-dockerize-layers
  - roberthensing 8y ago
    What I like about Nix is that this kind of functionality can be integrated into the Nixpkgs package set, as a function over packages. It's just as easy as writing a machine image or an virtual machine network test, because Nixpkgs has functions you can use for those as well.
costrouc 8y ago
I really like this. I think it is a great compromise between the limitations of docker and everything packaged in one file while taking advantage of nix. I'm curious about one other optimization that could be made. You could instead look at the dependency graph of several nix docker builds and increase the cache reuse even more from that.
- roberthensing 8y ago
  This approach is already optimal for up to around 40 or 120 packages, depending on whether you want to support old Docker versions. "Where before you wouldn’t bother trying to have your application and database images share layers, with Nix the layer sharing is completely automatic."
kjullien 8y ago
Wouldn't a multi-stage dockerfile fix the problems of having N layers or is it N for the entire file?
tlrobinson 8y ago
Neat. It seems like Nix could become the preferred way to build Docker images, if it's made a bit more user friendly. It's hard to beat the simplicity of FROM ubuntu:latest RUN apt-get update RUN apt-get install -y mysql php
- jf 8y ago
  I'm very optimistic about Nix. If you haven't read it yet, I suggest starting with this article: https://www.infoq.com/articles/configuration-management-with-nix https://www.infoq.com/articles/configuration-management-with... I also gave a talk about why I'm excited about Nix: https://vimeo.com/223525975 https://vimeo.com/223525975
- notthemessiah 8y ago
  "Simplest" can be subjective. There's not much that can compete with editing your nixos config file and adding services.mattermost.enable = true; and then running # nixos-rebuild switch to have a mattermost instance running.
  - tlrobinson 8y ago
    I'm specifically talk about building a Docker container. The example in the article has a couple dozen lines of nix config and several shell commands.
prisar 8y ago
Docker layers for windows bases images are of huge sizes

137 points by grhmc 8y ago

Great post on the issues of Docker's linear layering method (which a tweet[1] by the author of the post also explains well independently of computers) and how to improve on it with Nix. The "old" way of building Docker images with Nix also provided some of these benefits, but for quick builds often ended up being slower due to the dependency on qemu. This is an awesome improvement on that! I'd recommend people building & deploying a lot of Docker images (especially in languages with many dependencies on native libraries and such, which can be tricky to get right but which Nix mostly just handles for you) to see if building images with Nix could improve your workflow! [1]: https://twitter.com/grhmc/status/1046529919897886720 https://twitter.com/grhmc/status/1046529919897886720

manveru 8y ago

Tried this out on the day it was released, and it's been awesome so far. Usually structuring images would take a lot of manual tweaking to get the layers just right, but with this I just adjust the number of layers and watch magic happen :) Average push size down from over 300MB to 15MB for an image of over 1.4GB, and takes like 15 lines of code to write.

zimbatm 8y ago

Nix and Docker integration is almost optimal now, no other tooling provides that type of functionality

jarvuschris 8y ago
[Habitat](http://habitat.sh/ http://habitat.sh/) builds on the concepts in Nix, and here is a relatively old experiment I found on optimizing Docker layering with it: https://github.com/chetan/habitat-dockerize-layers https://github.com/chetan/habitat-dockerize-layers
- roberthensing 8y ago
  What I like about Nix is that this kind of functionality can be integrated into the Nixpkgs package set, as a function over packages. It's just as easy as writing a machine image or an virtual machine network test, because Nixpkgs has functions you can use for those as well.

costrouc 8y ago

I really like this. I think it is a great compromise between the limitations of docker and everything packaged in one file while taking advantage of nix. I'm curious about one other optimization that could be made. You could instead look at the dependency graph of several nix docker builds and increase the cache reuse even more from that.

roberthensing 8y ago
This approach is already optimal for up to around 40 or 120 packages, depending on whether you want to support old Docker versions. "Where before you wouldn’t bother trying to have your application and database images share layers, with Nix the layer sharing is completely automatic."

kjullien 8y ago

Wouldn't a multi-stage dockerfile fix the problems of having N layers or is it N for the entire file?

tlrobinson 8y ago

Neat. It seems like Nix could become the preferred way to build Docker images, if it's made a bit more user friendly. It's hard to beat the simplicity of FROM ubuntu:latest RUN apt-get update RUN apt-get install -y mysql php

jf 8y ago
I'm very optimistic about Nix. If you haven't read it yet, I suggest starting with this article: https://www.infoq.com/articles/configuration-management-with-nix https://www.infoq.com/articles/configuration-management-with... I also gave a talk about why I'm excited about Nix: https://vimeo.com/223525975 https://vimeo.com/223525975
notthemessiah 8y ago
"Simplest" can be subjective. There's not much that can compete with editing your nixos config file and adding services.mattermost.enable = true; and then running # nixos-rebuild switch to have a mattermost instance running.
- tlrobinson 8y ago
  I'm specifically talk about building a Docker container. The example in the article has a couple dozen lines of nix config and several shell commands.

prisar 8y ago

Docker layers for windows bases images are of huge sizes