4 ms·

The big difference is that nothing about Dockerfiles implies that the processes they perform to build an image are deterministic or repeatable. Will a build st

by benley 10y ago

The big difference is that nothing about Dockerfiles implies that the processes they perform to build an image are deterministic or repeatable. Will a build step that runs `curl https://github.com/something/whatever` https://github.com/something/whatever` do the same thing in six months that it does today? Docker doesn't help with that at all. Nix improves determinism by (almost) guaranteeing that if you build the same nix expression six months from now, you'll get exactly the same dependencies, all the way down to glibc. Either that or the build will just fail, if remote dependencies aren't downloadable anymore, which is always tons of fun.

pacala 10y ago
In practice, one can check the Dockerfile lineage of a given Docker image and assess the quality of the commands. Good practices are to use package managers with explicit versions, for example "apt-get install subversion-tools=1.3.2-5~bpo1". Or, if you so desire, just pipe NixOS outputs into Docker, as somebody else in this thread mentioned.
thejosh 10y ago
YES. Finally someone understands what really annoys me when I say that docker isn't 100% reproducible if you aren't using version pinning or something similar. If you have 2 developers, and one of them does a build the next day, you could have them with two different versions of a package when the version went up.
- koverstreet 10y ago
  That isn't even the bad part - the bad part is that give some big complicated docker image (or any build in general!), you really have no practical way of knowing what's different. That should scare people.
- mgkimsal 10y ago
  sounds like 'emerge hell' from gentoo 12 years ago.
  - wmertens 10y ago
    Ex-Gentoo dev here. It is not the same. Gentoo emerge runs in place, and you can definitely break your system with a broken build. NixOS on the other hand, builds everything in sandboxes that only expose the requested dependencies. Builds are almost 100% deterministic. The cherry on top is that your system installation is simply a package consisting of all other packages and configuration symlinked together, and you replace your entire system in one go (atomically, by writing a symlink). If the build fails at any point along the chain, you get an error and your system remains unchanged. You can totally switch from one major release to the next (and back) without hassle.
    - mgkimsal 10y ago
      the non-deterministic nature is what I was getting at. I worked at a place where they'd spin up a new gentoo box for a dev with stock "emerge foo" and let everything run up. Then 3 weeks later, they'd do it again for the next dev, and they'd have many diff versions. I know they weren't "doing it right", but they were doing it the default way they learned, and it caused a lot of problems. Thanks for the info/clarification though.
      - darkarmani 10y ago
        It's been a LONG time, but when you build gentoo packages you can save the builds, right? Then if you bring up identical servers, you can just install from precompiled packages.
girvo 10y ago
You're not wrong, but Docker comes at it from a different angle; Docker expects you to be handing around tagged images, to solve a similar set (but not the same set!) of problems that reproducible builds solve. Ideally, I want _both_ of them: tagged Docker images being sent between environments (with `docker run -e SOME_ENV=testing imagename` for changing the internal config), but reproducible builds in my Dockerfile. I wonder if Nix can be used to achieve that last part?
- ramblenode 10y ago
  Yes, you can do this. There is a utility "nix-docker" that will convert a Nix configuration file into a BusyBox Docker image (see e.g. http://zef.me/blog/6049/nix-docker http://zef.me/blog/6049/nix-docker)
  - girvo 10y ago
    Oh brilliant! Thanks for that, definitely going to give that a shot :)
  - mateuszf 10y ago
    Unfortunately the repo says: > DISCLAIMER: This project is no longer actively maintained and probably broken
  - rokgarbas 10y ago
    with nix you can now build docker images ... docs are in the manual http://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools http://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools and it is actually rather cool since you don't rely on any docker command :)
- xyzzy_plugh 10y ago
  The problem is tagged images are fundamentally broken. If I can't reproduce, bit-for-bit, what is in the tag, how can I vouch for the tag? Images are nothing but a caching technique for the results of deterministic builds. If you're using them for any other reason, you have a flawed process that is going to come back to bite you some day.

by benley 10y ago

pacala 10y ago

In practice, one can check the Dockerfile lineage of a given Docker image and assess the quality of the commands. Good practices are to use package managers with explicit versions, for example "apt-get install subversion-tools=1.3.2-5~bpo1". Or, if you so desire, just pipe NixOS outputs into Docker, as somebody else in this thread mentioned.

thejosh 10y ago

YES. Finally someone understands what really annoys me when I say that docker isn't 100% reproducible if you aren't using version pinning or something similar. If you have 2 developers, and one of them does a build the next day, you could have them with two different versions of a package when the version went up.

koverstreet 10y ago
That isn't even the bad part - the bad part is that give some big complicated docker image (or any build in general!), you really have no practical way of knowing what's different. That should scare people.
mgkimsal 10y ago
sounds like 'emerge hell' from gentoo 12 years ago.
- wmertens 10y ago
  Ex-Gentoo dev here. It is not the same. Gentoo emerge runs in place, and you can definitely break your system with a broken build. NixOS on the other hand, builds everything in sandboxes that only expose the requested dependencies. Builds are almost 100% deterministic. The cherry on top is that your system installation is simply a package consisting of all other packages and configuration symlinked together, and you replace your entire system in one go (atomically, by writing a symlink). If the build fails at any point along the chain, you get an error and your system remains unchanged. You can totally switch from one major release to the next (and back) without hassle.
  - mgkimsal 10y ago
    the non-deterministic nature is what I was getting at. I worked at a place where they'd spin up a new gentoo box for a dev with stock "emerge foo" and let everything run up. Then 3 weeks later, they'd do it again for the next dev, and they'd have many diff versions. I know they weren't "doing it right", but they were doing it the default way they learned, and it caused a lot of problems. Thanks for the info/clarification though.
    - darkarmani 10y ago
      It's been a LONG time, but when you build gentoo packages you can save the builds, right? Then if you bring up identical servers, you can just install from precompiled packages.

girvo 10y ago

You're not wrong, but Docker comes at it from a different angle; Docker expects you to be handing around tagged images, to solve a similar set (but not the same set!) of problems that reproducible builds solve. Ideally, I want _both_ of them: tagged Docker images being sent between environments (with `docker run -e SOME_ENV=testing imagename` for changing the internal config), but reproducible builds in my Dockerfile. I wonder if Nix can be used to achieve that last part?

ramblenode 10y ago
Yes, you can do this. There is a utility "nix-docker" that will convert a Nix configuration file into a BusyBox Docker image (see e.g. http://zef.me/blog/6049/nix-docker http://zef.me/blog/6049/nix-docker)
- girvo 10y ago
  Oh brilliant! Thanks for that, definitely going to give that a shot :)
- mateuszf 10y ago
  Unfortunately the repo says: > DISCLAIMER: This project is no longer actively maintained and probably broken
- rokgarbas 10y ago
  with nix you can now build docker images ... docs are in the manual http://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools http://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools and it is actually rather cool since you don't rely on any docker command :)
xyzzy_plugh 10y ago
The problem is tagged images are fundamentally broken. If I can't reproduce, bit-for-bit, what is in the tag, how can I vouch for the tag? Images are nothing but a caching technique for the results of deterministic builds. If you're using them for any other reason, you have a flawed process that is going to come back to bite you some day.