5 ms·

Even if Docker doesn't really care about this, the DockerFiles themselves can also describe a reproducible process, and they are certainly "incremental" due to

Even if Docker doesn't really care about this, the DockerFiles themselves can also describe a reproducible process, and they are certainly "incremental" due to caching (not sure of what you mean by "incremental"). Does nix power merit such a big compromise against simplicity and readability?

progval 6y ago
Because Dockerfiles delegate all the complexity to a package manager (apt, apk, ...), which has its own languages (eg. debian/rules for apt). Using Docker without any package manager doing this job would be a completely different story. And btw, you can also use Nix to build docker images. For example, here is what would be the equivalent of a Dockerfile but with Nix: https://nixos.org/nixpkgs/manual/#ex-dockerTools-buildImage https://nixos.org/nixpkgs/manual/#ex-dockerTools-buildImage > the DockerFiles themselves can also describe a reproducible process It's trickier. You need to pin all versions of all packages you use, even indirect dependencies.
catern 6y ago
Dockerfiles which just pull packages from distribution repositories are not reproducible in the same way that Nix expressions are. Rebuilding the Dockerfile will give you different results if the packages in the distribution repositories change. A Nix expression specifies the entire tree of dependencies, and can be built from scratch anywhere at any time and get the same result.
throwaway894345 6y ago
> the DockerFiles themselves can also describe a reproducible process This is true, but Docker does almost nothing to support reproducibility. As soon as you do an apt-get, reproducibility goes out the window. > they are certainly "incremental" due to caching Caching is layer-based. Docker has no awareness of whether or not a particular dependency has changed or what is necessary to rebuild it. Docker only understands layers, which are inherently linear--you can only try in vain to force your dependency graph into that linear layer structure. An incremental build tool only rebuilds things that have changed (or whose dependencies have changed). > Does nix power merit such a big compromise against simplicity and readability? This is a good question. First of all, understand that Nix isn't intended to be a build tool, it's a package toolkit. It's intended to replace all of the stuff that the debian and centos people use to build and manage packages in apt and yum repos. That said, I personally think Docker's build system is sooooo bad that under many circumstances, Nix does a better job. For example, if your whole repo is Go or Rust or some other language with a sane build and deployment story, then they already have incremental, reproducible build tools and it's fine for Docker to call into them (or to invoke them outside of Docker and just have Docker copy in the static artifact). But if you have a heterogeneous tree including C or Python or other languages which lack sane build tooling, then you need something to stitch all of that together in some (somewhat) reproducible, incremental fashion if you hope to be able to build reliably and in some timely fashion. That said, there are a few other tools that are purpose-built for this problem; however, they all tend to be poorly designed, buggy, and hard to extend. I'm thinking of Bazel, Pants, Buck, etc--all of which are clones of Google's internal build system, Blaze. So far, Nix seems to be the best in class, even though it doesn't aspire to be in the class at all.
- mehrdadn 6y ago
  Great description, thanks. Question: do you know how Nix deals with language package managers (like pip)? If I do sudo pip install (leaving aside the usual debate as to its merits), does that then wreck my Nix install, especially if it happens to contain another version of the same package? And does it instantly make things non-reproducible again (just like the apt-get issue you mentioned)? Or does Nix somehow get around it?
  - throwaway894345 6y ago
    Nix doesn't pay any attention to the rest of your system. It keeps everything it needs (including its own Python interpreter) in its own "nix store" directory, at /nix/store. You can crudely think of it as having its own virtualenv for every distinct Python target you use.
  - joepie91_ 6y ago
    It's totally possible to introduce mutable, non-reproducible state on a system that uses Nix, even if that system is NixOS. However, you can't do so in your Nix store, which is where all the Nix-installed things live (at least, not without a lot of hackery - Nix tries very hard to make the entire store immutable). Basically this works out to: anything you install or configure with Nix, is immutable and reproducible. Anything you don't is at your own risk. Unfortunately there are still some "pockets of state" that haven't been solved yet, most notably applications storing state in ~. For most every language dependency management system, there's a Nix equivalent of some sort that allows you to manage those dependencies without invoking the non-reproducible mutable tooling.
- paul_milovanov 6y ago
  Bazel is buggy?! Bazel is not a clone, it's the refactoring of the internal build system, basically. I am 99% certain that Blaze currently has Bazel at the core. And it's pretty damn robust.
  - mehrdadn 6y ago
    Bazel is definitely buggy. One of the silliest ones is this one (and as far as this one goes, I do not understand how this issue exists when Blaze has been used in Google for such a long itme): https://github.com/bazelbuild/bazel/issues/9419 https://github.com/bazelbuild/bazel/issues/9419 One of the more fundamental bugs is this one: https://github.com/bazelbuild/bazel/issues/4558 https://github.com/bazelbuild/bazel/issues/4558 I've even had to clean --expunge and clear my disk cache to fix some build errors before. Bazel paints a rosy picture of the world, but the real world is messy, and Bazel builds aren't as reproducible as it makes you want to think. (Which BTW is one thing that's had me asking so many questions about Nix.) At least the fact that it does not track changes outside the workspace means you can update your system compilers (or Python, or whatever) and end up with build outputs that will not be reproduced from scratch. It takes a lot of time to track the issues down, so I don't necessarily know what goes wrong every time; these are just two issues I remember off the top of my head. But if you haven't run into bugs with Bazel then you haven't used it seriously enough. And then there's the Python support which is even more half-baked right now. Their own comments readily mention that the Python support is not idiomatic. (Not just in terms of syntax, but some of it goes against Bazel's underlying design.) Of course I say this with the full realization that "fixing" these things can be a huge undertaking (up to and including hooking the compiler programs manually), but that doesn't mean they're not bugs.
takeda 6y ago
Dockerfiles are as reproducible as a bash script (and that's not a hyperbole). Yes, they are incremental but as you pointed out it is per statement. This requires: 1) that it is on you to specify the correct order 2) you are limited to 42 layers 3) it doesn't work well even if you know what to do and care about it: - in most setups that I've seen once you trigger build system it still starts from scratch (for example each time, docker images pull all dependencies again and again every build). nix build won't build stuff that was already done, unless you explicitly instruct it with --check option - typically people use `COPY . .`, because it is less work, but in reality certain files need to be pulled earlier than others, but then you're more likely to run into layer limit, also it's more work, more prone to errors (like typos, forgetting about a file etc). With declarative configuration, nix will only rebuild derivations where input changed. It doesn't matter what order you specify dependencies. > Does nix power merit such a big compromise against simplicity and readability? The simplicity causes complexity everywhere else. Is this really that complex? Here's for example everything that's needed to build pgbouncer[1] (you can ignore the meta section, that's informational only). I'm using that since I used it recently. Nix actually isn't a complicated language, just different. I believe the big problem people are having is just different programming paradigm, but the programming paradigm used by nix is really what is needed to solve the problem correctly (purely functional -> for the same inputs (source code, dependencies, system etc) you always get the same output; lazily evaluated -> build only things you actually depend on). The properties of the language allow to define how to build specific program in a declarative way. [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sql/pgbouncer/default.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sq...

progval 6y ago

Because Dockerfiles delegate all the complexity to a package manager (apt, apk, ...), which has its own languages (eg. debian/rules for apt). Using Docker without any package manager doing this job would be a completely different story. And btw, you can also use Nix to build docker images. For example, here is what would be the equivalent of a Dockerfile but with Nix: https://nixos.org/nixpkgs/manual/#ex-dockerTools-buildImage https://nixos.org/nixpkgs/manual/#ex-dockerTools-buildImage > the DockerFiles themselves can also describe a reproducible process It's trickier. You need to pin all versions of all packages you use, even indirect dependencies.

catern 6y ago

Dockerfiles which just pull packages from distribution repositories are not reproducible in the same way that Nix expressions are. Rebuilding the Dockerfile will give you different results if the packages in the distribution repositories change. A Nix expression specifies the entire tree of dependencies, and can be built from scratch anywhere at any time and get the same result.

throwaway894345 6y ago

> the DockerFiles themselves can also describe a reproducible process This is true, but Docker does almost nothing to support reproducibility. As soon as you do an apt-get, reproducibility goes out the window. > they are certainly "incremental" due to caching Caching is layer-based. Docker has no awareness of whether or not a particular dependency has changed or what is necessary to rebuild it. Docker only understands layers, which are inherently linear--you can only try in vain to force your dependency graph into that linear layer structure. An incremental build tool only rebuilds things that have changed (or whose dependencies have changed). > Does nix power merit such a big compromise against simplicity and readability? This is a good question. First of all, understand that Nix isn't intended to be a build tool, it's a package toolkit. It's intended to replace all of the stuff that the debian and centos people use to build and manage packages in apt and yum repos. That said, I personally think Docker's build system is sooooo bad that under many circumstances, Nix does a better job. For example, if your whole repo is Go or Rust or some other language with a sane build and deployment story, then they already have incremental, reproducible build tools and it's fine for Docker to call into them (or to invoke them outside of Docker and just have Docker copy in the static artifact). But if you have a heterogeneous tree including C or Python or other languages which lack sane build tooling, then you need something to stitch all of that together in some (somewhat) reproducible, incremental fashion if you hope to be able to build reliably and in some timely fashion. That said, there are a few other tools that are purpose-built for this problem; however, they all tend to be poorly designed, buggy, and hard to extend. I'm thinking of Bazel, Pants, Buck, etc--all of which are clones of Google's internal build system, Blaze. So far, Nix seems to be the best in class, even though it doesn't aspire to be in the class at all.

mehrdadn 6y ago
Great description, thanks. Question: do you know how Nix deals with language package managers (like pip)? If I do sudo pip install (leaving aside the usual debate as to its merits), does that then wreck my Nix install, especially if it happens to contain another version of the same package? And does it instantly make things non-reproducible again (just like the apt-get issue you mentioned)? Or does Nix somehow get around it?
- throwaway894345 6y ago
  Nix doesn't pay any attention to the rest of your system. It keeps everything it needs (including its own Python interpreter) in its own "nix store" directory, at /nix/store. You can crudely think of it as having its own virtualenv for every distinct Python target you use.
- joepie91_ 6y ago
  It's totally possible to introduce mutable, non-reproducible state on a system that uses Nix, even if that system is NixOS. However, you can't do so in your Nix store, which is where all the Nix-installed things live (at least, not without a lot of hackery - Nix tries very hard to make the entire store immutable). Basically this works out to: anything you install or configure with Nix, is immutable and reproducible. Anything you don't is at your own risk. Unfortunately there are still some "pockets of state" that haven't been solved yet, most notably applications storing state in ~. For most every language dependency management system, there's a Nix equivalent of some sort that allows you to manage those dependencies without invoking the non-reproducible mutable tooling.
paul_milovanov 6y ago
Bazel is buggy?! Bazel is not a clone, it's the refactoring of the internal build system, basically. I am 99% certain that Blaze currently has Bazel at the core. And it's pretty damn robust.
- mehrdadn 6y ago
  Bazel is definitely buggy. One of the silliest ones is this one (and as far as this one goes, I do not understand how this issue exists when Blaze has been used in Google for such a long itme): https://github.com/bazelbuild/bazel/issues/9419 https://github.com/bazelbuild/bazel/issues/9419 One of the more fundamental bugs is this one: https://github.com/bazelbuild/bazel/issues/4558 https://github.com/bazelbuild/bazel/issues/4558 I've even had to clean --expunge and clear my disk cache to fix some build errors before. Bazel paints a rosy picture of the world, but the real world is messy, and Bazel builds aren't as reproducible as it makes you want to think. (Which BTW is one thing that's had me asking so many questions about Nix.) At least the fact that it does not track changes outside the workspace means you can update your system compilers (or Python, or whatever) and end up with build outputs that will not be reproduced from scratch. It takes a lot of time to track the issues down, so I don't necessarily know what goes wrong every time; these are just two issues I remember off the top of my head. But if you haven't run into bugs with Bazel then you haven't used it seriously enough. And then there's the Python support which is even more half-baked right now. Their own comments readily mention that the Python support is not idiomatic. (Not just in terms of syntax, but some of it goes against Bazel's underlying design.) Of course I say this with the full realization that "fixing" these things can be a huge undertaking (up to and including hooking the compiler programs manually), but that doesn't mean they're not bugs.

takeda 6y ago

Dockerfiles are as reproducible as a bash script (and that's not a hyperbole). Yes, they are incremental but as you pointed out it is per statement. This requires: 1) that it is on you to specify the correct order 2) you are limited to 42 layers 3) it doesn't work well even if you know what to do and care about it: - in most setups that I've seen once you trigger build system it still starts from scratch (for example each time, docker images pull all dependencies again and again every build). nix build won't build stuff that was already done, unless you explicitly instruct it with --check option - typically people use `COPY . .`, because it is less work, but in reality certain files need to be pulled earlier than others, but then you're more likely to run into layer limit, also it's more work, more prone to errors (like typos, forgetting about a file etc). With declarative configuration, nix will only rebuild derivations where input changed. It doesn't matter what order you specify dependencies. > Does nix power merit such a big compromise against simplicity and readability? The simplicity causes complexity everywhere else. Is this really that complex? Here's for example everything that's needed to build pgbouncer[1] (you can ignore the meta section, that's informational only). I'm using that since I used it recently. Nix actually isn't a complicated language, just different. I believe the big problem people are having is just different programming paradigm, but the programming paradigm used by nix is really what is needed to solve the problem correctly (purely functional -> for the same inputs (source code, dependencies, system etc) you always get the same output; lazily evaluated -> build only things you actually depend on). The properties of the language allow to define how to build specific program in a declarative way. [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sql/pgbouncer/default.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sq...