4 ms·

That might be the case if Docker did in fact guarantee (or at least make it easy to guarantee) deterministic builds -- but it doesn't really even try: 1. Image

That might be the case if Docker did in fact guarantee (or at least make it easy to guarantee) deterministic builds -- but it doesn't really even try: 1. Image tags ("latest", etc.) can change over time. Does any layer in your Dockerfile -- including inside transitive deps -- build on an existing layer identified by tag? If so, you never had reproducibility. 2. Plenty of Dockerfiles include things like "apt-get some-tool" or its moral equivalent, which will pull down whatever is the latest version of that tool. It's currently common and considered normal to use these "features". Until that changes, Docker mostly adds only the impression of reproducibility, but genuine weight and pain.

tsimionescu 2y ago
The advantage that Docker brings isn't perfect guaranteed reusability, it's complete independence (or as close you can easily get while not wasting resources on VMs) from the system on which the build is running, plus some resuability in practice in a certain period of time, and given other good practices. Sure, if I try to rebuild a docker image from 5 years ago, it may fail, or produce something different, because it was pulling in some packages that have changed significantly in apt, or perhaps pip has changed encryption and no longer accepts TLS1.1 or whatever. And if you use ':latest' or if your team has a habit of reusing build numbers or custom tags, you may break the assumptions even sooner. But even so, especially if using always incremented build numbers as image tags, a docker image will work the same way om the Jenkins system, the GitHub Actions pipeline, and every coworker's local build, for a while.
akdev1l 2y ago
1. Use a hash for the base images. 2. The meaning of “latest” is dependent on what base images you are using. Using UBI images for example means your versions are not going to change because redhat versions don’t really change. But really containerizing the build environment is not related to deterministic builds as there’s a lot more work needed to guarantee that. Including possible changes in the application itself. Lastly you don’t need to use docker or dockerfiles to build containers. You can use whatever tooling you want to create a rootfs and then create an image out of that. A nifty way of actually guaranteeing reproducibility is to use nix to build a container image.
- akoboldfrying 2y ago
  >But really containerizing the build environment is not related to deterministic builds What would you say the goal of containerising builds is, if not reproducibility?
  - carlmr 2y ago
    They didn't say reproducibility, they said determinism. If you use a container with the same apt-get commands and the same OS, you already separate out almost all the reproducibility issues. What you get will be a very similar environment. But it's not deterministic as you point out, that takes a lot more effort.
carlmr 2y ago
I agree that Docker doesn't do the best job here, you can still get a lot better reproducibility than without though. You can use specific image hashes to work around image tag changes. For problem 2 ideally use something like NixOS as a base or at least nix package manager. But IMO Nix is quite complicated, even though the working model is really good. So this is mostly if you really need deterministic builds. Docker already gets you 80% of the way with 20% of the effort.

tsimionescu 2y ago

The advantage that Docker brings isn't perfect guaranteed reusability, it's complete independence (or as close you can easily get while not wasting resources on VMs) from the system on which the build is running, plus some resuability in practice in a certain period of time, and given other good practices. Sure, if I try to rebuild a docker image from 5 years ago, it may fail, or produce something different, because it was pulling in some packages that have changed significantly in apt, or perhaps pip has changed encryption and no longer accepts TLS1.1 or whatever. And if you use ':latest' or if your team has a habit of reusing build numbers or custom tags, you may break the assumptions even sooner. But even so, especially if using always incremented build numbers as image tags, a docker image will work the same way om the Jenkins system, the GitHub Actions pipeline, and every coworker's local build, for a while.

akdev1l 2y ago

1. Use a hash for the base images. 2. The meaning of “latest” is dependent on what base images you are using. Using UBI images for example means your versions are not going to change because redhat versions don’t really change. But really containerizing the build environment is not related to deterministic builds as there’s a lot more work needed to guarantee that. Including possible changes in the application itself. Lastly you don’t need to use docker or dockerfiles to build containers. You can use whatever tooling you want to create a rootfs and then create an image out of that. A nifty way of actually guaranteeing reproducibility is to use nix to build a container image.

akoboldfrying 2y ago
>But really containerizing the build environment is not related to deterministic builds What would you say the goal of containerising builds is, if not reproducibility?
- carlmr 2y ago
  They didn't say reproducibility, they said determinism. If you use a container with the same apt-get commands and the same OS, you already separate out almost all the reproducibility issues. What you get will be a very similar environment. But it's not deterministic as you point out, that takes a lot more effort.

carlmr 2y ago

I agree that Docker doesn't do the best job here, you can still get a lot better reproducibility than without though. You can use specific image hashes to work around image tag changes. For problem 2 ideally use something like NixOS as a base or at least nix package manager. But IMO Nix is quite complicated, even though the working model is really good. So this is mostly if you really need deterministic builds. Docker already gets you 80% of the way with 20% of the effort.