6 ms·

Docker builds are not deterministic, I don't get where you get that idea. I can't count the hours lost because the last guy who left one year ago built the imag

Docker builds are not deterministic, I don't get where you get that idea. I can't count the hours lost because the last guy who left one year ago built the image using duck tape and sed commands everywhere. The image is set in stone, but so is a zip file, there's nothing special here. Building an image using nix solves many problems regarding not only reproducible environments that can be tested outside a container but also fully horizontal dependency management where each dependency gets a layer that's not stacked on one another like a typical apt/npm/cargo/pip command. And I don't have to reverse engineer the world just to see what files changed in the filesystem since everything has its place and has a systematic BOM.

jhanoncomm 3y ago
So is it right, to make docker reproducible it needs to either build dependencies from source from say a git hash or use other package managers that are reproducible or rely on base images that are reproducible. And that all relies on discipline. Just like using a dynamically typed programming language can in theory have no type errors at run time, if you are careful enough.
- yjftsjthsd-h 3y ago
  Right; you could write a Dockerfile that went something like FROM base-image@e70197813aa3b7c86586e6ecbbf0e18d2643dfc8a788aac79e8c906b9e2b0785 RUN pkg install foo=1.2.3 bar=2.3.4 RUN git clone https://some/source.git && cd source && git checkout f8b02f5809843d97553a1df02997a5896ba3c1c6 RUN gcc --reproducible-flags source/foo.c -o foo but that's (IME) really rare; you're more likely to find `FROM debian:10` (which isn't too likely to change but is not pinned) and `RUN git clone -b v1.2.3 repo.git` (which is probably fixed but could change)... And then there's the Dockerfiles that just `RUN git clone repo.git` and run with whatever happened to be in the latest commit at the moment...
  - cpuguy83 3y ago
    It is likely just as rare for someone to use nix for this, though.
    - yjftsjthsd-h 3y ago
      Possible; I don't have a feel for the relative likelihoods. I think the thing nix has going for it is that you can write a nix package definition without having to actually hardcode anything in and nix itself will give you the defaults to make ex. compilers be deterministic/reproducible, and automate handling flake.lock so you don't have to actually pay attention to the pins yourself. Or put differently; you can make either one reproducible, but nix is designed to help you do that while docker really doesn't care.
    - clhodapp 3y ago
      It's actually how nix works by default. When you pull in a dependency, you are actually pulling in a full description of how to build it. And it pulls in full descriptions of how to build its dependencies and so on. The only reason nix isn't dog slow is that it has really strong caching so it doesn't have to build everything from source.
      - cpuguy83 3y ago
        This is literally how docker works as well. The difference is docker doesn't bring a toolchain for those artifacts.
        clhodapp 3y ago
        Docker can resolve dependencies in a very similar manner to nix, via multi-stage builds. Each FROM makes one dependency available. However, you can only have direct access to the content from one of the dependencies resolved this way. The other ones, you have to COPY over the relevant content --from at build time.
        cpuguy83 3y ago
        I'm not exactly sure what you are referring to here. You can have as many "FROM"'s as you want. "FROM scratch" along with "ADD" is also valid (for non-image dependencies). From there you do not need to copy things, you can mount the reference into another stage directly. Also, this is the Dockerfile format, the underlying build API's are _far_ more powerful than what Dockerfile exposes.
  - janjongboom 3y ago
    And that assumes that `foo` and `bar` are not overwritten or deleted in your package repository, and that the git repository remains available.
  - nullify88 3y ago
    Maintaining something like that is a pain unless you have tooling like Renovate to inform and update the digests and versions.
- MadnessASAP 3y ago
  You can also use a hammer to put a screw in the wall. Dockerfiles being at their core a set of instructions for producing a container image could of course be used to make a reproducible image. Although you'd have to be painfully verbose to ensure that you got the exact same output. You would actually likely need 2 files, the first being the build environment that the second actually get built in. Or you could use Nix that is actually intended to do this and provides the necessary framework for reproducibility.
  - inopinatus 3y ago
    fun fact: there actually is a class of impact driver[1] that couples longitudinal force to rotation to prevent cam-out on screws like the Phillips head when high torque is required [1] https://en.wikipedia.org/wiki/Impact_driver#Manual_impact_drivers https://en.wikipedia.org/wiki/Impact_driver#Manual_impact_dr...
    - jhanoncomm 3y ago
      My favourite “well actually” on HN. As a reluctant DIYer, thanks!

jhanoncomm 3y ago

So is it right, to make docker reproducible it needs to either build dependencies from source from say a git hash or use other package managers that are reproducible or rely on base images that are reproducible. And that all relies on discipline. Just like using a dynamically typed programming language can in theory have no type errors at run time, if you are careful enough.

yjftsjthsd-h 3y ago
Right; you could write a Dockerfile that went something like FROM base-image@e70197813aa3b7c86586e6ecbbf0e18d2643dfc8a788aac79e8c906b9e2b0785 RUN pkg install foo=1.2.3 bar=2.3.4 RUN git clone https://some/source.git && cd source && git checkout f8b02f5809843d97553a1df02997a5896ba3c1c6 RUN gcc --reproducible-flags source/foo.c -o foo but that's (IME) really rare; you're more likely to find `FROM debian:10` (which isn't too likely to change but is not pinned) and `RUN git clone -b v1.2.3 repo.git` (which is probably fixed but could change)... And then there's the Dockerfiles that just `RUN git clone repo.git` and run with whatever happened to be in the latest commit at the moment...
- cpuguy83 3y ago
  It is likely just as rare for someone to use nix for this, though.
  - yjftsjthsd-h 3y ago
    Possible; I don't have a feel for the relative likelihoods. I think the thing nix has going for it is that you can write a nix package definition without having to actually hardcode anything in and nix itself will give you the defaults to make ex. compilers be deterministic/reproducible, and automate handling flake.lock so you don't have to actually pay attention to the pins yourself. Or put differently; you can make either one reproducible, but nix is designed to help you do that while docker really doesn't care.
  - clhodapp 3y ago
    It's actually how nix works by default. When you pull in a dependency, you are actually pulling in a full description of how to build it. And it pulls in full descriptions of how to build its dependencies and so on. The only reason nix isn't dog slow is that it has really strong caching so it doesn't have to build everything from source.
    - cpuguy83 3y ago
      This is literally how docker works as well. The difference is docker doesn't bring a toolchain for those artifacts.
      - clhodapp 3y ago
        Docker can resolve dependencies in a very similar manner to nix, via multi-stage builds. Each FROM makes one dependency available. However, you can only have direct access to the content from one of the dependencies resolved this way. The other ones, you have to COPY over the relevant content --from at build time.
        cpuguy83 3y ago
        I'm not exactly sure what you are referring to here. You can have as many "FROM"'s as you want. "FROM scratch" along with "ADD" is also valid (for non-image dependencies). From there you do not need to copy things, you can mount the reference into another stage directly. Also, this is the Dockerfile format, the underlying build API's are _far_ more powerful than what Dockerfile exposes.
- janjongboom 3y ago
  And that assumes that `foo` and `bar` are not overwritten or deleted in your package repository, and that the git repository remains available.
- nullify88 3y ago
  Maintaining something like that is a pain unless you have tooling like Renovate to inform and update the digests and versions.
MadnessASAP 3y ago
You can also use a hammer to put a screw in the wall. Dockerfiles being at their core a set of instructions for producing a container image could of course be used to make a reproducible image. Although you'd have to be painfully verbose to ensure that you got the exact same output. You would actually likely need 2 files, the first being the build environment that the second actually get built in. Or you could use Nix that is actually intended to do this and provides the necessary framework for reproducibility.
- inopinatus 3y ago
  fun fact: there actually is a class of impact driver[1] that couples longitudinal force to rotation to prevent cam-out on screws like the Phillips head when high torque is required [1] https://en.wikipedia.org/wiki/Impact_driver#Manual_impact_drivers https://en.wikipedia.org/wiki/Impact_driver#Manual_impact_dr...
  - jhanoncomm 3y ago
    My favourite “well actually” on HN. As a reluctant DIYer, thanks!