3 ms·

This is not how docker works. Docker, exactly like nix, is based on a graph of content addressable dependencies. What you are describing is chaining a bunch of

This is not how docker works. Docker, exactly like nix, is based on a graph of content addressable dependencies. What you are describing is chaining a bunch of commands together. Yes, this forms a dependency chain stored in separate layers and is part of the cache chain. Nix suffers the exact same problems with reproducibility. The thing it provides is the toolchain of dependencies that are reproducible. Docker does not provide your dependencies. If the inputs change then so does the output. If the output itself is not reproducible (like, say an artifact with a build-time embedded in it) then you have something that is inherently not reproducible and two people trying to build the same exact nix package will have different results. EDIT: Fixed a sentence I apparently got distracted while writing and didn't complete (about layer caching).

Foxboron 3y ago
Nix is not content addressable though, the hashes is based off on the derivation files which are equal to the lock files you would find in other package managers. > The thing it provides is the toolchain of dependencies that are reproducible. [...] If the inputs change then so does the output. If the output itself is not reproducible (like, say an artifact with a build-time embedded in it) then you have something that is inherently not reproducible and two people trying to build the same exact nix package will have different results. There are no guarantees they are reproducible. The only guarantees Nix gives you is that the build environment is the same which allows you to make some claims about the system behaving the same way. But they are certainly no guarantees about artifacts being bit-by-bit identical.
- cpuguy83 3y ago
  It's content addressable, just what are you addressing? The content address of a docker image is a json blob (referencing other objects). The content address of a Dockerfile "RUN" command is the content address of what came before it and the command being run.
  - Foxboron 3y ago
    >It's content addressable, just what are you addressing? In the case of Nix it's addressed by the input. Not the content of the build. It's an important distinction and one Nix also makes. https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-store-make-content-addressed https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3... But doing this is going to give you a slight headace as most of the package repository in Nix is not checked for reproducible builds and there are no way to guarantee the hashes are actually static.
    - cpuguy83 3y ago
      Right, all builds are dependent on their inputs. Your inputs determine your outputs. If your input(s) change, then so does your output. We are saying the same thing here, I'm just trying to point out this is exactly how docker build works, but rather it is more about what you are willing to put into your docker build.
      - Foxboron 3y ago
        I think we are talking past each other. I'm just trying to clear up a misconception on how nix works, not anything about the docker portion of what you have written.
georgyo 3y ago
It would seem you don't understand how either work. They are basically opposites in how they actually work. Docker layers are completely independent from each other. A docker layer is a sha256 sum of that layer. Separately there is an image manifest, which is also fetched with a sha256 of that manifest, states the order of the layers and at runtime those layers are stacked up on each other. With docker, there is no explicit dependency chain. A layer is just a tarball or some JSON. Some tooling can take advantage of this fact. How nix builds docker images takes advantage of this. Nix on the other hand, the output hash is not tied to the output hash, because the output hash is irrelevant to how it was produced. You also cannot know in advance the output hash of something. IE. If I do say "echo foo > bar.txt", I cannot know the sha256 sum of bar.txt until the code runs. But before the code runs I can know the hash all the inputs that will create bar.txt. This fundamental difference means two builds, executing the same code can share the outputs. Provided that the build environment is trust worthy.
- cpuguy83 3y ago
  You are describing the makeup of an OCI image, which is the _output_ of a typical docker build (and also the output of the nix image builder). While docker build can/does output OCI images, that is only an output. How that output comes to be is not the output itself, same as the nix side the article is talking about.
  - georgyo 3y ago
    > How that output comes to be is not the output itself, same as the nix side the article is talking about. I see the confusion now. The nix image builders OCI layer's contents are _only_ nix store paths which _do_ include the input. Nix store paths are are guaranteed to never overlap each other, and such the order of layering them in the docker manifest does not matter. But the docker layers are just tar balls of nix store paths. Each layer in the image has no dependence on previous or future layers at all. It is just one or more nix store paths.
    - cpuguy83 3y ago

Foxboron 3y ago

Nix is not content addressable though, the hashes is based off on the derivation files which are equal to the lock files you would find in other package managers. > The thing it provides is the toolchain of dependencies that are reproducible. [...] If the inputs change then so does the output. If the output itself is not reproducible (like, say an artifact with a build-time embedded in it) then you have something that is inherently not reproducible and two people trying to build the same exact nix package will have different results. There are no guarantees they are reproducible. The only guarantees Nix gives you is that the build environment is the same which allows you to make some claims about the system behaving the same way. But they are certainly no guarantees about artifacts being bit-by-bit identical.

cpuguy83 3y ago
It's content addressable, just what are you addressing? The content address of a docker image is a json blob (referencing other objects). The content address of a Dockerfile "RUN" command is the content address of what came before it and the command being run.
- Foxboron 3y ago
  >It's content addressable, just what are you addressing? In the case of Nix it's addressed by the input. Not the content of the build. It's an important distinction and one Nix also makes. https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-store-make-content-addressed https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3... But doing this is going to give you a slight headace as most of the package repository in Nix is not checked for reproducible builds and there are no way to guarantee the hashes are actually static.
  - cpuguy83 3y ago
    Right, all builds are dependent on their inputs. Your inputs determine your outputs. If your input(s) change, then so does your output. We are saying the same thing here, I'm just trying to point out this is exactly how docker build works, but rather it is more about what you are willing to put into your docker build.
    - Foxboron 3y ago
      I think we are talking past each other. I'm just trying to clear up a misconception on how nix works, not anything about the docker portion of what you have written.

georgyo 3y ago

It would seem you don't understand how either work. They are basically opposites in how they actually work. Docker layers are completely independent from each other. A docker layer is a sha256 sum of that layer. Separately there is an image manifest, which is also fetched with a sha256 of that manifest, states the order of the layers and at runtime those layers are stacked up on each other. With docker, there is no explicit dependency chain. A layer is just a tarball or some JSON. Some tooling can take advantage of this fact. How nix builds docker images takes advantage of this. Nix on the other hand, the output hash is not tied to the output hash, because the output hash is irrelevant to how it was produced. You also cannot know in advance the output hash of something. IE. If I do say "echo foo > bar.txt", I cannot know the sha256 sum of bar.txt until the code runs. But before the code runs I can know the hash all the inputs that will create bar.txt. This fundamental difference means two builds, executing the same code can share the outputs. Provided that the build environment is trust worthy.

cpuguy83 3y ago
You are describing the makeup of an OCI image, which is the _output_ of a typical docker build (and also the output of the nix image builder). While docker build can/does output OCI images, that is only an output. How that output comes to be is not the output itself, same as the nix side the article is talking about.
- georgyo 3y ago
  > How that output comes to be is not the output itself, same as the nix side the article is talking about. I see the confusion now. The nix image builders OCI layer's contents are _only_ nix store paths which _do_ include the input. Nix store paths are are guaranteed to never overlap each other, and such the order of layering them in the docker manifest does not matter. But the docker layers are just tar balls of nix store paths. Each layer in the image has no dependence on previous or future layers at all. It is just one or more nix store paths.
  - cpuguy83 3y ago