3 ms·

I studied this issue a bit several years ago, though I wasn't in contact with the toolchain developers. I think the simplest answer is that reproducible builds

by schoen 9y ago

I studied this issue a bit several years ago, though I wasn't in contact with the toolchain developers. I think the simplest answer is that reproducible builds only became an explicit priority recently in the last few years, while many of these compilers have existed for longer than that. The metadata is also useful for debugging purposes and debuggability is traditionally a high priority for compiler authors. People working on reproducible build projects have generally been able to modify compilers and/or create post-compilation tools to have the effect of removing the metadata. It's true that they could benefit from more help from upstream toolchains, though.

taeric 9y ago
Indeed. Provenance is a more common concern than reproducibility. Consider, even once you have it reproducible, it will be important track down where a build came from if it does differ. To that end, I would expect the tooling to be able to distinguish that the metadata about a build should not count towards its "byte similarity."
- schoen 9y ago
  Most people working on reproducible builds would probably suggest using digital signatures for provenance purposes (and never using any binaries that didn't come with a digital signature). However, present-day digital signatures don't necessarily include some of the metadata you might want (such as which person and/or machine was responsible for the compilation, not just which organization or entity). There are tools that can remove the metadata after the fact, but they aren't necessarily standardized enough to allow for a widely-understood comparable hash value. For example, https://packages.debian.org/sid/strip-nondeterminism https://packages.debian.org/sid/strip-nondeterminism
  - taeric 9y ago
    I get how this can accomplish the goal of validating that "Person X signed off on binary Y." However, it complicates the question of "looking at binary Y, how can I determine where/how it was built?" Yes, trusting metadata is just trusting something in the binary. It is very convenient, though. (Well, where it is ever used, it is convenient.) I think a compromise would be to combine them. Put an envelope over the binary that can have a "content" section and a "content-signature" section. The content should be reproducible, and the signature would vouch for it. You could then add "metadata" fields and each of those could additionally be signed, without having any change in the binary that is produced. Honestly, I expect this is how most builds are done. I have not looked into it, though.
    - andrewflnr 9y ago
      I don't see how that's a compromise. It just looks like an obvious solution to me. :) In short, you trust X to tell you about Y's provenance and include that in the signature. The rest is just details.
      - taeric 9y ago
        Completely agreed. I just want all of this included as part of Y. That is, don't make me track two files, when one works. :) Please tell me this is how things are done.

by schoen 9y ago

taeric 9y ago

Indeed. Provenance is a more common concern than reproducibility. Consider, even once you have it reproducible, it will be important track down where a build came from if it does differ. To that end, I would expect the tooling to be able to distinguish that the metadata about a build should not count towards its "byte similarity."

schoen 9y ago
Most people working on reproducible builds would probably suggest using digital signatures for provenance purposes (and never using any binaries that didn't come with a digital signature). However, present-day digital signatures don't necessarily include some of the metadata you might want (such as which person and/or machine was responsible for the compilation, not just which organization or entity). There are tools that can remove the metadata after the fact, but they aren't necessarily standardized enough to allow for a widely-understood comparable hash value. For example, https://packages.debian.org/sid/strip-nondeterminism https://packages.debian.org/sid/strip-nondeterminism
- taeric 9y ago
  I get how this can accomplish the goal of validating that "Person X signed off on binary Y." However, it complicates the question of "looking at binary Y, how can I determine where/how it was built?" Yes, trusting metadata is just trusting something in the binary. It is very convenient, though. (Well, where it is ever used, it is convenient.) I think a compromise would be to combine them. Put an envelope over the binary that can have a "content" section and a "content-signature" section. The content should be reproducible, and the signature would vouch for it. You could then add "metadata" fields and each of those could additionally be signed, without having any change in the binary that is produced. Honestly, I expect this is how most builds are done. I have not looked into it, though.
  - andrewflnr 9y ago
    I don't see how that's a compromise. It just looks like an obvious solution to me. :) In short, you trust X to tell you about Y's provenance and include that in the signature. The rest is just details.
    - taeric 9y ago
      Completely agreed. I just want all of this included as part of Y. That is, don't make me track two files, when one works. :) Please tell me this is how things are done.