4 ms·
I think build reproducibility is a cargo cult. Most people here are debating you on the security angle, but in the case of Nix (and Guix) there is another impo
by danieldk 5y ago
I think build reproducibility is a cargo cult.
Most people here are debating you on the security angle, but in the case of Nix (and Guix) there is another important angle - reproducible builds make a content-addressed store possible.
In Nix, the store is traditionally addressed by the hash of the derivation (the recipe that builds the package). For example, lr96h... in the path
/nix/store/lr96h3dlny8aiba9p3rmxcxfda0ijj08-coreutils-8.32
is the hash of the (normalized) derivation that was used to build coreutils. Since the derivation includes build inputs, either changing the derivation for coreutils itself or one of its inputs (dependencies) results in a different hash and a rebuild of coreutils.
This also means that if somebody changes the derivation of coreutils every package that depends on coreutils will be rebuilt, even if this change does not result in a different output path (compiled package).
This is being addressed by the new work on the content-addressed Nix store (although content-addressing aws already discussed in Eelco Dolstra's PhD thesis about Nix). In the content-addressed store, the hash in the path, such as the on above is a hash of the output path (the built package), rather than a hash of the normalized derivation. This means that if the derivation of coreutils is changed in such a way that it does not change the output path, none of the packages that depend on coreutils are rebuilt.
However, this only works reliably with reproducible builds, because if there is non-determinism in the build, how do you know whether a change in the output path is changed as a result of changing a derivation or as a result of uninteresting non-determinisms (the output hash would change in both cases).
- londons_explore 5y agoWhere the dependency chain is long, this substantially reduces build work during development too. I'd guess that more than half of the invocations of gcc done by Make for example end up producing the exact same bit for bit output as some previous invocation.
- taviso 5y agoI would point out that is literally what ccache (and Google goma) does, but doesn't require deterministic builds. Instead, it records hashes of preprocessed input and compiler commandlines. They don't make any security claims about this, it's just for speeding up builds.
- Ericson2314 5y agoWhat we currently do --- hashing inputs --- is the same ccache way. We just don't yet sandbox with the granularity yet. What we want to id hash outputs. Say I replace 1 + 2 with 0 + 3. That will cause ccache to rebuild. We don't want downstream stuff to also be rebuilt. C-linking withing a package is nice in parallelizable, but in the general case there is more dependency chains and now that sort of thing starts to matter.
- taviso 5y agoI don't really have any complaints about using deterministic builds for non-security reasons, but the number one claim most proponents make is that it somehow prevents backdoors. Literally the first claim on reproducible-builds.org is that build determinism will prevent threats of violence and blackmail.
- pxc 5y agoAnother non-security angle: doesn't computer science also face a kind of replicability crisis related to the ability to acquire and compile source code associated with some published papers? Reproducible builds directly address that. And it seems like even when that problem is resolved for the empirical component of computer science, bit-identical reproducibility could be valuable in case binaries are never submitted or distributed. This NixOS release is in a way a benchmark for how far we can currently get on a 'useful' system with that kind of reproducibility.