3 ms·

> there exist no reproducibility monitoring at the scale of the Nix package set (nixpkgs) I think it would be fairly easy to do this monitoring with a bit of c

> there exist no reproducibility monitoring at the scale of the Nix package set (nixpkgs) I think it would be fairly easy to do this monitoring with a bit of community participation. At least I'd enable telemetry ¯\_(ツ)_/¯. By default the pkz57 in /nix/store/pkz57...-nushell-0.97.1 is a hash of the build inputs for that package. If you hash the contents of that dir, you get an identifier for the build output. If we then make a big list of such pairs as built by different people on different machines at different times, and capture the frequency of each pair we'll either see this: /nix/store/pkz5..., sha256:0drrxa..., 121 users or we'll see this: /nix/store/pkz5..., sha256:0drrxa..., 100 users /nix/store/pkz5..., sha256:gvpbk5..., 20 users /nix/store/pkz5..., sha256:1fwwfe..., 1 user The former being an indicator that the build is reproducible, and the latter giving hints about why not (supposing these users are willing to share a bit more about the circumstances of that build). I'd call it a "build chromatograph". I expect that knowing whether you're one of the 100 or the odd-man-out could be relevant for certain scenarios.

gf000 2y ago
I'm not sure the "distribution" would be all that helpful. A single compiler that does some parallel work and collect the results of that work in a list in order of completion (and similar) are probably the most common cause of non-determinism. Given that, your chromatograph would be mostly "determined" by the source code's peculiarities, instead of the offending compiler itself. (E.g. I have n classes a compiler would process in parallel, so given a single point of timing non-determinism n! different combinations could possibly exist (assuming they all cause a different outputs). The only information I could conclude from such a distribution is that there is a common sequence of completing tasks). But your idea is cool, and simply reporting back non-matching local builds would be helpful (I believe the binary property of whether a different output could be built is the only relevant fact) -- also, if we were to mark a package as non-reproducible, we could recursively mark everything else that has it as a (transitive) input.
- __MatrixMan__ 2y ago
  I think there are only a few cases where it would be helpful: - one hash to rule them all, perfectly reproducible - a big mess, consider avoiding it - just two or four cohorts, package maintainer may want to investigate - everbody agrees on the output hash except for you, something local is compromized I don't anticipiate peering into the mess and coming up with many useful conclusions. > if we were to mark a package as non-reproducible, we could recursively mark everything else that has it as a (transitive) input. I like that idea, to sort of carve out a space within the already-pretty-reliable nixpkgs which can be expected upon to be perfectly reproducible. I'd strive to get my packages included in that set, and to select my dependencies from it.

gf000 2y ago

I'm not sure the "distribution" would be all that helpful. A single compiler that does some parallel work and collect the results of that work in a list in order of completion (and similar) are probably the most common cause of non-determinism. Given that, your chromatograph would be mostly "determined" by the source code's peculiarities, instead of the offending compiler itself. (E.g. I have n classes a compiler would process in parallel, so given a single point of timing non-determinism n! different combinations could possibly exist (assuming they all cause a different outputs). The only information I could conclude from such a distribution is that there is a common sequence of completing tasks). But your idea is cool, and simply reporting back non-matching local builds would be helpful (I believe the binary property of whether a different output could be built is the only relevant fact) -- also, if we were to mark a package as non-reproducible, we could recursively mark everything else that has it as a (transitive) input.

__MatrixMan__ 2y ago
I think there are only a few cases where it would be helpful: - one hash to rule them all, perfectly reproducible - a big mess, consider avoiding it - just two or four cohorts, package maintainer may want to investigate - everbody agrees on the output hash except for you, something local is compromized I don't anticipiate peering into the mess and coming up with many useful conclusions. > if we were to mark a package as non-reproducible, we could recursively mark everything else that has it as a (transitive) input. I like that idea, to sort of carve out a space within the already-pretty-reliable nixpkgs which can be expected upon to be perfectly reproducible. I'd strive to get my packages included in that set, and to select my dependencies from it.