3 ms·

The intent of reproducible builds, as defined by the reproducible builds project[0], is to have the entire dependency tree built from the complete, correspondin

The intent of reproducible builds, as defined by the reproducible builds project[0], is to have the entire dependency tree built from the complete, corresponding source code of the many projects in that tree, starting from a minimal set of "bootstrap binaries"[1]. Using precompiled binaries, without a doubt, invalidates that goal. Reproducible builds are an ongoing, unsolved problem. In Guix, some things are reproducible, some things aren't, but our goal remains to maximize reproducibility. [0] https://reproducible-builds.org/ https://reproducible-builds.org/ [1] https://www.gnu.org/software/guix/manual/html_node/Bootstrapping.html https://www.gnu.org/software/guix/manual/html_node/Bootstrap...

edude03 10y ago
I'm pretty sure the goal of reproducible builds is to be able to get the same binary as made somewhere from the source code. Arguably binaries are a form of memoizing / caching if you look at compilation as a pure functional problem. For example, if I have build system that says "with file a hash = 1234 and file b with hash = 4321 I should get a binary with hash = 9876 you can save time compelling that package to check its hash by simply asking the binary repo for a package with hash 9876.
SwellJoe 10y ago
That's just wrong-headed. Every binary in the chain can be signed and reproducible (theoretically; we aren't quite there, realistically speaking, and the bootstrap, as you note, is tricky). Forcing every user to build everything from source every time is just a mess, and provides little benefit over signed (reproducible) binaries for the vast majority of users. It sounds like you (and the guix project) are conflating two orthogonal concepts and making "reproducible builds" into a concept that won't work in widespread use, and maybe even has negative value. We want reproducible builds so that we can be certain that the binary a vendor ships us is built exactly the way they say it was. We don't want it so that everyone is forced to build everything from scratch every time. Reproducible builds means we can build an identical binary from source to prove that it is what we think it is. It should not mean we must build an identical binary from source every time we install a thing.
- davexunit 10y ago
  >Forcing every user to build everything from source every time >We don't want it so that everyone is forced to build everything from scratch every time. I believe it is you who is confused. I never suggested this is what reproducible builds are about. https://www.gnu.org/software/guix/manual/html_node/Substitutes.html https://www.gnu.org/software/guix/manual/html_node/Substitut...
  - corysama 10y ago
    When two programmers argue and both think the other sounds crazy, it's usually because they are saying the same words but meaning different things. SwellJoe: Go back and substitute "source verifiable" every time davexunit says "reproducible" and everything he says makes sense. davexunit: Read everyone else's comments from the POV where "reproducible" mean "reliable bitwise replication" without consideration of scientifically reproducible experiments or source verification, and everything they say makes sense.
  - SwellJoe 10y ago
    I believe several people were confused by your statement. So, at least I'm not alone. I interpreted your statement of "Using pre-compiled binaries means that builds aren't reproducible, a goal of both Nix and Guix" to mean, "Guix does not distribute pre-compiled binaries, because they can't be reproduced". That still seems an entirely reasonable interpretation of that sentence, to me, as someone with no familiarity with Guix. But, I now see that your intended meaning was that Guix does not distribute stuff like nvidia blobs, or Flash player, which is a perfectly reasonable choice for them, and nothing less than I would expect of a GNU project (and Fedora, my current favorite desktop distro, has the same policy, so I don't have a problem with it).
- lfam 10y ago
  > We don't want it so that everyone is forced to build everything from scratch every time. We don't want this either :) Let's not boil the oceans. Nix and Guix are both a hybrid of build-from-source and binary package management systems. The package recipes and management tools describe how to build some software from source, and then transparently provide binaries for those recipes when the binary is available from a user-approved provider of the binaries. But, by providing the full build recipes, all the way down to the bootstrap, users can easily rebuild binaries from source to check that their selected binary provider is honest, or to find reproducibility bugs. In the case of a package recipe for some software that does not have source code available (a binary "blob"), the recipe would just download the blob from somewhere. We don't package those things in Guix, whereas Nix does. Although, it's not hard to make custom recipes that would do that, if you wanted to.

edude03 10y ago

I'm pretty sure the goal of reproducible builds is to be able to get the same binary as made somewhere from the source code. Arguably binaries are a form of memoizing / caching if you look at compilation as a pure functional problem. For example, if I have build system that says "with file a hash = 1234 and file b with hash = 4321 I should get a binary with hash = 9876 you can save time compelling that package to check its hash by simply asking the binary repo for a package with hash 9876.

SwellJoe 10y ago

That's just wrong-headed. Every binary in the chain can be signed and reproducible (theoretically; we aren't quite there, realistically speaking, and the bootstrap, as you note, is tricky). Forcing every user to build everything from source every time is just a mess, and provides little benefit over signed (reproducible) binaries for the vast majority of users. It sounds like you (and the guix project) are conflating two orthogonal concepts and making "reproducible builds" into a concept that won't work in widespread use, and maybe even has negative value. We want reproducible builds so that we can be certain that the binary a vendor ships us is built exactly the way they say it was. We don't want it so that everyone is forced to build everything from scratch every time. Reproducible builds means we can build an identical binary from source to prove that it is what we think it is. It should not mean we must build an identical binary from source every time we install a thing.

davexunit 10y ago
>Forcing every user to build everything from source every time >We don't want it so that everyone is forced to build everything from scratch every time. I believe it is you who is confused. I never suggested this is what reproducible builds are about. https://www.gnu.org/software/guix/manual/html_node/Substitutes.html https://www.gnu.org/software/guix/manual/html_node/Substitut...
- corysama 10y ago
  When two programmers argue and both think the other sounds crazy, it's usually because they are saying the same words but meaning different things. SwellJoe: Go back and substitute "source verifiable" every time davexunit says "reproducible" and everything he says makes sense. davexunit: Read everyone else's comments from the POV where "reproducible" mean "reliable bitwise replication" without consideration of scientifically reproducible experiments or source verification, and everything they say makes sense.
- SwellJoe 10y ago
  I believe several people were confused by your statement. So, at least I'm not alone. I interpreted your statement of "Using pre-compiled binaries means that builds aren't reproducible, a goal of both Nix and Guix" to mean, "Guix does not distribute pre-compiled binaries, because they can't be reproduced". That still seems an entirely reasonable interpretation of that sentence, to me, as someone with no familiarity with Guix. But, I now see that your intended meaning was that Guix does not distribute stuff like nvidia blobs, or Flash player, which is a perfectly reasonable choice for them, and nothing less than I would expect of a GNU project (and Fedora, my current favorite desktop distro, has the same policy, so I don't have a problem with it).
lfam 10y ago
> We don't want it so that everyone is forced to build everything from scratch every time. We don't want this either :) Let's not boil the oceans. Nix and Guix are both a hybrid of build-from-source and binary package management systems. The package recipes and management tools describe how to build some software from source, and then transparently provide binaries for those recipes when the binary is available from a user-approved provider of the binaries. But, by providing the full build recipes, all the way down to the bootstrap, users can easily rebuild binaries from source to check that their selected binary provider is honest, or to find reproducibility bugs. In the case of a package recipe for some software that does not have source code available (a binary "blob"), the recipe would just download the blob from somewhere. We don't package those things in Guix, whereas Nix does. Although, it's not hard to make custom recipes that would do that, if you wanted to.