4 ms·

Isn’t what we really need reproducible binaries? The problem of reproducible (and thus verifiable) software is 99% solved practically with languages that build

Isn’t what we really need reproducible binaries? The problem of reproducible (and thus verifiable) software is 99% solved practically with languages that build reproducible binaries. Requiring at least one human to build and sign a production release (which we know will be identical to what is produced by CI) is probably a reasonable way to solve this problem. While this usually implies static linking, even a dynamic link would work if we had a way to ask for a dynamic lib by content hash, and we aggressively shunted dynamic per-platform content to other runtime files.

woodruffw 3y ago
Reproducibility has two operational constraints: it assumes the user is actually reproducing the binary (not the case in Homebrew, where the binary is pre-built and distributed to users), and it assumes that users have some way to ascertain whether the build has actually reproduced (versus comparing it to a digest delivered by the potentially malicious index, which could then be whatever malicious build the index intends). Homebrew packages whatever software its users want; artificially limiting their packages to only languages that make reproducibility easy would significantly diminish its value. Edit: to be absolutely clear, reproducibility is a great idea, and a great long term goal. It is not a sufficient substitute for digital signatures, especially not in binary distribution schemes like Homebrew.
- yjftsjthsd-h 3y ago
  I don't think every user has to reproduce the build; it should suffice to have a handful of verifiers building and checking the published versions. Someone needs to do it, but not everyone. (Edit: yes, you do need a way to make sure everyone is seeing the same published hashes; no idea how to do that)
  - hardwaresofton 3y ago
    Yup, thought it was clear in my comment, the software producers should be building and signing THEIR copy from their machine. Seeing the same public hashes can work just like people check shasums now, also possible to independently verify at any time
    - woodruffw 3y ago
      > Seeing the same public hashes can work just like people check shasums now, also possible to independently verify at any time “Verify at any time” devolves to “verify never.” Verification needs to be mandatory to have any security value, which comes back the original point about operationalization.
  - woodruffw 3y ago
    > I don't think every user has to reproduce the build; it should suffice to have a handful of verifiers building and checking the published versions. Someone needs to do it, but not everyone. This works, but you still need a gossiping mechanism (and accompanying threshold model, etc.). That gets very complicated very fast!
- csande17 3y ago
  I'm not sure I understand this line of argument. A system where every user rebuilds all packages themselves, just to use the resulting file hashes to verify whether they can trust an identical binary they already downloaded from the Internet, is obviously kind of goofy. Instead, I think the use case for reproducibility is allowing multiple trusted parties to independently build and generate signatures for the same binary. In GP's example, open source contributors could run their own builds independently (maybe on air-gapped machines) to produce multiple trustworthy signatures for the resulting binaries. F-Droid does this kind of thing today; you can get apps verified by both the original author and the F-Droid infrastructure. I think the main difference between that and your scheme is your scheme relies on whoever runs the CI/CD machines (Microsoft?) not tampering with them. But the underlying signature technology is probably still useful in a reproducible-builds world—assuming you don't break the reproducibility of the bottles by e.g. embedding the signatures into the binaries somehow.
  - woodruffw 3y ago
    > Instead, I think the use case for reproducibility is allowing multiple trusted parties to independently build and generate signatures for the same binary. In GP's example, open source contributors could run their own builds independently (maybe on air-gapped machines) to produce multiple trustworthy signatures for the resulting binaries. F-Droid does this kind of thing today; you can get apps verified by both the original author and the F-Droid infrastructure. You can do this, but I don’t think it has much of a value (or security) proposition on its own. The ideal world here would be “reproducible signed builds + occasional independent rebuilds with gossiping,” but that is significantly harder to operationalize than just build signing or reproducibility.
    - csande17 3y ago
      > You can do this, but I don’t think it has much of a value (or security) proposition on its own. The value prop is pretty straightforward in the open-source space: allowing clients to choose to only trust binaries signed by N independent actors. F-Droid achieves this today: if an application needs to be built/signed by both the original author and F-Droid to be considered valid, neither party can push a compromised binary on their own. ("Compromised" in the sense of "doesn't match the source code it claims to", which AFAICT is all you're targeting with this Homebrew scheme too.) GNU Guix is working on a similar scheme where the client can choose to only accept binaries built by K of N "substitute servers". And even today, they ship a one-click "challenge" command that rebuilds a bunch of binaries to report whether a substitute server is shipping something suspicious; users can then gossip about this via their favorite social media platform. I'd argue that both of these solutions create a lot more value than your scheme, where most of the people who would previously be able to tamper with the single, central instance of the build infrastructure are still able to do that without being detected.
- hardwaresofton 3y ago
  See the other comment, the idea is that the person producing the binary signs a version as an exemplar of sorts. Also there’s no need to limit the publishing of unsigned packages at first, there needs to be a transition period. The problem is that if no one makes the hard decision to make this a priority then it never becomes one and we keep operating on hope and sheer luck. Hope any luck definitely have worked great thus far but maybe we can work differently going forward
zamalek 3y ago
It's called Nix and it works on MacOS. https://nixos.org/ https://nixos.org/ It's honestly what MacOS almost bearable for me prior to me convincing my employer to allow me to use Linux.
mikemcquaid 3y ago
Homebrew Project leader here. Homebrew tries fairly hard already to make builds reproducible. Unfortunately, no one has been dedicated enough build infrastructure to make sure this does regress so we end up with issues like https://github.com/Homebrew/brew/issues/16012 https://github.com/Homebrew/brew/issues/16012 To be clear, though, some Homebrew packages are already fully reproducible across OSs, some on the same OS and we welcome any help in improving this further. Thanks for raising the issue.
- hardwaresofton 3y ago
  Thanks for the often thankless work you do! I just want to be clear, this wasn’t something that I thought was wrong with homebrew, it was more about the ecosystem embracing reproducibility so it’s easier for you all and similar projects, overall! I personally think that approaches like nix/guix ask for the forest when they only need the tree — I often only care if the software I built is reproducible, and I don’t think that always requires the entire package store/OS to be.
- inferiorhuman 3y ago
  Reproducible… but unsupported. Nice. https://github.com/Homebrew/homebrew-core/issues/108964#issuecomment-1228240629 https://github.com/Homebrew/homebrew-core/issues/108964#issu...

woodruffw 3y ago

Reproducibility has two operational constraints: it assumes the user is actually reproducing the binary (not the case in Homebrew, where the binary is pre-built and distributed to users), and it assumes that users have some way to ascertain whether the build has actually reproduced (versus comparing it to a digest delivered by the potentially malicious index, which could then be whatever malicious build the index intends). Homebrew packages whatever software its users want; artificially limiting their packages to only languages that make reproducibility easy would significantly diminish its value. Edit: to be absolutely clear, reproducibility is a great idea, and a great long term goal. It is not a sufficient substitute for digital signatures, especially not in binary distribution schemes like Homebrew.

yjftsjthsd-h 3y ago
I don't think every user has to reproduce the build; it should suffice to have a handful of verifiers building and checking the published versions. Someone needs to do it, but not everyone. (Edit: yes, you do need a way to make sure everyone is seeing the same published hashes; no idea how to do that)
- hardwaresofton 3y ago
  Yup, thought it was clear in my comment, the software producers should be building and signing THEIR copy from their machine. Seeing the same public hashes can work just like people check shasums now, also possible to independently verify at any time
  - woodruffw 3y ago
    > Seeing the same public hashes can work just like people check shasums now, also possible to independently verify at any time “Verify at any time” devolves to “verify never.” Verification needs to be mandatory to have any security value, which comes back the original point about operationalization.
- woodruffw 3y ago
  > I don't think every user has to reproduce the build; it should suffice to have a handful of verifiers building and checking the published versions. Someone needs to do it, but not everyone. This works, but you still need a gossiping mechanism (and accompanying threshold model, etc.). That gets very complicated very fast!
csande17 3y ago
I'm not sure I understand this line of argument. A system where every user rebuilds all packages themselves, just to use the resulting file hashes to verify whether they can trust an identical binary they already downloaded from the Internet, is obviously kind of goofy. Instead, I think the use case for reproducibility is allowing multiple trusted parties to independently build and generate signatures for the same binary. In GP's example, open source contributors could run their own builds independently (maybe on air-gapped machines) to produce multiple trustworthy signatures for the resulting binaries. F-Droid does this kind of thing today; you can get apps verified by both the original author and the F-Droid infrastructure. I think the main difference between that and your scheme is your scheme relies on whoever runs the CI/CD machines (Microsoft?) not tampering with them. But the underlying signature technology is probably still useful in a reproducible-builds world—assuming you don't break the reproducibility of the bottles by e.g. embedding the signatures into the binaries somehow.
- woodruffw 3y ago
  > Instead, I think the use case for reproducibility is allowing multiple trusted parties to independently build and generate signatures for the same binary. In GP's example, open source contributors could run their own builds independently (maybe on air-gapped machines) to produce multiple trustworthy signatures for the resulting binaries. F-Droid does this kind of thing today; you can get apps verified by both the original author and the F-Droid infrastructure. You can do this, but I don’t think it has much of a value (or security) proposition on its own. The ideal world here would be “reproducible signed builds + occasional independent rebuilds with gossiping,” but that is significantly harder to operationalize than just build signing or reproducibility.
  - csande17 3y ago
    > You can do this, but I don’t think it has much of a value (or security) proposition on its own. The value prop is pretty straightforward in the open-source space: allowing clients to choose to only trust binaries signed by N independent actors. F-Droid achieves this today: if an application needs to be built/signed by both the original author and F-Droid to be considered valid, neither party can push a compromised binary on their own. ("Compromised" in the sense of "doesn't match the source code it claims to", which AFAICT is all you're targeting with this Homebrew scheme too.) GNU Guix is working on a similar scheme where the client can choose to only accept binaries built by K of N "substitute servers". And even today, they ship a one-click "challenge" command that rebuilds a bunch of binaries to report whether a substitute server is shipping something suspicious; users can then gossip about this via their favorite social media platform. I'd argue that both of these solutions create a lot more value than your scheme, where most of the people who would previously be able to tamper with the single, central instance of the build infrastructure are still able to do that without being detected.
hardwaresofton 3y ago
See the other comment, the idea is that the person producing the binary signs a version as an exemplar of sorts. Also there’s no need to limit the publishing of unsigned packages at first, there needs to be a transition period. The problem is that if no one makes the hard decision to make this a priority then it never becomes one and we keep operating on hope and sheer luck. Hope any luck definitely have worked great thus far but maybe we can work differently going forward

zamalek 3y ago

It's called Nix and it works on MacOS. https://nixos.org/ https://nixos.org/ It's honestly what MacOS almost bearable for me prior to me convincing my employer to allow me to use Linux.

mikemcquaid 3y ago

Homebrew Project leader here. Homebrew tries fairly hard already to make builds reproducible. Unfortunately, no one has been dedicated enough build infrastructure to make sure this does regress so we end up with issues like https://github.com/Homebrew/brew/issues/16012 https://github.com/Homebrew/brew/issues/16012 To be clear, though, some Homebrew packages are already fully reproducible across OSs, some on the same OS and we welcome any help in improving this further. Thanks for raising the issue.

hardwaresofton 3y ago
Thanks for the often thankless work you do! I just want to be clear, this wasn’t something that I thought was wrong with homebrew, it was more about the ecosystem embracing reproducibility so it’s easier for you all and similar projects, overall! I personally think that approaches like nix/guix ask for the forest when they only need the tree — I often only care if the software I built is reproducible, and I don’t think that always requires the entire package store/OS to be.
inferiorhuman 3y ago
Reproducible… but unsupported. Nice. https://github.com/Homebrew/homebrew-core/issues/108964#issuecomment-1228240629 https://github.com/Homebrew/homebrew-core/issues/108964#issu...