7 ms·

Interesting use case for IPFS, which has often felt like an (admittedly cool) solution in search of a problem. Can someone enlighten me as to real-world exampl

Interesting use case for IPFS, which has often felt like an (admittedly cool) solution in search of a problem. Can someone enlighten me as to real-world examples where actually reproducible builds are critical?

arianvanp 6y ago
For one. They allow for distributed incremental builds. Improving your developer productivity. IT doesn't matter anymore where an object file is compiled. You just need to know its hash, ask if it exists, or otherwise compile yourself. One problem with NixOS currently is that certain dependencies in our package tree are very painful to change. If we touch glibc, we need to recompile 60.000 packages and it takes A LOT of compute power to do that. With reproducible, content-addressed builds we can do things like early cutoff optimisations; making these changes less painful. SImple example; if somebody just changed a source code comment in glibc; then we get the same build artifact, and can skip building 60.000 packages. If you do this at the source code level instead of the package level (like Bazel) then somebody can change how glibc does domain resolution; but packages that don't depend on that don't need to be recompiled either.
- teacpde 6y ago
  I understand why changing comments doesn’t matter, but how do you decide which dependencies don’t need to be rebuilt when certain part of the source code is changed?
  - kevincox 6y ago
    The most "proper" way to do this is splitting the result into interface + implementation. For example if you are dynamically linking to a library instead of being given access to the entire library you are given a simplified "signature" which has the minimal amount of info required to linking. This interface would change when you add or remove a function, but not when a function body changes. The example was given for linking but this can be done for many steps of the build process.
  - Ericson2314 6y ago
    Nix does this simply and stupidly --- if you can observ it, it's part of the cache key. This is wonderful because it's totally sound. All efficiency methods (basically, hiding things, as the other comment says) is left to the build plan itself. Switching to Nix for batch jobs is switching from DOS to an OS with actual process isolation. A complete paradigm shift.
    - sterlind 6y ago
      It does still require trust, though. You can lie about the contents since the hash only covers the build inputs, so someone could break the sandboxing. Of course, since builds are reproducible you can catch this if you build it yourself and compare the actual content-hashes.
      - Ericson2314 6y ago
        I would say the trust issues with build remotes and substitutors are fairly independent of the caching policy? The caching model is about when the drv changes ((drv, output) is the cache key, basically), the trust model is one one can download something to avoiding build to fill in the value for the new cache key. Note also that with the new floating content-addressed derivations, the trust stuff is much easier to think about.
Taek 6y ago
Aren't reproducible builds pretty well established as being important for security? How do you know that the developers are providing you with the binaries that they claim? Most people do not compile their own software, they used signed binary distributions being given to them. If you have reproducible builds, third parties can more easily verify independently that the code being distributed is the same as the code published in the open source repositories. This doesn't just keep dev teams honest, it also provides defenses against situations like those where a hacker gains control of the website being used for distribution. I believe reproducible builds have helped to catch this real life issue at least once in the cryptocurrency community (with Monero). To me, that's enough to justify their existence.
chpatrick 6y ago
Our team uses Nix. It lets us share a binary cache of modified/own packages between the developers, CI, and our production machines. If you push something to CI that needs some new dependencies, the CI machine will build them and when you want to get a local dev environment or deploy, the binaries will be fetched from the CI machine. We can also send the binaries between machines to save compile time. This is possible because we have confidence that it doesn't matter which machine built them because the environment is so strictly controlled.
Foxboron 6y ago
Confusingly enough, when NixOS talks about reproducible builds they are not talking about https://reproducible-builds.org/ https://reproducible-builds.org/. They are talking about reproducible systems. NixOS doesn't solve reproducible builds as-in "bit-for-bit identical binary distributed files", but it does solve "same input" -> "functionally same system". This is the core idea of NixOS and what makes the package manager and the spawned ecosystem (like Guix) quite neat. I also think this isn't any more controversial then say having your infrastructure reproducible with terraform and ansible/salt. It gives you machines as cattle, and not pets, with the added flexibility that gives you. NixOS is just a complete package where in other systems there are several components accomplishing the goal. As to why reproducible builds is important is to ensure a strong connection between the upstream source code and the distributed binary build. This gives you confidence that everything going into the build has been declared and that the binary can be reproduced bit-for-bit identical if you so wish. This can be both important in a supply chain process. I recently saw tailscale provided pre-compiled binaries without signatures. What does those binaries contain? They are not signed, so the releases could very well be unauthenticated and someone could have compromised the server distributing the binaries. So I tried reproducing the tailscale binaries without much success as the build process is proprietary. https://github.com/tailscale/tailscale/issues/779 https://github.com/tailscale/tailscale/issues/779 So what does the tailscale binaries contain? I can't get a bit-for-bit identical binary without putting into a great deal of effort so for all intents and purposes they can very well be proprietary. Now I don't think the tailscale people are malicious, nor that the binaries have been replaced. But it's a real-world example I recently went through.
- solarkraft 6y ago
  Even when the binaries are signed you'd still have to trust Tailscale they didn't modify the code before compilation, maybe putting in a simple back door. They probably don't, but you can't really know. When you have binary reproducible builds such a thing would immediately be obvious and hell would break loose if they attempted it.
- Ericson2314 6y ago
  The floating content-addressed derivations we've worked on as part of this Nix × IPFS work get's us a lot closer to reproducible builds by making non-determinstic builds far more noticiable. See these: https://github.com/NixOS/rfcs/pull/62 https://github.com/NixOS/rfcs/pull/62 https://github.com/NixOS/nix/issues/4087 https://github.com/NixOS/nix/issues/4087
zelly 6y ago
There are much more real-world use cases for reproducible builds than for IPFS. It saves time and money. It's not just academic. Reproducible builds (as in bit-for-bit) means I can trust a (secure) checksum of a binary I downloaded somewhere, let's say from IPFS. This is defense-in-depth, adding another layer of security to prevent sabotage. Reproducibility means the whole system can reliably be built from source and I don't need to manually make sure the system has the correct constellation of implicit dependencies and their version numbers. This saves time and money in deployment and will pay off in a very big way 10 years down the line when that code is still in use somewhere.

arianvanp 6y ago

For one. They allow for distributed incremental builds. Improving your developer productivity. IT doesn't matter anymore where an object file is compiled. You just need to know its hash, ask if it exists, or otherwise compile yourself. One problem with NixOS currently is that certain dependencies in our package tree are very painful to change. If we touch glibc, we need to recompile 60.000 packages and it takes A LOT of compute power to do that. With reproducible, content-addressed builds we can do things like early cutoff optimisations; making these changes less painful. SImple example; if somebody just changed a source code comment in glibc; then we get the same build artifact, and can skip building 60.000 packages. If you do this at the source code level instead of the package level (like Bazel) then somebody can change how glibc does domain resolution; but packages that don't depend on that don't need to be recompiled either.

teacpde 6y ago
I understand why changing comments doesn’t matter, but how do you decide which dependencies don’t need to be rebuilt when certain part of the source code is changed?
- kevincox 6y ago
  The most "proper" way to do this is splitting the result into interface + implementation. For example if you are dynamically linking to a library instead of being given access to the entire library you are given a simplified "signature" which has the minimal amount of info required to linking. This interface would change when you add or remove a function, but not when a function body changes. The example was given for linking but this can be done for many steps of the build process.
- Ericson2314 6y ago
  Nix does this simply and stupidly --- if you can observ it, it's part of the cache key. This is wonderful because it's totally sound. All efficiency methods (basically, hiding things, as the other comment says) is left to the build plan itself. Switching to Nix for batch jobs is switching from DOS to an OS with actual process isolation. A complete paradigm shift.
  - sterlind 6y ago
    It does still require trust, though. You can lie about the contents since the hash only covers the build inputs, so someone could break the sandboxing. Of course, since builds are reproducible you can catch this if you build it yourself and compare the actual content-hashes.
    - Ericson2314 6y ago
      I would say the trust issues with build remotes and substitutors are fairly independent of the caching policy? The caching model is about when the drv changes ((drv, output) is the cache key, basically), the trust model is one one can download something to avoiding build to fill in the value for the new cache key. Note also that with the new floating content-addressed derivations, the trust stuff is much easier to think about.

Taek 6y ago

Aren't reproducible builds pretty well established as being important for security? How do you know that the developers are providing you with the binaries that they claim? Most people do not compile their own software, they used signed binary distributions being given to them. If you have reproducible builds, third parties can more easily verify independently that the code being distributed is the same as the code published in the open source repositories. This doesn't just keep dev teams honest, it also provides defenses against situations like those where a hacker gains control of the website being used for distribution. I believe reproducible builds have helped to catch this real life issue at least once in the cryptocurrency community (with Monero). To me, that's enough to justify their existence.

chpatrick 6y ago

Our team uses Nix. It lets us share a binary cache of modified/own packages between the developers, CI, and our production machines. If you push something to CI that needs some new dependencies, the CI machine will build them and when you want to get a local dev environment or deploy, the binaries will be fetched from the CI machine. We can also send the binaries between machines to save compile time. This is possible because we have confidence that it doesn't matter which machine built them because the environment is so strictly controlled.

Foxboron 6y ago

Confusingly enough, when NixOS talks about reproducible builds they are not talking about https://reproducible-builds.org/ https://reproducible-builds.org/. They are talking about reproducible systems. NixOS doesn't solve reproducible builds as-in "bit-for-bit identical binary distributed files", but it does solve "same input" -> "functionally same system". This is the core idea of NixOS and what makes the package manager and the spawned ecosystem (like Guix) quite neat. I also think this isn't any more controversial then say having your infrastructure reproducible with terraform and ansible/salt. It gives you machines as cattle, and not pets, with the added flexibility that gives you. NixOS is just a complete package where in other systems there are several components accomplishing the goal. As to why reproducible builds is important is to ensure a strong connection between the upstream source code and the distributed binary build. This gives you confidence that everything going into the build has been declared and that the binary can be reproduced bit-for-bit identical if you so wish. This can be both important in a supply chain process. I recently saw tailscale provided pre-compiled binaries without signatures. What does those binaries contain? They are not signed, so the releases could very well be unauthenticated and someone could have compromised the server distributing the binaries. So I tried reproducing the tailscale binaries without much success as the build process is proprietary. https://github.com/tailscale/tailscale/issues/779 https://github.com/tailscale/tailscale/issues/779 So what does the tailscale binaries contain? I can't get a bit-for-bit identical binary without putting into a great deal of effort so for all intents and purposes they can very well be proprietary. Now I don't think the tailscale people are malicious, nor that the binaries have been replaced. But it's a real-world example I recently went through.

solarkraft 6y ago
Even when the binaries are signed you'd still have to trust Tailscale they didn't modify the code before compilation, maybe putting in a simple back door. They probably don't, but you can't really know. When you have binary reproducible builds such a thing would immediately be obvious and hell would break loose if they attempted it.
Ericson2314 6y ago
The floating content-addressed derivations we've worked on as part of this Nix × IPFS work get's us a lot closer to reproducible builds by making non-determinstic builds far more noticiable. See these: https://github.com/NixOS/rfcs/pull/62 https://github.com/NixOS/rfcs/pull/62 https://github.com/NixOS/nix/issues/4087 https://github.com/NixOS/nix/issues/4087

zelly 6y ago

There are much more real-world use cases for reproducible builds than for IPFS. It saves time and money. It's not just academic. Reproducible builds (as in bit-for-bit) means I can trust a (secure) checksum of a binary I downloaded somewhere, let's say from IPFS. This is defense-in-depth, adding another layer of security to prevent sabotage. Reproducibility means the whole system can reliably be built from source and I don't need to manually make sure the system has the correct constellation of implicit dependencies and their version numbers. This saves time and money in deployment and will pay off in a very big way 10 years down the line when that code is still in use somewhere.