3 ms·

Another option to this is Repeatr [1]. Repeatr is for -- like it says on the tin -- repeating things; and hopefully, reproducing them. It's all the containmen

Another option to this is Repeatr [1]. Repeatr is for -- like it says on the tin -- repeating things; and hopefully, reproducing them. It's all the containment (runc, underneath), plus the ability to pull in various filesystems, specified by hash. For concrete examples, you can see Repeatr building Repeatr reproducibly here [2], for its own releases (it's a Go project, so this looks a lot like what's going on in Fillipo's blog here). Another example is this formula for reproducible builds of Runc [3] -- it uses cgo, and became reproducible with go1.7, which was very exciting! The big gain is Repeatr's syntax for multiple inputs and built-in hash verification means you can assemble reproducible environments like this out of multiple filesystems (say, busybox from one place, golang compiler separately), they all cache separately, they all download in parallel, you never have to write your own "tar -xcvf" boilerplate... and most importantly, you get built-in sanity-checking that the upstream tarballs you're using haven't changed either. No centralized "registry" server required, either. Give it a try! Disclaimer: author, of course :) --- [1] http://repeatr.io http://repeatr.io , https://github.com/polydawn/repeatr https://github.com/polydawn/repeatr [2] https://github.com/polydawn/repeatr/blob/master/meta/releases/repeatr-release-v0.14.frm https://github.com/polydawn/repeatr/blob/master/meta/release... [3] https://github.com/polydawn/formulary/blob/master/formulas/runc/runc.frm https://github.com/polydawn/formulary/blob/master/formulas/r...

rocqua 9y ago
For getting true value from reproducible builds, the environment should be as free as possible. If a build requires a certain image to be used, you need to trust that image not to be backdoored. Ideally, the only binary dependency in the environmental requirements is the compiler itself. Everything else should be source-code or plain-text. Getting the compiler out of the chain would be even better, but trusting trust is a hard problem.
- heavenlyhash 9y ago
  Yes, when you turn it up to 11, reproducible builds can be seen as a recursive problem. However, that's not a doom and gloom issue: like the old saying goes: How do we eat an elephant? One bite at a time. The saying still works even if it's a recursive stack of elephants: just keep going one bite at a time. Repeatr formulas are a precise identification of one build. That doesn't mean you shouldn't make a build that's even more portable. It means you can use more than one formula to precisely describe, and test, how portable and flexible your build is. You can also use one formula to describe one step of a build (say, compiling a base image), and then output the resulting filesystem... it's assigned a hash, and you can use this as an input to another formula. Thus, you can use binary images, AND have a complete (recursive!) reproducibility path. You can use reppl [1] for creating pipelines like this (though it's not yet geared for public re-sharing and comparing of results -- that's a big todo, if you want to lend a hand!). > trusting trust is a hard problem. Imagine you had two formulas, both with fully hash-pinned input filesystems, which both build GCC. One uses a GCC binary as an input. The other uses a clang binary as an input instead. When you run each of them, you get a hash of the result filesystem. The remaining step to address Trusting Trust is left as an exercise to the reader :) --- [1] https://github.com/polydawn/reppl/ https://github.com/polydawn/reppl/

rocqua 9y ago

For getting true value from reproducible builds, the environment should be as free as possible. If a build requires a certain image to be used, you need to trust that image not to be backdoored. Ideally, the only binary dependency in the environmental requirements is the compiler itself. Everything else should be source-code or plain-text. Getting the compiler out of the chain would be even better, but trusting trust is a hard problem.

heavenlyhash 9y ago
Yes, when you turn it up to 11, reproducible builds can be seen as a recursive problem. However, that's not a doom and gloom issue: like the old saying goes: How do we eat an elephant? One bite at a time. The saying still works even if it's a recursive stack of elephants: just keep going one bite at a time. Repeatr formulas are a precise identification of one build. That doesn't mean you shouldn't make a build that's even more portable. It means you can use more than one formula to precisely describe, and test, how portable and flexible your build is. You can also use one formula to describe one step of a build (say, compiling a base image), and then output the resulting filesystem... it's assigned a hash, and you can use this as an input to another formula. Thus, you can use binary images, AND have a complete (recursive!) reproducibility path. You can use reppl [1] for creating pipelines like this (though it's not yet geared for public re-sharing and comparing of results -- that's a big todo, if you want to lend a hand!). > trusting trust is a hard problem. Imagine you had two formulas, both with fully hash-pinned input filesystems, which both build GCC. One uses a GCC binary as an input. The other uses a clang binary as an input instead. When you run each of them, you get a hash of the result filesystem. The remaining step to address Trusting Trust is left as an exercise to the reader :) --- [1] https://github.com/polydawn/reppl/ https://github.com/polydawn/reppl/