4 ms·
I maintain the reproducible builds effort for my company and, please, let me tell you that this is the main pitfall of the whole effort. There is always going
by no-dr-onboard 2y ago
I maintain the reproducible builds effort for my company and, please, let me tell you that this is the main pitfall of the whole effort.
There is always going to be a degree of un-reproducibility just due to the nature of math. If you don't have the same system, same compiler version (down to the minor or patch level), same dependency versions, same build flags, filesystem ordering, OS handling etc. . .you're going to get differences.
The RB project has readily disclosed that there is a degree of "significantly reproducible" sussing that each end user is going to have to do. The fact that the Debian maintainers chose not to display the degree of reproducibility is probably because showing low reproducibility scores undermines the efforts to evangelize the movement.
I think that's understandable, but also is a bit of a two edged sword. If we don't disclose scores, we allow for the misrepresentation that "this is safe because it has the word reproducible in it". If we disclose scores, we get articles like this saying "wow, thats a really low score, wtf" and short lived paranoia gives way to ambivalence about the whole thing.
It's difficult to capture the nuance in this in pithy tidbits, hence blog post on HN with me explaining this :).
- BonusPlay 2y agoFrom my experience, you either go full reproducible builds with nix, or none at all. Sitting in the middle results with additional downsides from modifying pipeline without core upsides of reproducible builds.
- kardos 2y agoThe reproducible builds effort means same input (code, libraries, compiler, flags, build-environment), should give the same output (build artifacts). I don't think RB is trying to get reproducible across platforms and across varying compilers/dependencies -- that is a much bigger and much harder goal. > There is always going to be a degree of un-reproducibility just due to the nature of math. Fundamentally the math is deterministic (reproducible): if you do the same sequence of mathematical operations you get the same result. I gather you're getting at the non-associativity of floating point (eg additions), which is a fair point, but if you arrange to do your floating point operations in the same sequence then it will be reproducible.
- bobmcnamara 2y agoFor these sorts of things, please do not assume hardware is reliable and compilers are bug free.
- kardos 2y agoSounds like a mechanism to test for bad hardware (akin to memtest86) and find compiler bugs =)
- bobmcnamara 2y agoBingo - even if we do everything right the percentage of matching builds isn't expected to be 100%. If it is, there just aren't enough samples. My first Pentium could usually but not always math correctly until Intel replaced it during a recall.
- yjftsjthsd-h 2y agoIn a slightly different form, this has already happened: https://julialang.org/blog/2020/09/rr-memory-magic/ https://julialang.org/blog/2020/09/rr-memory-magic/
- 8organicbits 2y agoBug free is less important for reproducible builds, so long as the bugs are reproducible. Compiler bugs are of course still a problem for other reasons.
- bobmcnamara 2y agoMy point is only that even when the software is correct, the reproducible build rate will either be 100% or we've got a representative sample set. Until then, those issues will continue to be lumped together with all the software reasons it wasn't reproducible.
- orbital-decay 2y agoMultithreaded code can easily be non-deterministic, with things like OS preemption leaking into the sandbox. Guaranteed determinism is hard, it needs a deterministic emulator.
- actionfromafar 2y agoSo we need our toolchains to run in WASM, too. Haha, only serious.
- torcete 2y agoSince hardware is the most difficult piece to replicate exactly for a reproducible build, should we say that for a complete reproducible environment we need an abstract virtual machine?