2 ms·

The GCC developers in particular were hostile to such efforts for a long time, IIRC. (This is a non-trivial issue because randomized data structures exist and c

The GCC developers in particular were hostile to such efforts for a long time, IIRC. (This is a non-trivial issue because randomized data structures exist and can be a good idea to use: treaps, universal hashes, etc. I’d guess it also pays for compiler heuristics to be randomized sometimes. Incremental compilation is much harder to achieve when you require bit-for-bit identical output. Even just stripping your compile paths from debug info is not entirely straightforward.)

pas 5y ago
How/why was the randomness part not "solveable" via using fixed seeds?
- bruce343434 5y ago
  the security benefit of things like stack canaries rest on them being random and not known beforehand, I guess. Otherwise stack smashing malware could know to avoid them.
  - mananaysiempre 5y ago
    Wait, how is that relevant? Nothing says stack canaries have to use the same RNG as the main program, let alone the same seed, and there are cases such as this one where they probably shouldn’t, so it makes sense to separate them.
moonchild 5y ago
> Incremental compilation is much harder to achieve when you require bit-for-bit identical output Presumably, incremental compilation is only for development. For release, you would do a clean build, which would be reproducible. > Even just stripping your compile paths from debug info is not entirely straightforward Just use the same paths.
- mananaysiempre 5y ago
  > Presumably, incremental compilation is only for development. For release, you would do a clean build, which would be reproducible. I’d say that’s exactly the wrong approach: given how hard incremental anything is, it would make sense to insist on bit-exact output and then fuzz the everliving crap out of it until bit-exactness was reached. (The GCC maintainers do not agree.) But yes, you could do that. It’s not impossible to do reproducible builds with GCC 4.7 or whatever, it’s just intensely unpleasant, especially as a distro maintainer faced with yet another bespoke build system. (Saying that with all the self-awareness of a person making their own build system.) > Just use the same paths. I mean, sure, but then you have to build and debug in a chroot and waste half a day of your life figuring out how to do that and just generally feel stupid. And your debug info is still useless to anybody not using the exact same setup. Can’t we just embed relative paths instead, or even arbitrary prefixes it the code is coming from more than one place? In recent GCC versions we can, just chuck the right incantation into CPPFLAGS and you’re golden. All of this is not really difficult except insofar as getting a large and complicated program to do anything is difficult. (Stares in the direction of the 17-year-old Firefox bug for XDG basedir support.) That’s why I said it wasn’t a GCC problem so much as a maintainer attitude problem.
adonovan 5y ago
GCC used to attempt certain optimizations (or more generally, choose different code-generation strategies) only if there was plenty of memory available. We discovered this in the course of designing Google's internal build system, which prizes reproducibility.

pas 5y ago

How/why was the randomness part not "solveable" via using fixed seeds?

bruce343434 5y ago
the security benefit of things like stack canaries rest on them being random and not known beforehand, I guess. Otherwise stack smashing malware could know to avoid them.
- mananaysiempre 5y ago
  Wait, how is that relevant? Nothing says stack canaries have to use the same RNG as the main program, let alone the same seed, and there are cases such as this one where they probably shouldn’t, so it makes sense to separate them.

moonchild 5y ago

> Incremental compilation is much harder to achieve when you require bit-for-bit identical output Presumably, incremental compilation is only for development. For release, you would do a clean build, which would be reproducible. > Even just stripping your compile paths from debug info is not entirely straightforward Just use the same paths.

mananaysiempre 5y ago
> Presumably, incremental compilation is only for development. For release, you would do a clean build, which would be reproducible. I’d say that’s exactly the wrong approach: given how hard incremental anything is, it would make sense to insist on bit-exact output and then fuzz the everliving crap out of it until bit-exactness was reached. (The GCC maintainers do not agree.) But yes, you could do that. It’s not impossible to do reproducible builds with GCC 4.7 or whatever, it’s just intensely unpleasant, especially as a distro maintainer faced with yet another bespoke build system. (Saying that with all the self-awareness of a person making their own build system.) > Just use the same paths. I mean, sure, but then you have to build and debug in a chroot and waste half a day of your life figuring out how to do that and just generally feel stupid. And your debug info is still useless to anybody not using the exact same setup. Can’t we just embed relative paths instead, or even arbitrary prefixes it the code is coming from more than one place? In recent GCC versions we can, just chuck the right incantation into CPPFLAGS and you’re golden. All of this is not really difficult except insofar as getting a large and complicated program to do anything is difficult. (Stares in the direction of the 17-year-old Firefox bug for XDG basedir support.) That’s why I said it wasn’t a GCC problem so much as a maintainer attitude problem.

adonovan 5y ago

GCC used to attempt certain optimizations (or more generally, choose different code-generation strategies) only if there was plenty of memory available. We discovered this in the course of designing Google's internal build system, which prizes reproducibility.