4 ms·

>Any packages that absolutely cannot be built in a reproducible way[1] ... [1] Such as random noise added to kernel and firmware data structures during local bu

>Any packages that absolutely cannot be built in a reproducible way[1] ... [1] Such as random noise added to kernel and firmware data structures during local builds, to be used as a last defense to avoid the herd using same keys effects, etc. Shouldn't this example of randomness still be pushed to the installation stage, instead of in the distribution. If Debian's binary package contains a "random" key, then we have a pretty large herd already using it.

blcknight 9y ago
It's a last defense, it's still a smaller herd than everyone
- nebabyte 9y ago
  Right, gizmo686 is arguing for a line of defense at a different place (installation) which I agree would be far more useful.
  - lambda 9y ago
    The point is that when installing, you are not rebuilding. The hypothetical example here is for some kind of randomizaiton that applies, for instance, to the kernel during build time, such as randomization of structure layout. Having to rebuild every kernel you install to get a security patch, on every system you install it on, would be an enormous amount of time spent (there's a reason Gentoo is a fairly niche distro). Such randomization can mitigate against certain kinds of attacks, especially making particular instances of an attack work differently between each kernel build, and thus increasing the difficulty of writing exploits that work across a broad range of kernel builds. There are about 30 different patch releases of the kernel for Debian Jessie alone; so such randomness, if applied across the different builds of a kernel within a release, across various different Debian releases, across different architectures, and also applied by other distros, can substantially increase the difficulty of writing an exploit that can affect a substantial fraction of Linux systems on the net. That helps provide a form of "herd immunity", where the herd is all deployed Linux systems on the net. It doesn't provide any real protection for a particular build; so it's no harder to write an exploit that targets all system running the exact same kernel. But it does dramatically increase the difficulty of writing a widespread worm which relies on the given exploit, and which can easily spread between a variety of different systems. Anyhow, that footnote was on a policy for explicit exceptions from the general policy. The general policy is that yes, builds should be reproducible, and any randomness should be generated locally. The footnote was giving a few example of potential exceptions which may be required, as a way of demonstrating that it would be fine to have the general requirement be reproducible builds with case-by-case, narrow exceptions for cases in which non-reproducible builds provide significant value. And note that such exceptions may be based on how an upstream project operates. If the upstream kernel has some modules that build non-reproducibly in such a way, it may be more viable to encode an explicit exception to the policy for that case, than to remove that non-reproducibility. It's important to keep in mind that someone applying a cleanup once accidentally removed almost all entropy from OpenSSL's random number generator on Debian (they even asked for review from upstream, and failed to get it, because of the confusing naming of upstream's mailing lists). Having an excessively rigid policy could mean that Debian maintainers would be forced to remove features that do provide some kind of benefit.
    - pgeorgi 9y ago
      > Having to rebuild every kernel you install to get a security patch, on every system you install it on, would be an enormous amount of time spent Rebuild? No. Just relink (and link in a just-generated binary containing the randomness seed). At boot time. Just what OpenBSD is experimenting with.
microcolonel 9y ago
Well even with packages like this, you could put the randomness in the manifest and just change it when you update the package. I don't see why there should be an exception for this if it's build-time randomness.
progval 9y ago
You don't always have randomness available at boot
- jacquesm 9y ago
  Sure but if it is hard baked into the distro it is no longer random any more than '4' is random.
- justinjlynn 9y ago
  It's a good thing most non-embedded systems have moved to event/dependency based inits then. Otherwise, you'll have to generate RNG state at install or image personalisation time and store RNG state between reboots to preserve available entropy. In the worst case, the system won't have persistent storage (to store RNG state for next boot) -- that's a rather specialised case and, in general, such systems have hardware randomness sources. Either way, it's a rather niche problem.
lamby 9y ago
> pushed to the installation stage, instead of in the distribution Indeed. During working on reproducibility in Debian, I found exactly this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833885 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833885
- exikyut 9y ago
  Nice. I'm very curious what sort of track you were chasing to spot that.
  - kseifried 9y ago
    grep -i secret/token/phrase/key/etc and you'll often find hard coded defaults =(.
  - lamby 9y ago
    Well, I did two builds of the package and they had different contents varying only on this secret key :) No special grep required!
    - exikyut 9y ago
      Oh of course. Rebuild the entire distro twice. Didn't think of that :)

blcknight 9y ago

It's a last defense, it's still a smaller herd than everyone

nebabyte 9y ago
Right, gizmo686 is arguing for a line of defense at a different place (installation) which I agree would be far more useful.
- lambda 9y ago
  The point is that when installing, you are not rebuilding. The hypothetical example here is for some kind of randomizaiton that applies, for instance, to the kernel during build time, such as randomization of structure layout. Having to rebuild every kernel you install to get a security patch, on every system you install it on, would be an enormous amount of time spent (there's a reason Gentoo is a fairly niche distro). Such randomization can mitigate against certain kinds of attacks, especially making particular instances of an attack work differently between each kernel build, and thus increasing the difficulty of writing exploits that work across a broad range of kernel builds. There are about 30 different patch releases of the kernel for Debian Jessie alone; so such randomness, if applied across the different builds of a kernel within a release, across various different Debian releases, across different architectures, and also applied by other distros, can substantially increase the difficulty of writing an exploit that can affect a substantial fraction of Linux systems on the net. That helps provide a form of "herd immunity", where the herd is all deployed Linux systems on the net. It doesn't provide any real protection for a particular build; so it's no harder to write an exploit that targets all system running the exact same kernel. But it does dramatically increase the difficulty of writing a widespread worm which relies on the given exploit, and which can easily spread between a variety of different systems. Anyhow, that footnote was on a policy for explicit exceptions from the general policy. The general policy is that yes, builds should be reproducible, and any randomness should be generated locally. The footnote was giving a few example of potential exceptions which may be required, as a way of demonstrating that it would be fine to have the general requirement be reproducible builds with case-by-case, narrow exceptions for cases in which non-reproducible builds provide significant value. And note that such exceptions may be based on how an upstream project operates. If the upstream kernel has some modules that build non-reproducibly in such a way, it may be more viable to encode an explicit exception to the policy for that case, than to remove that non-reproducibility. It's important to keep in mind that someone applying a cleanup once accidentally removed almost all entropy from OpenSSL's random number generator on Debian (they even asked for review from upstream, and failed to get it, because of the confusing naming of upstream's mailing lists). Having an excessively rigid policy could mean that Debian maintainers would be forced to remove features that do provide some kind of benefit.
  - pgeorgi 9y ago
    > Having to rebuild every kernel you install to get a security patch, on every system you install it on, would be an enormous amount of time spent Rebuild? No. Just relink (and link in a just-generated binary containing the randomness seed). At boot time. Just what OpenBSD is experimenting with.

microcolonel 9y ago

Well even with packages like this, you could put the randomness in the manifest and just change it when you update the package. I don't see why there should be an exception for this if it's build-time randomness.

progval 9y ago

You don't always have randomness available at boot

jacquesm 9y ago
Sure but if it is hard baked into the distro it is no longer random any more than '4' is random.
justinjlynn 9y ago
It's a good thing most non-embedded systems have moved to event/dependency based inits then. Otherwise, you'll have to generate RNG state at install or image personalisation time and store RNG state between reboots to preserve available entropy. In the worst case, the system won't have persistent storage (to store RNG state for next boot) -- that's a rather specialised case and, in general, such systems have hardware randomness sources. Either way, it's a rather niche problem.

lamby 9y ago

> pushed to the installation stage, instead of in the distribution Indeed. During working on reproducibility in Debian, I found exactly this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833885 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833885

exikyut 9y ago
Nice. I'm very curious what sort of track you were chasing to spot that.
- kseifried 9y ago
  grep -i secret/token/phrase/key/etc and you'll often find hard coded defaults =(.
- lamby 9y ago
  Well, I did two builds of the package and they had different contents varying only on this secret key :) No special grep required!
  - exikyut 9y ago
    Oh of course. Rebuild the entire distro twice. Didn't think of that :)