4 ms·

How you weight the entropy sources should be a function of the entropy sources, making it much more difficult to predict.

wahern 9y ago
Weighting functions are useless. If you mix entropy properly, in the absence of an _active_ attacker, the output will be no worse than the best entropy source. In the presence of an attacker they're just as useless, if not pernicious. Modern PRNG frameworks moved away from weighting years ago[1] because it not only has dubious benefit, but engineers suck at estimating entropy, so the estimates created a false sense of security. For example, the entropy estimates for hard drives were for a long time based on a single research publication, which as hard drives evolved over the years became increasingly irrelevant--assuming it wasn't irrelevant the day after it was published. Kernel engineers would try to add conservative margins, but if the original quantity has no basis in reality those efforts are just hand-waving. This is pretty much how all entropy estimates go. The research that goes into the estimates can be extremely rigorous and sound. The problem is not only ensuring the estimates are relevant to your mix of hardware, but that they remain relevant over time. New designs and new manufacturing methods come online on an almost daily basis, even for what appear to be identical products. Combined with the dubious benefit at the outset, trying to maintain the integrity of the estimates is little more than a fool's errand. With the exception of first-boot behavior, where you want to ensure there's "enough"[1] initial entropy, even accurate estimates don't provide any substantive value. Rather, mixing in a continuous stream of new entropy after you've gathered 128 or 256 bits can actually expose you to other issues, like exfiltration attacks that are more difficult to detect than they otherwise would be. (See https://blog.cr.yp.to/20140205-entropy.html https://blog.cr.yp.to/20140205-entropy.html) Because of the intrinsic difficulty of maintaining accurate estimates, some people would argue that the proper way to solve the first-boot issue is to not solve it at all. The only proper solution is an on-board, robust entropy source. Anything else provides a false sense of security (eventually if not initially), disincentivizing demand for and supply of systems with an on-board source.[2] The Via C3, circa 2003, provided the first on-chip source in the commodity x86 space. Unfortunately it took Intel over a decade to add the similar RDRAND, and longer still to make it ubiquitous across their product line. Now that AMD provides RDRAND and most ARM chips have something similar, we're nearly at a point where there's no excuse for entropy collection and estimation contrivances. IoT devices and other lower-end devices are still problematic, but those devices provide little if any hardware that entropy collection and estimation proponents can play with. Dealing with the long tail of legacy hardware is also a PITA. The PCEngines ALIX used an AMD SoC platform (Geode LX800) with an RNG source, but the more recent APU (G-series T40 and GX-412) lacks any on-board source.[3] This issue will eventually pass, but in the meantime solutions like the NeuG provide a solution. Which isn't to say that's the only use for NeuG. There's nothing wrong with using NeuG to supplement the vendor's RNG, just keep in mind issues like exfiltration attacks for trying to use too much of a good thing. [1] Compare Yarrow (1999) to Fortuna (2003). [2] By on-board I mean either on the chip, on a controller (both AMD and Intel put RNG sources on some select I/O controllers), or on a pre-installed device (e.g. HSM module on high-end Unix boxes). [3] There might actually be one on the NIC controller. I haven't been able to confirm, and in any event it's not detected by the OS and thus unsupported presuming there is one. Which alludes to another issue--the complexity created by supporting so many different mechanisms. All these device drivers and frameworks add attack surface.

wahern 9y ago

Weighting functions are useless. If you mix entropy properly, in the absence of an _active_ attacker, the output will be no worse than the best entropy source. In the presence of an attacker they're just as useless, if not pernicious. Modern PRNG frameworks moved away from weighting years ago[1] because it not only has dubious benefit, but engineers suck at estimating entropy, so the estimates created a false sense of security. For example, the entropy estimates for hard drives were for a long time based on a single research publication, which as hard drives evolved over the years became increasingly irrelevant--assuming it wasn't irrelevant the day after it was published. Kernel engineers would try to add conservative margins, but if the original quantity has no basis in reality those efforts are just hand-waving. This is pretty much how all entropy estimates go. The research that goes into the estimates can be extremely rigorous and sound. The problem is not only ensuring the estimates are relevant to your mix of hardware, but that they remain relevant over time. New designs and new manufacturing methods come online on an almost daily basis, even for what appear to be identical products. Combined with the dubious benefit at the outset, trying to maintain the integrity of the estimates is little more than a fool's errand. With the exception of first-boot behavior, where you want to ensure there's "enough"[1] initial entropy, even accurate estimates don't provide any substantive value. Rather, mixing in a continuous stream of new entropy after you've gathered 128 or 256 bits can actually expose you to other issues, like exfiltration attacks that are more difficult to detect than they otherwise would be. (See https://blog.cr.yp.to/20140205-entropy.html https://blog.cr.yp.to/20140205-entropy.html) Because of the intrinsic difficulty of maintaining accurate estimates, some people would argue that the proper way to solve the first-boot issue is to not solve it at all. The only proper solution is an on-board, robust entropy source. Anything else provides a false sense of security (eventually if not initially), disincentivizing demand for and supply of systems with an on-board source.[2] The Via C3, circa 2003, provided the first on-chip source in the commodity x86 space. Unfortunately it took Intel over a decade to add the similar RDRAND, and longer still to make it ubiquitous across their product line. Now that AMD provides RDRAND and most ARM chips have something similar, we're nearly at a point where there's no excuse for entropy collection and estimation contrivances. IoT devices and other lower-end devices are still problematic, but those devices provide little if any hardware that entropy collection and estimation proponents can play with. Dealing with the long tail of legacy hardware is also a PITA. The PCEngines ALIX used an AMD SoC platform (Geode LX800) with an RNG source, but the more recent APU (G-series T40 and GX-412) lacks any on-board source.[3] This issue will eventually pass, but in the meantime solutions like the NeuG provide a solution. Which isn't to say that's the only use for NeuG. There's nothing wrong with using NeuG to supplement the vendor's RNG, just keep in mind issues like exfiltration attacks for trying to use too much of a good thing. [1] Compare Yarrow (1999) to Fortuna (2003). [2] By on-board I mean either on the chip, on a controller (both AMD and Intel put RNG sources on some select I/O controllers), or on a pre-installed device (e.g. HSM module on high-end Unix boxes). [3] There might actually be one on the NIC controller. I haven't been able to confirm, and in any event it's not detected by the OS and thus unsupported presuming there is one. Which alludes to another issue--the complexity created by supporting so many different mechanisms. All these device drivers and frameworks add attack surface.