12 ms·
Uniting the Linux random-number devices
- Dork1234 5y agoCan Linux use Hardware RNG devices? Are these devices able to generate enough bits of randomness to work for boot?
- leeter 5y agoAppears so? https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/char/hw_random/ https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux... not sure that's used for /dev/random/ but I thought it was, but the question is when because early in the boot process that may not be loaded. There was also a general distrust of vendor specific hardware randoms in the past IIRC.
- goalieca 5y agoI do wonder if _fast_ quantum rng sources will become ubiquitous outside mobile applications.
- adgjlsfhk1 5y agoI doubt it. even if algorithms like sha get almost totally broken, you could get away with injecting a tiny number of bits of true randomness (like 1 in 2^20) and the result will be uncrackable.
- Someone 5y agoFTA: That entropy comes from sources like interrupt timing for various kinds of devices (e.g. disk, keyboard, network) and hardware RNGs if they are available. So yes, Linux can use hardware RNGs. Your second question probably is better stated as whether those can generate random bits at a sufficient rate. I would expect hardware RNGs of being capable of that for typical use cases.
- neatze 5y agoYou can run rng-tools to feed randomness from dedicated hardware like OneRNG or TrueRNG, it is substantially slower then /dev/urandom.
- zx2c4 5y agoYes. This happens via random.c's add_hwgenerator_randomness() hook, which the hwrng framework calls from a kthread.
- rurban 5y ago
- tinalumfoil 5y agoThe decision doesn't really make sense to me, and maybe I'm misunderstanding. So please correct me if I'm wrong. If the kernel aims to have 2 ways to get randomness: blocked until initialized (default) and best effort (ie GRND_INSECURE), and there's two device files why not map one to one and one to the other? It's easy to say why not: backwards compatibility and compatibility on systems without a good entropy source. Backwards compatibility because now any application that depended on non-blocking randomness in early boot is SOL, and just because it's hard to find an example doesn't mean no system will be affected by this. Systems without entropy because, because you have no guarantee from the hardware that (1) this jitter technique will work or (2) it will be secure. Putting (2) aside as "theoretical concerns", does this mean if I run Linux on a fully deterministic emulator the ext4 bug will lock up my system? That seems bad, why does my OS need to have entropy in the first place?
- majewsky 5y ago> If the kernel aims to have 2 ways to get randomness [...] and there's two device files why not map one to one and one to the other? Because both these files come with preconceived notions from various stages in the life of Unix regarding what guarantees they provide, and "best effort" works for neither of those. If this change had come in, say, 2005, they maybe could have gotten away with /dev/random = blocked until initialized and /dev/urandom = best effort, since that was the common wisdom at the time. But for the last 10 years, more and more people have switched to using /dev/urandom for everything since it is actually good enough for everything once initialized (and most application devs only care about the "once initialized" phase since they don't work on early-boot stuff). Switching /dev/urandom to GRND_INSECURE now would therefore be a potentially bad idea.
- tinalumfoil 5y agoBut saying /dev/urandom is best effort doesn't change those expectations, in fact it keeps those expectations the same. Saying most apps, "don't work on early-boot stuff" so aren't affected doesn't mean we should risk breaking systems who will inevitably have software that is going to run during early boot. It just feels like the argument for this change is, this is irrelevant for 99.99% of applications, so who cares? The 0.01% care! EDIT: > Switching /dev/urandom to GRND_INSECURE now would therefore be a potentially bad idea And, again, maybe I'm misunderstanding. The Jason Donenfeld email seems to say this is effectively the behavior we have. Ie, no guarantees of "initialization" or "sufficient entropy" on the urandom device.
- daneel_w 5y agoOpenBSD solved this problem long ago. Is the reason behind Linux' hesitancy/opposition to their solution of technical or philosophical nature?
- majewsky 5y agoHow did they solve it?
- daneel_w 5y agoThe bootloader seeds the kernel from disk, the kernel continually mixes data into the entropy pool from various sources. arc4random(), which provides for kernel/userspace/devices (including /dev/random which is symlinked to urandom), can never block. Add.: the seed file is unique per installation, and is also updated continously by the system.
- pdw 5y agoE.g. Linux can't assume that the system has a writable disk.
- daneel_w 5y agoNor can OpenBSD. The system doesn't crash if /etc/random.seed isn't writable.
- dspillett 5y agoBut what if it wasn't writeable recently but has been in the past? How do you know that the seed data isn't stale and you are starting up the entropy pool in exactly the same state as last time, and the time before that, and ... In some security contexts this could be a significant concern.
- daneel_w 5y agoThe seed file will go stale if you deny the system to update it. It's the first source for the entropy pool, but it's not the only source. I really have no idea how large effect the random subsystem suffers as a whole if that source is allowed to go stale.
- jepler 5y agoAs far as I can tell, there's little to nothing objectionable about this change; it makes urandom behave _more like_ random, by not yielding bytes before the kernel's entropy pool is in a good state (GRND_INSECURE). Systems where this would make urandom block for an objectionably long time (because CPU execution time jitter is unavailable or is believed to have low entropy) are largely hypothetical. I think you can still have specific reservations about CPU execution time jitter, though my experience and reading makes me believe this is probably a pretty good source of entropy; personally, I feel the ball is firmly in the court of jitter skeptics to show why the entropy measures from actual running systems are wildly high estimates. [https://www.chronox.de/jent/doc/CPU-Jitter-NPTRNG.html#toc-Appendix-C https://www.chronox.de/jent/doc/CPU-Jitter-NPTRNG.html#toc-A...] I also think you can still have specific reservations about how the kernel 'shepherds' its pool of random bits. I honestly am out of touch with the latest algorithms, both in research and in the Linux kernel. It would seem best if the kernel used a cryptographic algorithm where inferring the hidden random pool's state from outputs implies a useful attack on the cryptographic algorithm itself (i.e., has a proof of security). I don't think that Linux does this at the moment, based on recent discussion at https://lwn.net/ml/linux-kernel/20220201161342.154666-1-Jason@zx2c4.com/ https://lwn.net/ml/linux-kernel/20220201161342.154666-1-Jaso... But let's take a moment to happily reflect: for applications, running well after system boot-up has completed, it is now soooo easy to have your fill of cryptographic-quality random numbers than it was in the bad old days.
- zx2c4 5y ago> I think you can still have specific reservations about CPU execution time jitter, though my experience [...] Just want to point out that the Linus Jitter Dance is already in use today. It's been there for three years. I had nothing to do with that change. The change that I'm now proposing, which this article is about, changes nothing about the Linus Jitter Dance. Whether you like it or not, it's being used already, and has been for three years now, affecting all interfaces to the rng. I only mention it in my patch, for the sole purpose of indicating that blocking in /dev/urandom has been unproblematic for three years now, because it will unblock a second later. That's the only at all reason why I mention the Linus Jitter Dance. The only purpose of the patch is to make /dev/urandom block. > I also think you can still have specific reservations about how the kernel 'shepherds' its pool of random bits. [...] It would seem best if the kernel used a cryptographic algorithm Actually, it will do this for 5.18, authored a few weeks ago: https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/commit/?id=d232fc449c656c2bbc4ddec117c3b7c75a9cfa56 https://git.kernel.org/pub/scm/linux/kernel/git/crng/random....
- zx2c4 5y agoI just sent a v1 of this patch: https://lore.kernel.org/lkml/20220217162848.303601-1-Jason@zx2c4.com/ https://lore.kernel.org/lkml/20220217162848.303601-1-Jason@z... We'll see if that elicits any real objections. Hopefully not, and this will be part of 5.18!
- stingraycharles 5y agoSeems like a no-brainer to me; as I understand, this effectively make urandom “always secure”, which is a very good thing; makes for an even more convincing argument when some colleague insists on using /dev/random “because it’s more secure”. Do I understand correctly that with this patch, the only drawback is for weird architectures that do not have an instruction counter or another instruction to gather random data? And in those cases, the old behavior (“urandom may be insecure shortly after boot”) may even be worse than the new behavior (“may block while waiting for entropy”) ?
- zx2c4 5y agoRight, it unifies /dev/urandom, /dev/random, and getrandom(flags=0) to all do exactly the same thing. Most modern userspaces already use getrandom(flags=0). Nothing changes for them. They already count on the rng being seeded in one way or another. Rather, this changes /dev/urandom, which previously would give insecure randomness before being seeded. With this change, this doesn't happen any more, because it makes /dev/urandom wait until it has been seeded. In practice, the RNG get seeded by a large variety of things. As a last ditch effort, the Linus Jitter Dance will seed it. Taken together, what all the above amounts to is that the regression potential is limited to systems where: (A) /dev/urandom is still being used, rather than getrandom(flags=0), (B) the boot sequence, due to some bug, hard-depends on unseeded reads from /dev/urandom, (C) no ordinary sources of entropy, such as interrupts and input devices and disk drives, are available, (D) the CPU is so ancient as to be missing a cycle counter, defeating the last ditch Linus Jitter Dance, and (E) a new kernel will be installed on this old system. I argue that the set of machines where (A), (B), (C), (D), and (E) all hold is minuscule.
- 5y ago
- neatze 5y agoWhat would be performance with proposed changes when running command below ? dd if=/dev/random iflag=fullblock of=file.bin bs=1024 count=1024 status=progress
- zx2c4 5y agoNo changes. Totally unrelated.
- neatze 5y agoI meant /dev/urandom, sorry. Do you mean it is in reverse, in sense what getrandom() would be using ?
- SAI_Peregrinus 5y agoStill no changes. /dev/urandom will block until first seeded at boot. By the time you can run `dd`, it's seeded, because the system needs to have booted for that.
- westurner 5y agohttps://en.wikipedia.org/wiki//dev/random#Linux https://en.wikipedia.org/wiki//dev/random#Linux has : > In October 2016, with the release of Linux kernel version 4.8, the kernel's /dev/urandom was switched over to a ChaCha20-based cryptographic pseudorandom number generator (CPRNG) implementation [16] by Theodore Ts'o, based on Bernstein's well-regarded stream cipher ChaCha20. > In 2020, the Linux kernel version 5.6 /dev/random only blocks when the CPRNG hasn't initialized. Once initialized, /dev/random and /dev/urandom behave the same. [17]
- kroeckx 5y agoI'm still not really convinced about how much entropy is collected by the jitter entropy technique. I've been looking at https://github.com/smuellerDD/jitterentropy-library https://github.com/smuellerDD/jitterentropy-library previously, which is for instance used by OpenWRT. It's hard to do a proper estimation of the entropy, because depending on how you measure it, you get different results. The library has been changed, but it's probably still overestimating the entropy.
- EdSchouten 5y agoWhat I never understood is why the entire discussion revolves around the concept that the kernel gets launched in a vacuum where it is responsible for generating its own entropy. Why can’t the kernel (or the boot loader) restore/reload state from somewhere? I don’t buy the argument that it’s hard to build that. - For virtualized systems there could be a hypervisor call to request an initial random seed. - Run of the mill desktops/servers could use a reserved region (partition or boot record field) for preserving RNG state. - On embedded devices a boot loader like uBoot could load state from some piece of NVRAM/NAND/… There are hardly any platforms out there that are stateless in the literal sense. All of those approaches above would allow Linux to have a properly seeded RNG from the very first instruction that runs. No need to fully rely dubious things like timing execution/interrupts.
- eternityforest 5y agoSome applications really need to be stateless. What if you don't have wear leveling because you're on some bizzare embedded thing? You could have a one-time written random seed though, which would be enough when combined with an RTC, and would probably still be enough when combined with the almost universally present HWRNGs built into everything now. RTCs themselves nearly always seem to have about 56 bytes of memory, but I'd hate to see half of that used just for RTC, I'd rather it be exposed to applications via the FS somehow. But, it could be enough for a 64 bit counter, which would be combined with a one time written secret.
- derobert 5y agoA lot of these things do exist. Desktop/server Linux systems (used to at least) save some output from the PRNG to disk on shutdown and load it back on boot. But of course snapshots, cloning, etc. can foil that badly, causing the same seed to be used multiple times. And on initial install you're not going to have any of that (but initial install is also when you may need to generate long-lived random numbers like ssh host keys). Embedded devices it can be a real challenge. You must not re-use the seed data, so you effectively have to erase it from NVRAM/flash before use. But then if you lose power before you can generate a new one, you won't have one next boot. And you're adding flash writes, which decreases longevity and increases the chance of power failure in the middle of a write. Qemu/KVM has a virtual RNG so you can feed host randomness into the guests if you want. So there are hypervisor calls available.