9 ms·

Is there a good reason this doesn't just use urandom/getrandom? This doesn't look like it's in the "we're so early in boot I haven't restored the random seed ye

by lvh 7y ago

Is there a good reason this doesn't just use urandom/getrandom? This doesn't look like it's in the "we're so early in boot I haven't restored the random seed yet" case, for example.

jsnell 7y ago
No. There's a bad reason though. Want to guess what? https://github.com/systemd/systemd/blob/master/src/basic/random-util.c#L83 https://github.com/systemd/systemd/blob/master/src/basic/ran... Yes, the old "draining entropy" fallacy.
- dzdt 7y ago
  Why do you say draining entropy is a fallacy? It is certainly true that entropy recorded from I/O sources accumulates at a very limited rate.
  - deleted 7y ago
    [deleted]
  - i80and 7y ago
    There's no way to "use up" your entropy; this is a persistent myth that's been hanging around the Linux world for a very long time. * https://www.2uo.de/myths-about-urandom/#what-about-entropy-running-low https://www.2uo.de/myths-about-urandom/#what-about-entropy-r... * https://www.mail-archive.com/cryptography@randombit.net/msg04763.html https://www.mail-archive.com/cryptography@randombit.net/msg0...
  - deleted 7y ago
    [deleted]
  - wolf550e 7y ago
    Because after you gather ~256 bits of entropy from I/O sources (or rdrand^H^H^H rdseed [1]), using it to generate infinite output of /dev/urandom does not drain it. Adding more entropy only helps if the entropy pool inner state leaked, it is not needed to add more entropy because it drained through using it. Proof: 1. Take 256 bits of entropy. 2. Use HKDF to generate 128 bit key and 64 bit nonce. 3. Use key and nonce for AES-CTR with all possible 2^64 counter values to produce 2^68 bytes of output. 4. goto 2. If you can compute the inner state of the RNG (and thus predict future output) by just observing the output (not through side channels), using any amount of output, then all modern symmetric crypto is broken. If you can't, then using the entropy in an RNG does not drain the entropy pool. And here is a CCC talk about it: https://www.youtube.com/watch?v=OSfmtRc4VsE https://www.youtube.com/watch?v=OSfmtRc4VsE 1 - https://software.intel.com/en-us/blogs/2012/11/17/the-difference-between-rdrand-and-rdseed https://software.intel.com/en-us/blogs/2012/11/17/the-differ... EDIT: Thanks for 'dragontamer for pointing out difference between rdrand and rdseed.
    - dragontamer 7y ago
      > If you can compute the inner state of the RNG (and thus predict future output) by just observing the output (not through side channels), using any amount of output, then all modern symmetric crypto is broken. If you can't, then using the entropy in an RNG does not drain the entropy pool. This seems wrong to me. If you can predict #1 (the 256-bits of entropy in step 1), then you can break the system. This is highly likely, as Intel only assures 65-bits of entropy across two rdrand calls. (64-bits of entropy from the first call, +1 bit from the 2nd call, since RDRAND is generated from an internal random number generator. So there's going to be a correlation between the two values). In effect, if #1 is created with four calls to RDRAND, you only have around 68-bits of entropy, which can be brute-forced faster than a true source of 256-bits of entropy. Therefore, your RNG is broken. https://software.intel.com/en-us/blogs/2012/11/17/the-difference-between-rdrand-and-rdseed https://software.intel.com/en-us/blogs/2012/11/17/the-differ... RDSEED is the instruction that guarantees 64-bits of independent / multiplicative entropy, but executes slower as a result. ---------- In effect, you're "simplifying" the problem. #1, getting 256 bits of entropy, is the hard part of the problem. You cannot ignore this part of the problem. In any case, it seems to me that a large number of people don't know how to properly use the RDRAND function, in this thread as well as in the Github thread. RDRAND has a chance of failure, AND it doesn't even hold multiplicative entropy guarantees. So this seems like a case of EVERYONE hasn't read the docs yet. Please people, do NOT use RDRAND as a source of entropy. Use RDSEED as a source of entropy. RDRAND is only a random number generator with only 64-bits of entropy guaranteed at any given state, and probably additive entropy at that.
      - wolf550e 7y ago
        I can ignore "how to do #1" when discussing whether entropy is drained by using it or not. You need to get 256 bits of high quality entropy. We assume there are ways to do that. On a device that has no way to do that, you can't do crypto (unless you accept the entropy from outside, which is key escrow, but might be ok for an IoT device that communicates only to its mothership).
        dragontamer 7y ago
  - nwallin 7y ago
    Which one of these statements is false: 1. When secure your TLS 1.2 connection with a 128 bit AES key, you can transmit a small amount of data (on the order of ~128 bits) before the entropy is drained from your CSPRNG state. 2. When you generate at least ~128 bits of entropy in your kernel's CSPRNG state, you can generate a very large amount of random bits before the entropy is drained from your CSPRNG state.
toast0 7y ago
Or better yet, the getrandom syscall, which has better semantics and avoids the filesystem.
- lvh 7y ago
  I mentioned getrandom.
helper 7y ago
I'd love to read a "cryptographic doom principle"-esque latacora blog post on all the failures over the years of not using your OS provided CSRNG.
- loeg 7y ago
  https://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ https://sockpuppet.org/blog/2014/02/25/safely-generate-rando...
jontro 7y ago
Quoting from https://github.com/systemd/systemd/issues/11810#issuecomment-490233047 https://github.com/systemd/systemd/issues/11810#issuecomment... BTW, the reason we use RDRAND in some cases instead of getrandom() [which we use in many others] is that we need to generate uuids early on (since every service we starts gets one passed, the "invocation ID", and for other stuff too), but getrandom complains in dmesg or blocks if we call it before the pool is initialized. Since systemd is one of the earliest programs that runs and thus very likely comes into contact with an uninitialized pool we attempt to avoid that by using RDRAND when generating uuids, since it should be good enough for that, as the usecase needs a "mid-quality" rng source: not crypt quality and not totally guessable either.
- loeg 7y ago
  Why not use getrandom() with GRND_NONBLOCK? Worst case, they could run RDRAND in a loop and write() it into /dev/random until getrandom() is unblocked. Then they're still using the ordinary kernel random device, more or less. They wouldn't have these catastrophic collisions due to near-zero entropy.
  - JdeBP 7y ago
    Because systemd is not x86-64-only. The actual worst case is where there is no RDRAND, because then the systemd people have the chicken-and-egg situation of not being able to seed urandom without running a system service unit, and not being able to run a systemd service unit without being able to assign a random GUID to it as its instance ID. The design problem is that every service unit has an instance ID, whether meaningful and useful to it or not, the journal and other things have a machine ID, and journal files have header IDs. This requires that re-seeding urandom be pushed right to the start of the entire user-mode boot process (which begins with systemd in the initramfs on some operating systems), that something else (such as RDRAND) be used until urandom is re-seeded, or that the boot process simply stop and block running the very first service unit until urandom is re-seeded by the kernel.
    - lvh 7y ago

by lvh 7y ago

Is there a good reason this doesn't just use urandom/getrandom? This doesn't look like it's in the "we're so early in boot I haven't restored the random seed yet" case, for example.

jsnell 7y ago

No. There's a bad reason though. Want to guess what? https://github.com/systemd/systemd/blob/master/src/basic/random-util.c#L83 https://github.com/systemd/systemd/blob/master/src/basic/ran... Yes, the old "draining entropy" fallacy.

dzdt 7y ago
Why do you say draining entropy is a fallacy? It is certainly true that entropy recorded from I/O sources accumulates at a very limited rate.
- deleted 7y ago
  [deleted]
- i80and 7y ago
  There's no way to "use up" your entropy; this is a persistent myth that's been hanging around the Linux world for a very long time. * https://www.2uo.de/myths-about-urandom/#what-about-entropy-running-low https://www.2uo.de/myths-about-urandom/#what-about-entropy-r... * https://www.mail-archive.com/cryptography@randombit.net/msg04763.html https://www.mail-archive.com/cryptography@randombit.net/msg0...
- deleted 7y ago
  [deleted]
- wolf550e 7y ago
  Because after you gather ~256 bits of entropy from I/O sources (or rdrand^H^H^H rdseed [1]), using it to generate infinite output of /dev/urandom does not drain it. Adding more entropy only helps if the entropy pool inner state leaked, it is not needed to add more entropy because it drained through using it. Proof: 1. Take 256 bits of entropy. 2. Use HKDF to generate 128 bit key and 64 bit nonce. 3. Use key and nonce for AES-CTR with all possible 2^64 counter values to produce 2^68 bytes of output. 4. goto 2. If you can compute the inner state of the RNG (and thus predict future output) by just observing the output (not through side channels), using any amount of output, then all modern symmetric crypto is broken. If you can't, then using the entropy in an RNG does not drain the entropy pool. And here is a CCC talk about it: https://www.youtube.com/watch?v=OSfmtRc4VsE https://www.youtube.com/watch?v=OSfmtRc4VsE 1 - https://software.intel.com/en-us/blogs/2012/11/17/the-difference-between-rdrand-and-rdseed https://software.intel.com/en-us/blogs/2012/11/17/the-differ... EDIT: Thanks for 'dragontamer for pointing out difference between rdrand and rdseed.
  - dragontamer 7y ago
    > If you can compute the inner state of the RNG (and thus predict future output) by just observing the output (not through side channels), using any amount of output, then all modern symmetric crypto is broken. If you can't, then using the entropy in an RNG does not drain the entropy pool. This seems wrong to me. If you can predict #1 (the 256-bits of entropy in step 1), then you can break the system. This is highly likely, as Intel only assures 65-bits of entropy across two rdrand calls. (64-bits of entropy from the first call, +1 bit from the 2nd call, since RDRAND is generated from an internal random number generator. So there's going to be a correlation between the two values). In effect, if #1 is created with four calls to RDRAND, you only have around 68-bits of entropy, which can be brute-forced faster than a true source of 256-bits of entropy. Therefore, your RNG is broken. https://software.intel.com/en-us/blogs/2012/11/17/the-difference-between-rdrand-and-rdseed https://software.intel.com/en-us/blogs/2012/11/17/the-differ... RDSEED is the instruction that guarantees 64-bits of independent / multiplicative entropy, but executes slower as a result. ---------- In effect, you're "simplifying" the problem. #1, getting 256 bits of entropy, is the hard part of the problem. You cannot ignore this part of the problem. In any case, it seems to me that a large number of people don't know how to properly use the RDRAND function, in this thread as well as in the Github thread. RDRAND has a chance of failure, AND it doesn't even hold multiplicative entropy guarantees. So this seems like a case of EVERYONE hasn't read the docs yet. Please people, do NOT use RDRAND as a source of entropy. Use RDSEED as a source of entropy. RDRAND is only a random number generator with only 64-bits of entropy guaranteed at any given state, and probably additive entropy at that.
    - wolf550e 7y ago
      I can ignore "how to do #1" when discussing whether entropy is drained by using it or not. You need to get 256 bits of high quality entropy. We assume there are ways to do that. On a device that has no way to do that, you can't do crypto (unless you accept the entropy from outside, which is key escrow, but might be ok for an IoT device that communicates only to its mothership).
      - dragontamer 7y ago
- nwallin 7y ago
  Which one of these statements is false: 1. When secure your TLS 1.2 connection with a 128 bit AES key, you can transmit a small amount of data (on the order of ~128 bits) before the entropy is drained from your CSPRNG state. 2. When you generate at least ~128 bits of entropy in your kernel's CSPRNG state, you can generate a very large amount of random bits before the entropy is drained from your CSPRNG state.

toast0 7y ago

Or better yet, the getrandom syscall, which has better semantics and avoids the filesystem.

lvh 7y ago
I mentioned getrandom.

helper 7y ago

I'd love to read a "cryptographic doom principle"-esque latacora blog post on all the failures over the years of not using your OS provided CSRNG.

loeg 7y ago
https://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ https://sockpuppet.org/blog/2014/02/25/safely-generate-rando...

jontro 7y ago

Quoting from https://github.com/systemd/systemd/issues/11810#issuecomment-490233047 https://github.com/systemd/systemd/issues/11810#issuecomment... BTW, the reason we use RDRAND in some cases instead of getrandom() [which we use in many others] is that we need to generate uuids early on (since every service we starts gets one passed, the "invocation ID", and for other stuff too), but getrandom complains in dmesg or blocks if we call it before the pool is initialized. Since systemd is one of the earliest programs that runs and thus very likely comes into contact with an uninitialized pool we attempt to avoid that by using RDRAND when generating uuids, since it should be good enough for that, as the usecase needs a "mid-quality" rng source: not crypt quality and not totally guessable either.

loeg 7y ago
Why not use getrandom() with GRND_NONBLOCK? Worst case, they could run RDRAND in a loop and write() it into /dev/random until getrandom() is unblocked. Then they're still using the ordinary kernel random device, more or less. They wouldn't have these catastrophic collisions due to near-zero entropy.
- JdeBP 7y ago
  Because systemd is not x86-64-only. The actual worst case is where there is no RDRAND, because then the systemd people have the chicken-and-egg situation of not being able to seed urandom without running a system service unit, and not being able to run a systemd service unit without being able to assign a random GUID to it as its instance ID. The design problem is that every service unit has an instance ID, whether meaningful and useful to it or not, the journal and other things have a machine ID, and journal files have header IDs. This requires that re-seeding urandom be pushed right to the start of the entire user-mode boot process (which begins with systemd in the initramfs on some operating systems), that something else (such as RDRAND) be used until urandom is re-seeded, or that the boot process simply stop and block running the very first service unit until urandom is re-seeded by the kernel.
  - lvh 7y ago