7 ms·

"Random number generation in OpenSSL suffers because entropy used to be really difficult. There were entropy files on disk that applications would read and writ

"Random number generation in OpenSSL suffers because entropy used to be really difficult. There were entropy files on disk that applications would read and write, timestamps and PIDs would be mixed into entropy pools and applications would try other tricks to gather entropy and mix it into the pool. That has all made OpenSSL complicated. BoringSSL just uses urandom—it's the right answer. (Although we'll probably do it via getrandom rather than /dev/urandom in the future.) There are no return values that you can forget to check: if anything goes wrong, it crashes the address space." There is a meme that pops up from time to time on HN, that "/dev/urandom is okay for stretching out random data but not sufficient for random seeds", that I've never understood. tptacek seems to be consistent in suggesting, "Just use /dev/urandom. Period." And it looks like people who also have a lot on the line concur. If this is the case, I'm wondering if we'll see a day when gpg might migrate from /dev/random to /dev/urandom?

eliteraspberrie 11y ago
libgcrypt (GPG) does the same thing OpenSSL did. It uses /dev/random to seed an internal PRNG. But you can switch to using just the system PRNG: gcry_control(GCRYCTL_SET_PREFERRED_RNG_TYPE, GCRY_RNG_TYPE_SYSTEM); I don't know if they would be convinced to use /dev/urandom though. They already rank /dev/random as having lower "security" than their own PRNG.
binarycrusader 11y ago
The advice of /dev/urandom vs. /dev/random is case-specific and doesn't apply to all operating systems. For Linux systems, in general, it would appear it doesn't matter and /dev/urandom is sufficient. However, on systems such as Solaris, the cryptography group (whom I know personally) specifically recommends the use of /dev/random in some cases. Those cases are largely limited to specific cryptography requirements such as a hard requirement for high entropy sources. Regardless, all of this is largely moot at this point, as all *NIX-like systems (including Solaris) seem to be standardising on the getrandom() (in Linux first) and getentropy() (in OpenBSD first) which sidestep many of the more subtle, potential issues.
- oconnor663 11y ago
  Linux is one of the cases where the difference matters a lot, because /dev/random is blocking. I think it's BSD where the two devices are the same.
  - binarycrusader 11y ago
    That doesn't match the repeated assertions of many posters here. I can only speak for Solaris knowledge-wise.
    - ghshephard 11y ago
      "That doesn't match the repeated assertions of many posters here" - I'm curious, what doesn't match?
    - keeperofdakeys 11y ago
      Here is a rough diagram which demonstrates it http://www.2uo.de/myths-about-urandom/structure-yes.png http://www.2uo.de/myths-about-urandom/structure-yes.png (from the detailed page http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/). /dev/random will block when the entropy estimate in low, but in a practical sense, the output of /dev/urandom and /dev/random are both random.
  - jakeogh 11y ago
    Better to block than spit out low entropy data.
    - oconnor663 11y ago
      That's true the way you say it, and that's definitely the problem that `getrandom()` solves by blocking if the random pool isn't initialized yet. But a lot of people take that to mean "you should never get more than N random bytes from a pool that has N bytes of entropy", and that part is wrong. All of crypto relies on being able to generate an arbitrary amount of good random bytes from a single 256-or-whatever-byte seed. Otherwise it wouldn't be safe to encrypt a long message with a short key.
pstrateman 11y ago
The problem is immediately after boot before anything has seeded the entropy pool. Like say when sshd generates host keys!
- elithrar 11y ago
  > The problem is immediately after boot before anything has seeded the entropy pool. With getrandom() on Linux it will block if you don't set GRND_NONBLOCK and the pool isn't initialized - http://man7.org/linux/man-pages/man2/getrandom.2.html http://man7.org/linux/man-pages/man2/getrandom.2.html - the desired behaviour.
- tangent128 11y ago
  Which is why the getrandom() syscall is preferred; it draws from the urandom pool, but will block until a quantity of initial entropy has been fed into the pool (ie, at early boot).
  - AnthonyMouse 11y ago
    Is there any actual reason why /dev/random doesn't do that?
    - ploxiln 11y ago
      getrandom() has a mode where it acts fully like /dev/random - and it's for some (misguided but very important to some big businesses) security certifications
    - KMag 11y ago
      Stubbornness of Linux kernel maintainers. My understanding that your proposed behavior is the behavior on FreeBSD and OS X, and people have proposed bringing Linux in line.
- tptacek 11y ago
  I'm not sure why this is supposed to be a tricky problem. Seed urandom from /dev/random, just for the side-effect of blocking, before you generate host keys. Never use /dev/random again.
nickpsecurity 11y ago
It comes from all the posts by smart people and even the man page saying to use /dev/random if security matters. Without digging into the two's code, I also took their word for it the few times I used it in my apps many years ago. Mostly just used CRNG's while dealing with entropy myself, though, so it was limited to a few designs & countering blocking. (Wasted effort in retrospect...) It took quite a while before someone said, "Hey, they go through the same CRNG." I'm like, WTF? Why have so many sources that should be knowledgeable about this been giving bad advice? Anyway, a number of write-ups (esp Huhn's) and people like Ptacek have been clearing up the issue well. Meme should weaken over time. Might help to straight-up delete the bad claims from whatever sources people are seeing them, though, where possible. Man page comes to mind if it hasn't been updated.
- Tomte 11y ago
  Thomas Pornin is relentlessly hunting down that meme on stackexchange. I guess Wikipedia would be the highest-profile source for most people. The English-speaking one mirrors the Linux man page, the German-speaking Wikipedia goes further and outright advises against /dev/urandom for "high requirements". This used to be even worse, though.
  - nickpsecurity 11y ago
    Yeah, the Wikipedia articles are a problem. I'll see what I can do when back in town on my main system.
zaroth 11y ago
It's just "because entropy used to be really difficult!" We now have relatively simple and proven algorithms (Yarrow, Fortuna) which can take 256-bits of entropy, and deliver effectively infinite streams of high quality entropy from that core without leaking any of the seed, or skewing the output. In such a system, pulling bits out of the core entropy bucket is counter-productive. As djb says; Cryptographers are certainly not responsible for this superstitious nonsense. Think about this for a moment: whoever wrote the /dev/random manual page seems to simultaneously believe that (1) we can't figure out how to deterministically expand one 256-bit /dev/random output into an endless stream of unpredictable keys (this is what we need from urandom), but (2) we _can_ figure out how to use a single key to safely encrypt many messages (this is what we need from ssl, pgp, etc.). For a cryptographer this doesn't even pass the laugh test. BSD and OS X are more modern than Linux in their choice of CS-PRNG; they used to use Yarrow, and then added Fortuna support in 2014 (behind a config flag) and will make it default in 11.0. Linux has used largely the same CS-PRNG for /dev/urandom since 1994, with some upgrades to their mixing function, to prevent DoS against /dev/random, and upgraded recently to include random bytes extracted from "RDRAM" provided by Intel CPUs. I'm not sure why we haven't seen the underlying CS-PRNG switched wholesale to Fortuna in Linux. For example, they are still using a SHA-1 based output hash, which seems like it might be time for that to go. The higher level take-away is that trying to avoid the modest code complexity of the CS-PRNG by going directly to /dev/random is more likely to lead to an exploitable catastrophic failure than sticking with /dev/urandom.

eliteraspberrie 11y ago

libgcrypt (GPG) does the same thing OpenSSL did. It uses /dev/random to seed an internal PRNG. But you can switch to using just the system PRNG: gcry_control(GCRYCTL_SET_PREFERRED_RNG_TYPE, GCRY_RNG_TYPE_SYSTEM); I don't know if they would be convinced to use /dev/urandom though. They already rank /dev/random as having lower "security" than their own PRNG.

binarycrusader 11y ago

The advice of /dev/urandom vs. /dev/random is case-specific and doesn't apply to all operating systems. For Linux systems, in general, it would appear it doesn't matter and /dev/urandom is sufficient. However, on systems such as Solaris, the cryptography group (whom I know personally) specifically recommends the use of /dev/random in some cases. Those cases are largely limited to specific cryptography requirements such as a hard requirement for high entropy sources. Regardless, all of this is largely moot at this point, as all *NIX-like systems (including Solaris) seem to be standardising on the getrandom() (in Linux first) and getentropy() (in OpenBSD first) which sidestep many of the more subtle, potential issues.

oconnor663 11y ago
Linux is one of the cases where the difference matters a lot, because /dev/random is blocking. I think it's BSD where the two devices are the same.
- binarycrusader 11y ago
  That doesn't match the repeated assertions of many posters here. I can only speak for Solaris knowledge-wise.
  - ghshephard 11y ago
    "That doesn't match the repeated assertions of many posters here" - I'm curious, what doesn't match?
  - keeperofdakeys 11y ago
    Here is a rough diagram which demonstrates it http://www.2uo.de/myths-about-urandom/structure-yes.png http://www.2uo.de/myths-about-urandom/structure-yes.png (from the detailed page http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/). /dev/random will block when the entropy estimate in low, but in a practical sense, the output of /dev/urandom and /dev/random are both random.
- jakeogh 11y ago
  Better to block than spit out low entropy data.
  - oconnor663 11y ago
    That's true the way you say it, and that's definitely the problem that `getrandom()` solves by blocking if the random pool isn't initialized yet. But a lot of people take that to mean "you should never get more than N random bytes from a pool that has N bytes of entropy", and that part is wrong. All of crypto relies on being able to generate an arbitrary amount of good random bytes from a single 256-or-whatever-byte seed. Otherwise it wouldn't be safe to encrypt a long message with a short key.

pstrateman 11y ago

The problem is immediately after boot before anything has seeded the entropy pool. Like say when sshd generates host keys!

elithrar 11y ago
> The problem is immediately after boot before anything has seeded the entropy pool. With getrandom() on Linux it will block if you don't set GRND_NONBLOCK and the pool isn't initialized - http://man7.org/linux/man-pages/man2/getrandom.2.html http://man7.org/linux/man-pages/man2/getrandom.2.html - the desired behaviour.
tangent128 11y ago
Which is why the getrandom() syscall is preferred; it draws from the urandom pool, but will block until a quantity of initial entropy has been fed into the pool (ie, at early boot).
- AnthonyMouse 11y ago
  Is there any actual reason why /dev/random doesn't do that?
  - ploxiln 11y ago
    getrandom() has a mode where it acts fully like /dev/random - and it's for some (misguided but very important to some big businesses) security certifications
  - KMag 11y ago
    Stubbornness of Linux kernel maintainers. My understanding that your proposed behavior is the behavior on FreeBSD and OS X, and people have proposed bringing Linux in line.
tptacek 11y ago
I'm not sure why this is supposed to be a tricky problem. Seed urandom from /dev/random, just for the side-effect of blocking, before you generate host keys. Never use /dev/random again.

nickpsecurity 11y ago

It comes from all the posts by smart people and even the man page saying to use /dev/random if security matters. Without digging into the two's code, I also took their word for it the few times I used it in my apps many years ago. Mostly just used CRNG's while dealing with entropy myself, though, so it was limited to a few designs & countering blocking. (Wasted effort in retrospect...) It took quite a while before someone said, "Hey, they go through the same CRNG." I'm like, WTF? Why have so many sources that should be knowledgeable about this been giving bad advice? Anyway, a number of write-ups (esp Huhn's) and people like Ptacek have been clearing up the issue well. Meme should weaken over time. Might help to straight-up delete the bad claims from whatever sources people are seeing them, though, where possible. Man page comes to mind if it hasn't been updated.

Tomte 11y ago
Thomas Pornin is relentlessly hunting down that meme on stackexchange. I guess Wikipedia would be the highest-profile source for most people. The English-speaking one mirrors the Linux man page, the German-speaking Wikipedia goes further and outright advises against /dev/urandom for "high requirements". This used to be even worse, though.
- nickpsecurity 11y ago
  Yeah, the Wikipedia articles are a problem. I'll see what I can do when back in town on my main system.

zaroth 11y ago

It's just "because entropy used to be really difficult!" We now have relatively simple and proven algorithms (Yarrow, Fortuna) which can take 256-bits of entropy, and deliver effectively infinite streams of high quality entropy from that core without leaking any of the seed, or skewing the output. In such a system, pulling bits out of the core entropy bucket is counter-productive. As djb says; Cryptographers are certainly not responsible for this superstitious nonsense. Think about this for a moment: whoever wrote the /dev/random manual page seems to simultaneously believe that (1) we can't figure out how to deterministically expand one 256-bit /dev/random output into an endless stream of unpredictable keys (this is what we need from urandom), but (2) we _can_ figure out how to use a single key to safely encrypt many messages (this is what we need from ssl, pgp, etc.). For a cryptographer this doesn't even pass the laugh test. BSD and OS X are more modern than Linux in their choice of CS-PRNG; they used to use Yarrow, and then added Fortuna support in 2014 (behind a config flag) and will make it default in 11.0. Linux has used largely the same CS-PRNG for /dev/urandom since 1994, with some upgrades to their mixing function, to prevent DoS against /dev/random, and upgraded recently to include random bytes extracted from "RDRAM" provided by Intel CPUs. I'm not sure why we haven't seen the underlying CS-PRNG switched wholesale to Fortuna in Linux. For example, they are still using a SHA-1 based output hash, which seems like it might be time for that to go. The higher level take-away is that trying to avoid the modest code complexity of the CS-PRNG by going directly to /dev/random is more likely to lead to an exploitable catastrophic failure than sticking with /dev/urandom.