5 ms·

I can't understand Why OpenSSL continues to use it's own PRNG implimentation when we have /dev/urandom and CryptGenRandom which are known to be good. This is b

I can't understand Why OpenSSL continues to use it's own PRNG implimentation when we have /dev/urandom and CryptGenRandom which are known to be good. This is basically what BoringSSL does (although if you have rdrand then it will get filtered through a ChaCha20 instance). I'm pretty sure OpenSSL doesn't even reseed its PRNG on Windows unless the calling application does it so I'm not sure how that's safe either. If you look at applications using OpenSSL like OpenVPN I don't see any calls to the PRNG init function to ensure it has enough entropy. I'm not sure of the security impact of this.

cyphar 11y ago
I think you mean /dev/random. On unixes, /dev/urandom produces a stream of random bytes that may not have high entropy, while /dev/random will block on reads until there's enough entropy in the pool.
- anaphor 11y ago
  nope, that's a myth http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/
  - cyphar 11y ago
    Interesting. I picked up that misconception from a Linux kernel comment a while ago. Thanks for telling me it's a myth. I knew that the two outputs came from the same source, I just assumed that the blocking had more-than-negligible effects.
  - slavik81 11y ago
    That link appears to confirm cyphar's assertion that /dev/urandom may produce a low-entropy output stream in certain circumstances. > Linux's /dev/urandom happily gives you not-so-random numbers before the kernel even had the chance to gather entropy. When is that? At system start, booting the computer.
    - geofft 11y ago
      One of the very early initscripts (on Debian, /etc/rcS.d/urandom or /lib/systemd/system/urandom.service, both of which happen well before normal initscripts) restores a random seed that was saved at last shutdown. After this script runs, the numbers are back to being cryptographically pseudorandom. Before this script runs, lots of other stuff isn't set up -- no networking, no remote filesystems, possibly not even swap -- so if you're writing code that needs to run there, you hopefully know that the system is in a weird state. And if you're writing something that runs a service that needs secure random numbers that early in the boot process, you're definitely doing something nonstandard and possibly misguided, and it's reasonable for the burden to be on you to take appropriate precautions, or switch to doing something normal. For normal application developers, who tend to wait until after the network is up to interact with the network, this special case doesn't apply.
      - slavik81 11y ago
        The usual time I've been concerned has been after creating a new virtual machine. Right after creating a new DO droplet I need to do a bunch of things to set up my application, like seed my CSRF token generator. Of course, /dev/random would block for ages at that point, so I instead generate the seed on a different machine.
        geofft 11y ago
        Ugh, yes, the practice of short-lived machines throws off the assumption of "when it was last shut down." You are totally right that this is a concern. The ideal solution to this would be for hypervisors to just pass a random seed to their guests. (There is even a full virtio-rng device in qemu, it just seems to have /dev/random semantics from a quick glance.) I don't know how we get to the point of convincing the big cloud providers to start doing this, though.
        jasonvorhe 11y ago
        Wouldn't it help to run something like havaged on the virtualization host that's feeding entropy to the virtualized nodes?
- ryuuchin 11y ago
  No I definitly mean /dev/urandom[1] or the getrandom syscall which would be equal to /dev/urandom. [1] http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ http://sockpuppet.org/blog/2014/02/25/safely-generate-random...
rootw0rm 11y ago
It depends upon which of the many functions you call to get your bits, iirc. I've been studying it a bit, though i've taken a little break, so i'd need to find my notes. it's certainly terrible on windows, but not completely broken...only semi-broken, lol. i think i remember there being some ancient reseeding mechanism that only works on XP in there somewhere too, but fuck it i'm drunk.
- erichurkman 11y ago
  Isn't 'semi-broken' equal to 'broken' for cryptographic purposes?
miles 11y ago
"/dev/urandom and CryptGenRandom which are known to be good" Check out "A good idea with bad usage: /dev/urandom": http://insanecoding.blogspot.com/2014/05/a-good-idea-with-bad-usage-devurandom.html http://insanecoding.blogspot.com/2014/05/a-good-idea-with-ba... Just stumbled onto it in the new submissions queue: https://news.ycombinator.com/item?id=11485876 https://news.ycombinator.com/item?id=11485876
- geofft 11y ago
  ... I'm not sure I can take an article seriously that suggests "but what if the attacker can modify files in /dev?". In any case, the meaningful concerns from that article have been addressed with the getrandom syscall on Linux, introduced a few months after this article was written: https://lwn.net/Articles/606141/ https://lwn.net/Articles/606141/ Perhaps we should start saying "getrandom" / "getentropy" instead of "/dev/urandom", but they're the same underlying CSPRNG (although getrandom has the distinct advantage of allowing you to tell if the urandom pool has been initialized, which /dev/urandom doesn't let you do), so I can understand being sloppy with usage. I would sort of assume anyone in a position to patch OpenSSL's RNG either upstream or in a distro is aware of getrandom and why it exists, but maybe that's a bad assumption.
  - ryuuchin 11y ago
    I forgot about the new getrandom syscall but yes that would be preferred if supported.
- maxerickson 11y ago
  That article says //Continue filling with other sources where it should say //exit with an error.
neerdowell 11y ago
/dev/urandom isn't reliable; it fails in a chroot, if file handles are exhausted, etc.
- jjnoakes 11y ago
  It fails in a poorly set up chroot, not any old chroot. It also fails in a poorly set up root file system in the same way. No chroot needed. I can't remember ever running out of file descriptors unless a program had a leak. But if you want to argue that position too, make sure you mention that cputime and memory could also be exhausted, leading to... well... any other method failing in a similar fashion. A system call definitely has some minor benefits over a file in /dev, bit the reverse is also true (access from shells, or any language really, with no built in support). But calling /dev/urandom unreliable is a little bit intellectually dishonest.
  - neerdowell 11y ago
    > It fails in a poorly set up chroot, not any old chroot. It fails for any program that calls chroot(2) and chroots to a location that hasn't had /dev/urandom constructed, which is... almost every program that sandboxes by calling chroot(2). > I can't remember ever running out of file descriptors unless a program had a leak. You've probably never used ulimits or login groups then. > But if you want to argue that position too, make sure you mention that cputime and memory could also be exhausted, leading to... well... any other method failing in a similar fashion. You're writing a library and in one of the functions you call arc4random(3). How would you handle memory being exhausted at the time when arc4random is called? Hint: you wouldn't. You'd let the OS handle it, probably by terminating something. Which might be the running process or another process. New situation, you're writing a library and in one of the functions you open() a file handle and read() from it. How would you handle that operation failing? Thought about it? Got your answer? Good. Now see the comments in LibreSSL for why your answer to that is wrong: https://github.com/libressl-portable/openbsd/blob/cb62fd8b9b1d3f0cdfb5445d0c53b560707a3376/src/lib/libcrypto/crypto/getentropy_linux.c#L120-L163 https://github.com/libressl-portable/openbsd/blob/cb62fd8b9b... > But calling /dev/urandom unreliable is a little bit intellectually dishonest. /dev/urandom is not a high-availability randomness source, therefore it is unreliable. It is not intellectually dishonest, it is a statement of fact.
caf 11y ago
Seeding a userspace CSPRNG from /dev/urandom or getrandom(2) is actually the right approach. The kernel pool is a resource shared among all processes, with the locking and therefore scalability constraints that implies.
- KMag 11y ago
  Also, it's a good idea to reseed your userspace CSPRNG from the kernel's entropy source periodically. This is precisely what OpenBSD did with their arc4random and arc4random_buf library calls. (Don't worry, they now use ChaCha20 instead of arcfour/RC-4, but have kept the function name unchanged, since it's a backward-compatible change... the only way a caller can tell RC-4 from ChaCha20 is by detecting small statistical biases in RC-4's outputs.)

cyphar 11y ago

I think you mean /dev/random. On unixes, /dev/urandom produces a stream of random bytes that may not have high entropy, while /dev/random will block on reads until there's enough entropy in the pool.

anaphor 11y ago
nope, that's a myth http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/
- cyphar 11y ago
  Interesting. I picked up that misconception from a Linux kernel comment a while ago. Thanks for telling me it's a myth. I knew that the two outputs came from the same source, I just assumed that the blocking had more-than-negligible effects.
- slavik81 11y ago
  That link appears to confirm cyphar's assertion that /dev/urandom may produce a low-entropy output stream in certain circumstances. > Linux's /dev/urandom happily gives you not-so-random numbers before the kernel even had the chance to gather entropy. When is that? At system start, booting the computer.
  - geofft 11y ago
    One of the very early initscripts (on Debian, /etc/rcS.d/urandom or /lib/systemd/system/urandom.service, both of which happen well before normal initscripts) restores a random seed that was saved at last shutdown. After this script runs, the numbers are back to being cryptographically pseudorandom. Before this script runs, lots of other stuff isn't set up -- no networking, no remote filesystems, possibly not even swap -- so if you're writing code that needs to run there, you hopefully know that the system is in a weird state. And if you're writing something that runs a service that needs secure random numbers that early in the boot process, you're definitely doing something nonstandard and possibly misguided, and it's reasonable for the burden to be on you to take appropriate precautions, or switch to doing something normal. For normal application developers, who tend to wait until after the network is up to interact with the network, this special case doesn't apply.
    - slavik81 11y ago
      The usual time I've been concerned has been after creating a new virtual machine. Right after creating a new DO droplet I need to do a bunch of things to set up my application, like seed my CSRF token generator. Of course, /dev/random would block for ages at that point, so I instead generate the seed on a different machine.
      - geofft 11y ago
        Ugh, yes, the practice of short-lived machines throws off the assumption of "when it was last shut down." You are totally right that this is a concern. The ideal solution to this would be for hypervisors to just pass a random seed to their guests. (There is even a full virtio-rng device in qemu, it just seems to have /dev/random semantics from a quick glance.) I don't know how we get to the point of convincing the big cloud providers to start doing this, though.
        jasonvorhe 11y ago
        Wouldn't it help to run something like havaged on the virtualization host that's feeding entropy to the virtualized nodes?
ryuuchin 11y ago
No I definitly mean /dev/urandom[1] or the getrandom syscall which would be equal to /dev/urandom. [1] http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ http://sockpuppet.org/blog/2014/02/25/safely-generate-random...

rootw0rm 11y ago

It depends upon which of the many functions you call to get your bits, iirc. I've been studying it a bit, though i've taken a little break, so i'd need to find my notes. it's certainly terrible on windows, but not completely broken...only semi-broken, lol. i think i remember there being some ancient reseeding mechanism that only works on XP in there somewhere too, but fuck it i'm drunk.

erichurkman 11y ago
Isn't 'semi-broken' equal to 'broken' for cryptographic purposes?

miles 11y ago

"/dev/urandom and CryptGenRandom which are known to be good" Check out "A good idea with bad usage: /dev/urandom": http://insanecoding.blogspot.com/2014/05/a-good-idea-with-bad-usage-devurandom.html http://insanecoding.blogspot.com/2014/05/a-good-idea-with-ba... Just stumbled onto it in the new submissions queue: https://news.ycombinator.com/item?id=11485876 https://news.ycombinator.com/item?id=11485876

geofft 11y ago
... I'm not sure I can take an article seriously that suggests "but what if the attacker can modify files in /dev?". In any case, the meaningful concerns from that article have been addressed with the getrandom syscall on Linux, introduced a few months after this article was written: https://lwn.net/Articles/606141/ https://lwn.net/Articles/606141/ Perhaps we should start saying "getrandom" / "getentropy" instead of "/dev/urandom", but they're the same underlying CSPRNG (although getrandom has the distinct advantage of allowing you to tell if the urandom pool has been initialized, which /dev/urandom doesn't let you do), so I can understand being sloppy with usage. I would sort of assume anyone in a position to patch OpenSSL's RNG either upstream or in a distro is aware of getrandom and why it exists, but maybe that's a bad assumption.
- ryuuchin 11y ago
  I forgot about the new getrandom syscall but yes that would be preferred if supported.
maxerickson 11y ago
That article says //Continue filling with other sources where it should say //exit with an error.

neerdowell 11y ago

/dev/urandom isn't reliable; it fails in a chroot, if file handles are exhausted, etc.

jjnoakes 11y ago
It fails in a poorly set up chroot, not any old chroot. It also fails in a poorly set up root file system in the same way. No chroot needed. I can't remember ever running out of file descriptors unless a program had a leak. But if you want to argue that position too, make sure you mention that cputime and memory could also be exhausted, leading to... well... any other method failing in a similar fashion. A system call definitely has some minor benefits over a file in /dev, bit the reverse is also true (access from shells, or any language really, with no built in support). But calling /dev/urandom unreliable is a little bit intellectually dishonest.
- neerdowell 11y ago
  > It fails in a poorly set up chroot, not any old chroot. It fails for any program that calls chroot(2) and chroots to a location that hasn't had /dev/urandom constructed, which is... almost every program that sandboxes by calling chroot(2). > I can't remember ever running out of file descriptors unless a program had a leak. You've probably never used ulimits or login groups then. > But if you want to argue that position too, make sure you mention that cputime and memory could also be exhausted, leading to... well... any other method failing in a similar fashion. You're writing a library and in one of the functions you call arc4random(3). How would you handle memory being exhausted at the time when arc4random is called? Hint: you wouldn't. You'd let the OS handle it, probably by terminating something. Which might be the running process or another process. New situation, you're writing a library and in one of the functions you open() a file handle and read() from it. How would you handle that operation failing? Thought about it? Got your answer? Good. Now see the comments in LibreSSL for why your answer to that is wrong: https://github.com/libressl-portable/openbsd/blob/cb62fd8b9b1d3f0cdfb5445d0c53b560707a3376/src/lib/libcrypto/crypto/getentropy_linux.c#L120-L163 https://github.com/libressl-portable/openbsd/blob/cb62fd8b9b... > But calling /dev/urandom unreliable is a little bit intellectually dishonest. /dev/urandom is not a high-availability randomness source, therefore it is unreliable. It is not intellectually dishonest, it is a statement of fact.

caf 11y ago

Seeding a userspace CSPRNG from /dev/urandom or getrandom(2) is actually the right approach. The kernel pool is a resource shared among all processes, with the locking and therefore scalability constraints that implies.

KMag 11y ago
Also, it's a good idea to reseed your userspace CSPRNG from the kernel's entropy source periodically. This is precisely what OpenBSD did with their arc4random and arc4random_buf library calls. (Don't worry, they now use ChaCha20 instead of arcfour/RC-4, but have kept the function name unchanged, since it's a backward-compatible change... the only way a caller can tell RC-4 from ChaCha20 is by detecting small statistical biases in RC-4's outputs.)