11 ms·

Myths about /dev/urandom (2014)

• schoen 10y ago
  A useful article, originally from 2014 but updated recently. Previous HN discussions: https://news.ycombinator.com/item?id=7359992 https://news.ycombinator.com/item?id=7359992 https://news.ycombinator.com/item?id=10149019 https://news.ycombinator.com/item?id=10149019
  • djsumdog 10y ago
    I remember seeing it before. It's a good post. What I think is really interesting is the question: how do you prove something is random. If you take a chunk of data that uses strong encryption, without headers or other identifiers, it should look like it has a random distribution. That's one of the reasons for the great concern around the Snowden leaks that implied some manufactures had compromised their hardware AES chips: https://theintercept.com/2016/01/04/a-redaction-re-visited-nsa-targeted-the-two-leading-encryption-chips/ https://theintercept.com/2016/01/04/a-redaction-re-visited-n...
• kbaker 10y ago
  tldr; just use /dev/urandom --- this should be at the top of the page. Maybe prefix the bold sections with Myth: so it is easier to read. I was really confused as to which side of the argument the author was on at first...
  • citrin_ru 10y ago
    Or better use getentropy(2)/getrandom(2) when it is possible.
    • JoshTriplett 10y ago
      True, but never pass GRND_RANDOM or GRND_NONBLOCK.
      • zrm 10y ago
        Why do these flags exist if they should never be used?
        • marcosdumay 10y ago
          There are probably good uses for them. But unless you are doing something very tightly integrated with Linux, you should just ignore them.
      • Gaelan 10y ago
        NONBLOCK just returns an error code instead of blocking. No harm in that, AFIAK.
        • marcosdumay 10y ago
          Just wait until people start using the EAGAIN value in keys.
    • eddieh 10y ago
      arc4random and arc4random_buf are probably better still.
      • nshepperd 10y ago
        Please don't use arc4random* for cryptographic purposes. RC4 is broken -- there are practical attacks against it. It's fine for non-secure purposes though (eg. monte carlo simulation).
        • tossedaway334 10y ago
          arc4random uses chacha20 on openBSD now, has for several years, OSX and FreeBSD are yet to update...
          • eddieh 10y ago
            The macOS Sierra man page says it uses the NIST-approved AES cipher and will be replaced as the techniques advance.
        • eddieh 10y ago
          The 'arc4' no longer refers to RC4 on macOS/iOS and OpenBSD. On those systems 'arc4random' is playfully a bacronym for "A Replacement Call for Random". The new arc4random* implementation will now be replaced as cryptographic techniques advance. It appears the Apple and OpenBSD implementations use the getentropy syscall and then add additional entropy mixing.
          • gbrown_ 10y ago
            Yup, OpenBSD have even changed calls to other entropy sources to returning strong random by default from arc4random. http://marc.info/?l=openbsd-cvs&m=141807513728073&w=2 http://marc.info/?l=openbsd-cvs&m=141807513728073&w=2
          • arthur2e5 10y ago
            libbsd is also using getentropy: https://cgit.freedesktop.org/libbsd/tree/src/arc4random.c https://cgit.freedesktop.org/libbsd/tree/src/arc4random.c
  • rconti 10y ago
    I, too, found it INCREDIBLY difficult to read. Perhaps because I don't have much understanding of random-number generators, but I had to re-read the first handful of entries about 3 times before I could catch on. I think a big part of it was my tendency to read the BOLD statements as the facts. Then when it said FACT beneath it, I had to start all over again.
• alpb 10y ago
  Also Filippo Varsaldo has a nice talk about this: https://www.youtube.com/watch?v=Q8JAlZ-HJQI https://www.youtube.com/watch?v=Q8JAlZ-HJQI
  • Foxboron 10y ago
    Please don't link to talks from CCCen. They contain adds and is not the offical channel from the C3VOC. Link to the mediacccde channel (https://www.youtube.com/user/mediacccde https://www.youtube.com/user/mediacccde) or better; https://media.ccc.de https://media.ccc.de. https://media.ccc.de/v/32c3-7441-the_plain_simple_reality_of_entropy https://media.ccc.de/v/32c3-7441-the_plain_simple_reality_of...
    • Encounter 10y ago
      Huh, they do a good job of making it look official. (Definitely more so than mediacccde.) I'll have to link from media.ccc.de from now on.
• bluedino 10y ago
  One to add - it's not fast at providing a value I remember learning this when I found /dev/urandom multiple times slower than /dev/null while writing garbage data to disk.
  • paulddraper 10y ago
    Oh really :)
• matt_wulfeck 10y ago
  This is really no problem now that Intel chips ship with RDRAND instructions. Linux should follow BSD/Mac's footsteps and incorporate a good CSPRNG non-blocking devices (after seed). I believe BSD is moving towards Fortuna, which is perfect for this use-case. With these instructions you'll always get a "good"[1] seed for your CSPRNG, and that includes virtual machines and clones. [1] Of course that depends on how much you trust Intel. Don't ask the crypto mailing lists whether this is a good idea :-)
  • hug 10y ago
    From Theodore Ts'o, 2012: "Create a new function, get_random_bytes_arch() which will use the architecture-specific hardware random number generator if it is present. Change get_random_bytes() to not use the HW RNG, even if it is avaiable. ... So it's much better to use the HW RNG to improve the existing random number generator, by mixing in any entropy returned by the HW RNG into /dev/random's entropy pool, but to always _use_ /dev/random's entropy pool." http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c2557a303ab6712bb6e09447df828c557c710ac9 http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...
    • ckuehl 10y ago
      I found interesting djb's blog post about a theoretical scenario where mixing entropy can actually be dangerous: https://blog.cr.yp.to/20140205-entropy.html https://blog.cr.yp.to/20140205-entropy.html
    • matt_wulfeck 10y ago
      I'm not sure I'd say it's better or worse. I think it's unnecessary. You only need like ~16 bytes of a random seed and your CSPRNG will run forever with unlimited unpredictable stream of bits. Intels instructions perform the same as a HWRNG only it's built in to every chipset >= Ivy bridge. I actually think Linux's rng-tools incorporates it as a mix automatically.
  • analogist 10y ago
    As of Kernel 4.8, Ted Ts'o has already switched /dev/urandom over to ChaCha20 (https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=818e607b57c94ade9824dad63a96c2ea6b21baf3 https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux....), so I would say a good CSPRNG's already done.
• jlgaddis 10y ago
  I bought a TrueRNG v3 [0] a month or so ago and have been using it with rngd / rng-tools. I wish rngd was easier to use with multiple entropy sources, however. Even w/ RDRAND (times two -- dual CPUs), a TPM, and the TrueRNG, it's difficult to (easily) tell which of these are being used and/or if more than one is being used. Ideally, I'd like to be able to tell rngd to use/mix the TrueRNG, RDRAND in both CPUs, the TPM, and any other entropy sources I may come up with, such as RTL-SDRs doing funky things [1]. I suppose I could just run multiple instances of rngd, though. Anyway, I've switched to just using /dev/random for pretty much everything (where it can be configured) since, with the TrueRNG, it never blocks on me now. [0]: http://ubld.it/truerng_v3 http://ubld.it/truerng_v3 [1]: https://www.google.com/search?q=rtl-str+entropy https://www.google.com/search?q=rtl-str+entropy
  • tptacek 10y ago
    Why would you do this? What problem is a USB hardware RNG solving for you?
    • jlgaddis 10y ago
      In this particular case, I'm evaluating it on my workstation. The intended use, however, is for an offline, headless machine (that is normally powered off) that is used to generate cryptographic keys. Do I absolutely require it? Probably not. For $50, though, I thought "might as well try it".
      • tptacek 10y ago
        I would recommend you not do this.
  • notfed 10y ago
    What I did instead was switch to /dev/urandom, which never blocks either.
  • pvg 10y ago
    These 'running out of entropy' graphs are not exactly a reassurance these people are interested in your security as much as in selling you something.
• ComodoHacker 10y ago
  So what are the cases when you should actually use /dev/random and not /dev/urandom? The article fails to address this in clear.
  • teamhappy 10y ago
    You shouldn't use /dev/random. The article makes that very clear.
    • ComodoHacker 10y ago
      /dev/random still exists for a reason, and I believe the reason isn't backward compatibility only.
      • marcosdumay 10y ago
        I do believe the reasons are exactly backward compatibility and politics¹. 1 - As in "oh just keep it there, it's easier than convincing everybody that it's useless".
      • teamhappy 10y ago
        "The /dev/random interface is considered a legacy interface, and /dev/urandom is preferred and sufficient in all use cases, with the exception of applications which require randomness during early boot time; ..." From http://man7.org/linux/man-pages/man4/urandom.4.html http://man7.org/linux/man-pages/man4/urandom.4.html
        • ComodoHacker 10y ago
          OK, "applications which require randomness during early boot time" is one such case. Thank you for pointing out.
          • tptacek 10y ago
            No, it isn't. During early boot time, you should use the system call interface, not the special files.
• technion 10y ago
  I've given up on the hope that these myths will ever die. Every time it's relevant, someone pipes up complaining urandom isn't safe for use, or will run out of entropy. Have a look at the bugs for Node, Ruby and Wordpress. And inevitably, the appeal to authority ends up referring to these projects. Serious question, if I submitted a patch for the man page detailing the content of this myths page, is there a chance it might go somewhere and put somewhat of an end to some of this?
  • pmiller2 10y ago
    Probably not. That would assume people actually read man pages.
    • technion 10y ago
      For end users I would agree. However: https://bugs.ruby-lang.org/issues/9569 https://bugs.ruby-lang.org/issues/9569 I many technical discussions, the man page becomes the compelling argument.
      • asdfaoeu 10y ago
        That's a different argument. Their usage is correct and you should use a CSPRNG to generate your random data like, arc4random or the ones provided by OpenSSL.
        • technion 10y ago
          No, you should not use a userspace CSPRNG. That's the whole point.
  • CiPHPerCoder 10y ago
    > Have a look at the bugs for Node, Ruby and Wordpress. Worth noting: WordPress has been using the OS's CSPRNG since 4.4. https://paragonie.com/blog/2015/10/coming-wordpress-4-4-csprng https://paragonie.com/blog/2015/10/coming-wordpress-4-4-cspr...
• CiPHPerCoder 10y ago
  The advice on this page is accurate and in line with what cryptography experts will recommend. That said, if you're looking for more of the how and less of the why: https://paragonie.com/blog/2016/05/how-generate-secure-random-numbers-in-various-programming-languages https://paragonie.com/blog/2016/05/how-generate-secure-rando... Contains snippets and recommendations for C/C++, Erlang, Go, Java, (Browser) JavaScript, .NET, Node.js, PHP, Python, Ruby, and Rust.
  • Dylan16807 10y ago
    Though you're answering a different (more relevant, but different) "how". That's "how do I find a library for random numbers". The article is focused on getting random numbers directly from the Linux kernel, and the "how" there is invoking getrandom(buffer, len, 0).
• deleted 10y ago
  [deleted]
• joelbondurant 10y ago
  If you think /dev/random and /dev/urandom are basically the same but for blocking, please consider getting sterilized.
  • grzm 10y ago
    please consider getting sterilized. There's never a need for uncivil personal attacks like this, regardless of the topic or how ignorant or wrong-headed you think someone to be.
    • joelbondurant 10y ago
      I said please.
      • sctb 10y ago
        This account has been posting mostly uncivil or unsubstantive comments. Please stop and re-read the guidelines before commenting again. https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
• joelbondurant 10y ago
  "And even if you need that peace of mind, let me tell you a secret: no practical attacks on AES, SHA-3 or other solid ciphers and hashes are known in the “unclassified” literature, either. Are you going to stop using those, as well? Of course not!" http://www.pcworld.com/article/3117728/security/why-quantum-computing-has-the-cybersecurity-world-white-knuckled.html http://www.pcworld.com/article/3117728/security/why-quantum-... https://en.wikipedia.org/wiki/Grover's_algorithm https://en.wikipedia.org/wiki/Grover's_algorithm
  • Tomte 10y ago
    "Practical attacks". "Quantum computers". I fail to see how these might be related.
    • joelbondurant 10y ago
      If you're working on storing cat videos on your personal website, I completely agree.
      • Tomte 10y ago
        No, I'm storing urandom articles on my personal website. Cat videos are much too popular and a lot bigger, and since I'm paying for traffic... [0] ;-) [0] regular traffic: ~30 MB/day, today (projected): ~3.5 GB
  • adrianN 10y ago
    Grover's algorithm, even assuming that we have hardware that can efficiently implement it, improves the runtime of a brute force attack on AES-256 from 10^49 * age of the universe to 10^10 * age of the universe. Hardly practical. All symmetric ciphers and all hashes are quantum-proof. Key exchange might become problematic, but that is a different problem
• deleted 10y ago
  [deleted]
• Paul-ish 10y ago
  While it may be generally better to use /dev/urandom, what about the Mining Your Ps and Qs paper?[1]. It finds that: > Every software package we examined relies on /dev/urandom to generate cryptographic keys; however, we find that Linux’s random number generator (RNG) can exhibit a boot-time entropy hole that causes urandom to produce deterministic output under conditions likely to occur in headless and embedded devices. In experiments with OpenSSL and Dropbear SSH, we show how repeated output from the system RNG can lead not only to repeated long-term keys but also to factorable RSA keys and repeated DSA ephemeral keys due to the behavior of application-specific entropy pools. This is mentioned a little at the end of the article. Would it be a breaking change for Linux to block urandom at startup? [1] https://factorable.net/weakkeys12.conference.pdf https://factorable.net/weakkeys12.conference.pdf
  • benchaney 10y ago
    It's unfortunate, but this issue lends a kernel of truth to all of the hysteria. /dev/urandom IS flawed. It is unfortunate, but the linux kernel has two different kernel RNG devices, and they are both different from proper security practice.
    • marcosdumay 10y ago
      The way it is now, the only recommendation you can give is to use the getrandom syscall. The special files are not foolproof.
  • ryuuchin 10y ago
    The way to work around this is to either use a method which blocks until seeded (getrandom) or to simply seed it yourself before using it.
    • Paul-ish 10y ago
      Wouldn't using a method that blocks mean using something like /dev/random, which is exactly what this article is against?
  • 10y ago
• Buge 10y ago
  /dev/urandom does have a problem with not enough entropy sometimes. Yes it never "runs out" of entropy when it has enough previously. But when a computer first boots up, sometimes it doesn't have enough entropy yet, and gives bad output. This causes problems in practice, allowing people to crack RSA private keys. https://factorable.net/weakkeys12.extended.pdf https://factorable.net/weakkeys12.extended.pdf
  • masklinn 10y ago
    Yes in that case you want to use getrandom/getentropy. Also note (as TFA indicates) that urandom not blocking on uninitialised entropy pool is mostly a Linux thing e.g. BSD urandom will block until the system the system is correctly initialised.
    • tveita 10y ago
      "that case" is every case unless you know and control exactly which OS, on which (potentially virtual) hardware and under which circumstances your code is going to run, now and in the future. That excludes anything that is meant to be used by other people. Fragile security is no security, and relying on undocumented assumptions like "is not running on Linux", "starts late in the boot process" and "is not running on an VM" is incredibly fragile. Unfortunately people who really should know better keep writing articles like this one, dispelling the "myth" that urandom works exactly as documented. Which leads us to situations like this: $ dmesg|grep random [ 0.469142] random: systemd-tmpfile: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.470297] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.470325] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.470877] random: udevadm: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.470890] random: udevadm: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.471936] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.471950] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.471969] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.472132] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 0.472142] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available) [ 1.909082] random: nonblocking pool is initialized Oops, hope none of that was for anything important, say generating a long-term cryptographic key! But of course if we're to believe TFA... "Fact: /dev/urandom is the preferred source of cryptographic randomness on UNIX-like systems." Just use getrandom. If you can, consider using only getrandom. But if your code must work on systems without getrandom or similar, before following the advice of this article, ask yourself if what you are doing is worse of blocking, or reading out a grand total 3 bits of entropy. You'd probably prefer the former.
• haarts 10y ago
  I am so confused about this all the time. My solution is to go see what Go is using: https://golang.org/src/crypto/rand/rand.go https://golang.org/src/crypto/rand/rand.go Trusting them to do the Right Thing. Is there a good reason why /dev/random AND /dev/urandom exist?
  • lmm 10y ago
    What do you want the behaviour to be when there isn't enough entropy to provide high-quality random numbers? For cases where you need the random number right now, you probably just want the best-quality number available (urandom). For cases where it's an important random number you'll be using for a long time, like SSH/SSL key generation, you probably want to block until more entropy is available (random).
    • tptacek 10y ago
      There's no such thing as a "low quality random number" from a CSPRNG. The outputs of a CSPRNG are either insecure, because the CSPRNG hasn't been seeded properly, or they're secure --- for all intents and purposes, forever. That's the problem with the old version of the Linux man page. The new version of the man page resolves this problem, and says outright that urandom is the preferred interface, and that /dev/random is obsolete; applications that run during early boot time should instead use the system call interface.
      • lmm 10y ago
        > The outputs of a CSPRNG are either insecure, because the CSPRNG hasn't been seeded properly In what sense is this not a "low quality random number"? Every CSPRNG I've seen will output numbers that pass many statistical tests for randomness even if seeded with e.g. zero - is that not a "low quality random number" in the usual sense of those words?
        • JdeBP 10y ago
          What they are getting at is that thinking of this in terms of the "quality of the randomness" is thinking about it in quite the wrong way that leads one right up the garden path; so stop even thinking about it like that. Discard that mental model. The randomness has the same quality. It's the same pseudo-random number generation algorithm. Only in one case, the world knows your seed value, and can predict anything that you do that derives from pseudo-randomness; whereas in the other case, the world does not know your seed value, should your seed value be discovered somehow you are regularly re-seeding anyway, and the world cannot predict your actions.
• haarts 10y ago
  I am so confused about this all the time. My solution is to go see what Go is using: https://golang.org/src/crypto/rand/rand.go https://golang.org/src/crypto/rand/rand.go Trusting them to do the Right Thing. Is there a good reason why /dev/random AND /dev/urandom exist?
• wruza 10y ago
  What I've got from this thread is that you never touch security-related prngs unless you're an expert in security, cryptography and operating systems, all at the same time, and even then you care a lot. Thanks for deeper debunking.
• anon335dtzbvc 10y ago
  its like aes 128 bit vs aes 256 bit, both are secure but when in doubt me, you and the nsa choose 256 bit