25 ms·

How to Generate Secure Random Numbers in Various Programming Languages

245 points by CiPHPerCoder 10y ago

hamandcheese 10y ago
Can anyone elaborate on why Ruby's SecureRandom shouldn't be used? I find this to be rather surprising and discomforting.
- tptacek 10y ago
  It defaults to the OpenSSL userland CSPRNG, if available, because Ruby's maintainers believe (incorrectly) that urandom is unsafe for key generation.
  - brandur 10y ago
    The README of the gem that the article recommends ("sysrandom") provides a good explanation [1]. And here is the contentious Ruby issue wherein the Ruby core developers insist on cargo culting according to an outdated man page despite comprehensive refutation from a number of experienced security engineers [2]. That said, the arguments against OpenSSL seem to be largely theoretical at this point ("has been a source of vulnerabilities in Ruby ..."), so the misgivings about `SecureRandom` may be slightly exaggerated. [1] https://github.com/cryptosphere/sysrandom#why https://github.com/cryptosphere/sysrandom#why [2] https://bugs.ruby-lang.org/issues/9569 https://bugs.ruby-lang.org/issues/9569
    - scrollaway 10y ago
      What a disaster of a bug report, on so many levels. Speaking as a foss maintainer, to other foss maintainers out there: When you're systematically adopting a defensive attitude towards someone who believes found an important flaw in your project, you're the one being rude. You're refusing to hear people out. I see this awful attitude far too often. On Github, it's even worse nowadays where a lot of maintainers immediately fall back to locking the bug report so that they don't have to think about it anymore. At least here there is some form of discussion going on. But that's with a lot of fingers in a lot of ears, and nobody trying to be practical...
    - raverbashing 10y ago
      > uby issue wherein the Ruby core developers insist on cargo culting according to an outdated man page despite comprehensive refutation from a number of experienced security engineers [2]. Why doesn't this surprise me...
    - Nokinside 10y ago
      Oh man. There is huge undiscovered land of material for cross-cultural and comparative research in communication in these developer forums. Ruby and Linux mailing lists could provide material for number of really interesting studies in many fields. Ruby community has hint of Japanese culture: design aesthetics, simplicity, importance of politeness, saving face and rigidity. Linux community has hint of Finnish culture: practicality, "pig energy" and "management by perkele" combined with 80's competitive hacker culture.
      - tptacek 10y ago
        This is a case where both the Ruby community and the Linux community are standing in the way of security.
        Nokinside 10y ago
        What's wrong with random numbers in Linux?
        tptacek 10y ago
        They have a /dev/random interface which (literally) randomly blocks, disrupting programs, and a man page that instructs the Ruby community to use /dev/random and not /dev/urandom.
        Nokinside 10y ago
        /dev/random is the blocking pseudorandom number generator in Unix-like operating systems and it provides only the entropy that can be obtained from environmental noise. I don't see how it can work any other way. What is the manpage we are discussing here? I think getrandom(2) and random(4) are quite good.
        tptacek 10y ago
        /dev/random keeps a counter that is effectively random that tells the interface to block. /dev/urandom has the same mechanics as /dev/random, the same inputs, and the same cryptographic constructions, but does not have the random blocking counter. /dev/urandom is the interface you want. /dev/random is unsuitable for production code because, again, it randomly blocks, which can freeze your application. Worse still: applications that use /dev/random tend to "work around" the problem by using /dev/random to seed a userspace RNG that doesn't block. That's not just inconvenient but overtly unsafe.
    - ufo 10y ago
      I think the maintainer has a point in saying that the `man 4 random` page should have been updated if its giving wrong advice. I just checked and on my system that manpage still says all that wrong stuff about /dev/urandom. Why is it still unchanged?
      - tptacek 10y ago
        Because Linux man pages are sometimes dumb. I can give you a more precise answer but I don't think it will improve the thread. You should, however, take my word for it. Or, better yet, take Daniel Bernstein's word for it. Take Adam Langley's implied word for it (he's behind crypto/rand in Go). Take Thomas Pornin's word for. You can take Thomas Pornin's word on almost anything! Take Filippo Valsorda's word for it. Or Thomas Huhn's. The man page is wrong. Should it be fixed? Yes, it should be fixed. But, no excuses. You can't introduce security problems out of obstinacy about man pages.
        CiPHPerCoder 10y ago
        You totally can, but more importantly, you shouldn't. :)
    - joepie91_ 10y ago
      > That said, the arguments against OpenSSL seem to be largely theoretical at this point I would disagree. This is a recent paper on the state of OpenSSL's userspace PRNG, for example, finding several issues: https://eprint.iacr.org/2016/367.pdf https://eprint.iacr.org/2016/367.pdf
  - veeti 10y ago
    Is that Linux man page ever going to be corrected?
    - cpach 10y ago
      I sure hope so! But don't hold your breath...
    - pwg 10y ago
      Given that those man-pages are a community open source project much like the kernel, the best way to see it fixed is to contribute a fix: https://www.kernel.org/doc/man-pages/contributing.html https://www.kernel.org/doc/man-pages/contributing.html
      - chowells 10y ago
        In this case, that's far from true. Many people have contributed correct documentation. Whoever is in charge of these things continually rejects it because of their mistaken beliefs about PRNGs.
      - deleted 10y ago
        [deleted]
    - Sir_Cmpwn 10y ago
      There's a patchset that's been under discussion for a while.
  - eveningcoffee 10y ago
    Anecdotal case here. I did a research about usage of secure random number generation few year ago and gave up with OpenSSL. They used indirection upon indirection and it was really hard to understand what would be actually called. Therefore I was really not surprised when problems related to OpenSSL random generation started to pup up later on.
- wastedhours 10y ago
  I always use SecureRandom to generate unique IDs, but not to actually secure things - am I to take it that it would still be wise to switch to Sysrandom in that case as the vuln is in going through OpenSSL to do it (rather than the generated random not being random etc)?
billhathaway 10y ago
Does Go's crypto/rand[0] package meet this criteria? [0] https://golang.org/src/crypto/rand/rand_linux.go https://golang.org/src/crypto/rand/rand_linux.go
- tptacek 10y ago
  Yes, use crypto/rand for Go.
- niftich 10y ago
  +1 to this question. If the rand_linux strategy linked above decides to fall back to rand_unix, it can result in a userspace AES-based generator [1] running on output from /dev/random [1] https://golang.org/src/crypto/rand/rand_unix.go https://golang.org/src/crypto/rand/rand_unix.go
  - tptacek 10y ago
    Unless you're on Plan 9, this isn't a concern.
niftich 10y ago
Some additional requests: - Rust (no out-of-the-box crypto lib AFAIK) - Erlang (ships-by-default 'crypto' uses openssl) - Java 7 and below are, unfortunately, still in common use, despite being end-of-life. There is a config file to be edited, or a switch to be passed to the java executable to switch the RNG source away from /dev/random, the pre-8 SecureRandom default. - Some obvious note about how client-side Javascript is a lost cause, in case anybody's tempted
- indolering 10y ago
  > Some obvious note about how client-side Javascript is a lost cause, in case anybody's tempted crypto.getRandomBytes is supported by everything that matters[0], won't be a problem in ~5 years. 0: http://caniuse.com/#feat=getrandomvalues http://caniuse.com/#feat=getrandomvalues
- matt_wulfeck 10y ago
  If on Linux it's almost always safe to read from /dev/urandom.
  - vram22 10y ago
    > it's almost always safe When is it not?
    - matt_wulfeck 10y ago
      If you have a fresh VM without a seed file, that isn't connected to the internet and has had little in the way of devices, and you immediately begin generating keys before the normal file system operations stir the generator, then the deterministic nature of the generator says you'll get a predictable key, or more likely, end up generating a key that's simply not unique. Just by being online and running for a few seconds will cause enough entropy to put the generator into a safe state, so this rarely happens in practice. In fact Ubuntu comes prepackaged with pollinate[0] which makes this basically a non-issue for everyone that's not wearing a tinfoil hat. 1. http://manpages.ubuntu.com/manpages/trusty/man1/pollinate.1.html http://manpages.ubuntu.com/manpages/trusty/man1/pollinate.1....
hackcasual 10y ago
> Other Unix-like (including OS X): /dev/urandom Do other OSes ensure there's enough initial entropy to safely use urandom?
- wahern 10y ago
  1) Define "enough" in terms of each and every specific piece of hardware, combined with the mixing heuristics and functions in every kernel of interest. It's one thing to say you want N bits of entropy; it's quite another to accurately quantify it. (Count the number of times you see a word like "assume", "estimate", etc, in papers discussing kernel entropy estimation, and you'll never look at entropy estimation the same way.) 2) If you managed that and were able to conclude in the negative for a specific combination of kernel, kernel revision, and hardware models and revisions, what then? Are you going to recreate all that infrastructure in your app or through a gargantuan dependency along the lines of egd or havaged (which we've already learned from experience is a poor idea)? My point being, the question is arguably irrelevant. If available entropy at boot time (usually first boot, as the shutdown/startup process recycles entropy pools) is insufficient, better to pressure vendors (software and hardware) to address it than to add another layer of cruft or a complex mitigation. The fix will probably be out in the next year or so (indeed, with RDRAND has been out for years); the hack will linger for far longer, perhaps eventually becoming the weakest link in the chain. For similar reasons I think the article's as well as libsodium's choice to only use the most recent arc4random implementation as shipped in recent OpenBSD releases is misguided. Every other *BSD provides arc4random, and arguably applications should always prefer to use the libc's arc4random implementation. Eventually they'll be upgraded, at which point your application will have a useless dependency adding at best needless complexity and at worst vulnerabilities. Once you look behind the interface of arc4random, you're obliged to look behind the interface of /dev/urandom, etc, and I doubt it will look much better. Until the recent overhaul (last month), the PRNG infrastructure in Linux relied on SHA-1, which isn't much better than RC4, especially when considering that the RC4 used in the classic arc4random implementations discarded the first 1k of output and reseeded regularly. SHA-1 hasn't been recommended for cryptographic purposes for several years, and while you could make pragmatic arguments for why it was still acceptable, you could do the same for the particular use of RC4 in older arc4random implementations. I don't see libsodium skipping /dev/urandom for kernels before 4.9. At some point you have to stop bending over backwards to support platforms with poor QoS and leave things as you found them as long as they're not utterly broken. If experience with egd and, more generally, the boots-and-suspenders approach to security is any measure, classic arc4random is well within the bounds of reasonable. Simplicity is the rule of the day; experience has taught that complexity is invariably a net loss in security, especially when the weakest link remains unchanged.
atheg33 10y ago
Further advancement in secure random number generation across languages will probably lead to more "don't roll your own crypto" reminder posts in the near future. But it's still a great feature to have in every programming language imho.
alrs 10y ago
An example in Go: https://github.com/alrs/radseed https://github.com/alrs/radseed
sjclemmy 10y ago
A few years ago I worked for a gift card manufacturer and I was tasked with creating random PAN numbers for a gift card product for a large high street retailer (sadly no longer with us). I read up on it and couldn't find any decent guide as to how to produce CSPRNG - and to be fair, there was very little information in 'layman' terms about what CS meant in this context. As I couldn't be certain that any of the software routes available to me were secure I ended up purchasing this http://www.protego.se/ http://www.protego.se/ to generate the random numbers. I am not convinced that random numbers were required for the security element of the gift cards, but I don't believe there were any reports of any fraudulent activity based on unknown numbers. Developments in this arena over the last 10 years would have made my job a lot easier!
- deleted 10y ago
  [deleted]
- koolba 10y ago
  > I read up on it and couldn't find any decent guide as to how to produce CSPRNG - and to be fair, there was very little information in 'layman' terms about what CS meant in this context. When you say "produce CSPRNG" do you mean writing a generator or simply generating numbers? Nobody[1] should be writing a CSPRNG and app developers should be using language/OS functions (ex: /dev/urandom). > As I couldn't be certain that any of the software routes available to me were secure I ended up purchasing this http://www.protego.se/ http://www.protego.se/ to generate the random numbers. Wow did you really look at that page and say, "This is going to be easier and more secure than using /dev/urandom?". Reading the rest of the info on the page, http://www.protego.se/trnsup.htm http://www.protego.se/trnsup.htm, does not give me a lot of confidence in the product. The explanation page of how it works is just a bunch of scare info against using a software solution. [1]: Okay obviously someone has to do it but you all know what I mean...
  - _asummers 10y ago
    Prefer to use getrandom instead of /dev/urandom now, if your OS supports it. http://man7.org/linux/man-pages/man2/getrandom.2.html http://man7.org/linux/man-pages/man2/getrandom.2.html
avodonosov 10y ago
In Common Lisp: (ql:quickload :secure-random) (secure-random:number 100) => 73 (secure-random:bytes 20 secure-random:*generator*) => <a simple array with 20 random bytes> (This library requires OpenSSL)
- junke 10y ago
  Or without OpenSSL (ql:quickload :isaac) ;; FIRST TIME (one of both) ;; using (random), which on sane implementations should rely on /dev/urandom (defvar *isaac* (isaac:init-common-lisp-random-seed)) ;; using /dev/urandom directly (defvar *isaac* (isaac:init-kernel-seed)) ;; Generate bits (isaac:rand-bits *isaac* 30) ISAAC is supposed to be secure, but I don't know enough about cryptography to judge it.
  - avodonosov 10y ago
    The main reason to use OpenSSL in secure-random is to get a seed - not all platforms have /dev/[u]ramdon. OpenSSL has code to initialize a seed from platform-dependent APIs
    - CiPHPerCoder 10y ago
      We can't condone any solution that depends on userspace PRNGs like OpenSSL instead of the kernel's CSPRNG. There are probably worse examples of userspace PRNGs that attempt to be cryptographically secure than what OpenSSL provides, but we wrote this to encourage good habits. Reliance on OpenSSL's PRNG is definitely not a good habit to get into. The "General Requirements for Inclusion" section is our basic evaluation criteria. Cross-platform APIs that wrap the operating system's CSPRNG aren't that difficult. See https://github.com/cryptosphere/sysrandom/blob/master/ext/sysrandom/randombytes_sysrandom.c https://github.com/cryptosphere/sysrandom/blob/master/ext/sy...
      - avodonosov 10y ago
        Hm, why userspace PRNGs are bad? Does code executed in kernel mode become better automatically? If OpenSSL has bad RNG, then a lot of SSL traffic is in danger... Also, do you think Java's SecureRandom is not a user space implementation? I guess it only uses /dev/[u]random for seed.
deleted 10y ago
[deleted]
throwawayReply 10y ago
I can't help but feel this sums up the difference between these two ecosystems: Cryptographically Secure Randomness in .NET (C#) The generally accepted solution is to use System.Security.Cryptography.RNGCryptoServiceProvider Cryptographically Secure Randomness in Node.js Don't use crypto.randomBytes() An unfair cherry picking perhaps, but an illustrative example.
jonathanstrange 10y ago
It might be a good idea to additionally check the interface for failure of the PRNG, e.g. that not all random numbers returned are 0.
jrapdx3 10y ago
FWIW in Tcl it's fairly easy to generate random numbers from /dev/urandom on Unix-like systems[0]. It involves opening /dev/urandom, reading 8 bytes from the device, binary scan of the returned value, converting from the hex format and scaling the result to fall within a specified min to max range. The tcllib has other PRNG procedures, and there's the "Random" package ISAAC implementation. Whether output quality of any of these procedures is adequate probably depends on the purpose. [0] http://tcl.wiki/29163 http://tcl.wiki/29163
- junke 10y ago
  Reading this story, I was curious about ISAAC, so I posted a link to its webpage in another post (https://news.ycombinator.com/item?id=12441451 https://news.ycombinator.com/item?id=12441451), hoping people would give their opinion on it. Do you have experience with ISAAC?
  - jrapdx3 10y ago
    That's the same link provided in the Tcl wiki. I haven't actually used the ISAAC generator since the urandom is considered reliable. However, urandom is only available on Linux/BSD systems, on Windows alternatives are necessary. From the comments on the ISAAC home page, the ISAAC and ISAAC-64 versions would appear adequate for most uses, but a reader more experienced in CSPRNG analysis might have more information.
- pwg 10y ago
  > reading 8 bytes from the device, binary scan of the returned value, converting from the hex format It is not necessary to round trip through a hex encoding. Tcl's binary scan is capable of converting an 8 byte value directly into a 64-bit signed or unsigned integer. You can even request big or little endian interpretation of the bytes. Check the "w" conversion flag in the man-page. For unsigned you'd add the "u" flag. For big-endian conversion use the "W" flag instead.
- zAy0LfpBZLC8mAC 10y ago
  > and scaling the result to fall within a specified min to max range. This is actually incorrect. Unless the size of the range is a power of two, this gives non-uniform probability. The bias might be acceptable if it's small enough, but it's something to be aware of, and, if you want to have something robust, to avoid. A general safe strategy is to check if the raw random value is below the maximum multiple of the size of the range that's no greater than the number of possible values for the raw random value, and if not, generate a new one and repeat. (In principle, any smaller multiple (>= 1) would work as well, but taking the maximum gives the smallest number of retries.)
gant 10y ago
PHP: WordPress, according to Automattic: https://api.wordpress.org/secret-key/1.1/salt https://api.wordpress.org/secret-key/1.1/salt
- richardwhiuk 10y ago
  Not sure what your point is here?
  - CiPHPerCoder 10y ago
    As of WordPress 4.4.0, wp_rand() is fine. https://paragonie.com/blog/2015/10/coming-wordpress-4-4-csprng https://paragonie.com/blog/2015/10/coming-wordpress-4-4-cspr... I had already suggested (to Dion Hulse) that this antipattern of pinging wordpress.org to define the salts during setup be removed in favor of always generating them locally. I don't know if/when they'll implement that (no matter what benefits a security fix might provide, WordPress always prioritizes "no BC breaks even on 0.0001% of the installed base" first), but if you're looking for a more secure CMS I happen to be building one here: https://github.com/paragonie/airship https://github.com/paragonie/airship
Nursie 10y ago
-- edit -- In case it's not clear I really, really agree with this article, I was having a rant about people I've worked with who make things needlessly complex, not the link, which is great advice! Just use /dev/urandom I can't believe how many crappy engineers I've encountered who are resistant to this advice. The onpy coherent objection I've ever heard is that "It never blocs therefore it can provide random numbers with insufficient entropy", which has been adequately refuted many times now. But still I encounter experienced, senior guys doing all sorts of Heath-Robinson crap with reading tons of entropy from TPM chips, feeding it to openssl or other systems as a seed and using those instead. It's completely unnecessary adds complexity (which in turn decreases security) and just... What's really frustrating is people being convinced that random numbers are hard and therefore that they really need to do all this 'hard' stuff. I guess it flatters their egos.
- x1798DE 10y ago
  The article specifically mentions that it is about how to use urandom, i.e. what are the language-specific wrappers in each language.
  - Nursie 10y ago
    Indeed, it's a great article :) Didn't mean to come across as negative about it, though I can see how people might get that impression!
- richardwhiuk 10y ago
  It's wrong to use during system startup as it may not have been seeded yet.
  - Nursie 10y ago
    That's true, but likely only true during the first ever boot due to entropy recycling.
    - Dylan16807 10y ago
eveningcoffee 10y ago
As general rule, before you start using language provided secure random number generator, you should at least informally verify the full chain of the generator on multiple platforms.
Hello71 10y ago
on the whole, redefining 'int' in Python is not a great idea: >>> int <type 'int'> >>> int('10') 10 >>> int = 10 >>> int('10') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'int' object is not callable
- 0003 10y ago
  Really hurts the article's credibility to intersect cryptographic domain knowledge with technical programming knowledge. That said, I did learn something from it, but I'll be doing a bit more research...
steaminghacker 10y ago
tangential question; does anyone know of a cheap hardware random number generator for PCs. thanks.
- bomdo 10y ago
  https://www.kickstarter.com/projects/moonbaseotago/onerng-an-open-source-entropy-generator https://www.kickstarter.com/projects/moonbaseotago/onerng-an... It's a ~$50 USB dongle, opensource and available for shipping. Anecdotally, I personally used it for an RF project that need uniform randomness and exhausted /dev/random on the target hardware. It worked fine, if a bit slow for my very specific usecase. That said, if you want to go even cheaper, seriously consider if you can just stick with /dev/urandom. See http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/ for an interesting read on why that might just work.
- Sami_Lehtinen 10y ago
  Crappy (free, obsolete) webcam, fm-receiver, tv-receiver + sha512? - Define cheap.
  - chrisseaton 10y ago
    I'm not normally a paranoid person but EM radiation seems like it's something that it would be easy for someone outside your house in a van to manipulate.
    - MichaelRenor 10y ago
      Discussions about random seed generators usually devolve into theoretical issues about why <random idea> can be manipulated. If you're really paranoid, just do this: $ echo "<bash face against keyboard multiple times>" >> /dev/random
    - Dylan16807 10y ago
      It should be basically impossible for them to manipulate the low bits of the signal. Then you hash it, which makes any bias irrelevant as long as enough bits are random.
deleted 10y ago
[deleted]
mneemonic 10y ago
I would love to see this explained. TY. import time, sys, random def diorandom(m, n): start = time.time() for x in range(0, n): r = m.randint(0, 2**255) elapsed = time.time() - start return elapsed def diorandint(n): start = time.time() for x in range(0, n): r = random.randint(0, 2**255) elapsed = time.time() - start return elapsed if __name__ == '__main__': m = random.SystemRandom() modo1 = diorandom(m, 100000) modo2 = diorandint(100000) print('100,000 iterations') print('SystemRandom().randint: {}'.format(modo1)) print('random.randint: {}'.format(modo2)) I actually would know why you didn't simply encouraged random.randint.
- koolba 10y ago
  > I actually would know why you didn't simply encouraged random.randint. Because it's not secure. Read the big fat warning in pink: https://docs.python.org/2/library/random.html https://docs.python.org/2/library/random.html
happy-go-lucky 10y ago
In Python snippet of the blog post, urandom() was used without importing the os module :)
- CiPHPerCoder 10y ago
  Oops! How did I miss that? Thanks for letting me know. Fixed, and a bunch of suggested additions are coming later today. :)
leeter 10y ago
The recommendation for windows is hopelessly out of date. On Vista and up CryptGenRandom is exported directly from Advapi32. That said... just use the CNG functions (BCryptGenRandom[1]), which is what both RtlGenRandom and CryptGenRandom use under the covers. BCrypt.dll is going to get loaded either way. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa375458(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
FrozenVoid 10y ago
The best option imho is implementing a modern Xorshift variant in the language, not depending on runtime/OS. http://xoroshiro.di.unimi.it/xoroshiro128plus.c http://xoroshiro.di.unimi.it/xoroshiro128plus.c

245 points by CiPHPerCoder 10y ago

Can anyone elaborate on why Ruby's SecureRandom shouldn't be used? I find this to be rather surprising and discomforting.

tptacek 10y ago
It defaults to the OpenSSL userland CSPRNG, if available, because Ruby's maintainers believe (incorrectly) that urandom is unsafe for key generation.
- brandur 10y ago
  The README of the gem that the article recommends ("sysrandom") provides a good explanation [1]. And here is the contentious Ruby issue wherein the Ruby core developers insist on cargo culting according to an outdated man page despite comprehensive refutation from a number of experienced security engineers [2]. That said, the arguments against OpenSSL seem to be largely theoretical at this point ("has been a source of vulnerabilities in Ruby ..."), so the misgivings about `SecureRandom` may be slightly exaggerated. [1] https://github.com/cryptosphere/sysrandom#why https://github.com/cryptosphere/sysrandom#why [2] https://bugs.ruby-lang.org/issues/9569 https://bugs.ruby-lang.org/issues/9569
  - scrollaway 10y ago
    What a disaster of a bug report, on so many levels. Speaking as a foss maintainer, to other foss maintainers out there: When you're systematically adopting a defensive attitude towards someone who believes found an important flaw in your project, you're the one being rude. You're refusing to hear people out. I see this awful attitude far too often. On Github, it's even worse nowadays where a lot of maintainers immediately fall back to locking the bug report so that they don't have to think about it anymore. At least here there is some form of discussion going on. But that's with a lot of fingers in a lot of ears, and nobody trying to be practical...
  - raverbashing 10y ago
    > uby issue wherein the Ruby core developers insist on cargo culting according to an outdated man page despite comprehensive refutation from a number of experienced security engineers [2]. Why doesn't this surprise me...
  - Nokinside 10y ago
    Oh man. There is huge undiscovered land of material for cross-cultural and comparative research in communication in these developer forums. Ruby and Linux mailing lists could provide material for number of really interesting studies in many fields. Ruby community has hint of Japanese culture: design aesthetics, simplicity, importance of politeness, saving face and rigidity. Linux community has hint of Finnish culture: practicality, "pig energy" and "management by perkele" combined with 80's competitive hacker culture.
    - tptacek 10y ago
      This is a case where both the Ruby community and the Linux community are standing in the way of security.
      - Nokinside 10y ago
        What's wrong with random numbers in Linux?
        tptacek 10y ago
        They have a /dev/random interface which (literally) randomly blocks, disrupting programs, and a man page that instructs the Ruby community to use /dev/random and not /dev/urandom.
        Nokinside 10y ago
        /dev/random is the blocking pseudorandom number generator in Unix-like operating systems and it provides only the entropy that can be obtained from environmental noise. I don't see how it can work any other way. What is the manpage we are discussing here? I think getrandom(2) and random(4) are quite good.
        tptacek 10y ago
        /dev/random keeps a counter that is effectively random that tells the interface to block. /dev/urandom has the same mechanics as /dev/random, the same inputs, and the same cryptographic constructions, but does not have the random blocking counter. /dev/urandom is the interface you want. /dev/random is unsuitable for production code because, again, it randomly blocks, which can freeze your application. Worse still: applications that use /dev/random tend to "work around" the problem by using /dev/random to seed a userspace RNG that doesn't block. That's not just inconvenient but overtly unsafe.
  - ufo 10y ago
    I think the maintainer has a point in saying that the `man 4 random` page should have been updated if its giving wrong advice. I just checked and on my system that manpage still says all that wrong stuff about /dev/urandom. Why is it still unchanged?
    - tptacek 10y ago
      Because Linux man pages are sometimes dumb. I can give you a more precise answer but I don't think it will improve the thread. You should, however, take my word for it. Or, better yet, take Daniel Bernstein's word for it. Take Adam Langley's implied word for it (he's behind crypto/rand in Go). Take Thomas Pornin's word for. You can take Thomas Pornin's word on almost anything! Take Filippo Valsorda's word for it. Or Thomas Huhn's. The man page is wrong. Should it be fixed? Yes, it should be fixed. But, no excuses. You can't introduce security problems out of obstinacy about man pages.
      - CiPHPerCoder 10y ago
        You totally can, but more importantly, you shouldn't. :)
  - joepie91_ 10y ago
    > That said, the arguments against OpenSSL seem to be largely theoretical at this point I would disagree. This is a recent paper on the state of OpenSSL's userspace PRNG, for example, finding several issues: https://eprint.iacr.org/2016/367.pdf https://eprint.iacr.org/2016/367.pdf
- veeti 10y ago
  Is that Linux man page ever going to be corrected?
  - cpach 10y ago
    I sure hope so! But don't hold your breath...
  - pwg 10y ago
    Given that those man-pages are a community open source project much like the kernel, the best way to see it fixed is to contribute a fix: https://www.kernel.org/doc/man-pages/contributing.html https://www.kernel.org/doc/man-pages/contributing.html
    - chowells 10y ago
      In this case, that's far from true. Many people have contributed correct documentation. Whoever is in charge of these things continually rejects it because of their mistaken beliefs about PRNGs.
    - deleted 10y ago
      [deleted]
  - Sir_Cmpwn 10y ago
    There's a patchset that's been under discussion for a while.
- eveningcoffee 10y ago
  Anecdotal case here. I did a research about usage of secure random number generation few year ago and gave up with OpenSSL. They used indirection upon indirection and it was really hard to understand what would be actually called. Therefore I was really not surprised when problems related to OpenSSL random generation started to pup up later on.
wastedhours 10y ago
I always use SecureRandom to generate unique IDs, but not to actually secure things - am I to take it that it would still be wise to switch to Sysrandom in that case as the vuln is in going through OpenSSL to do it (rather than the generated random not being random etc)?

billhathaway 10y ago

Does Go's crypto/rand[0] package meet this criteria? [0] https://golang.org/src/crypto/rand/rand_linux.go https://golang.org/src/crypto/rand/rand_linux.go

tptacek 10y ago
Yes, use crypto/rand for Go.
niftich 10y ago
+1 to this question. If the rand_linux strategy linked above decides to fall back to rand_unix, it can result in a userspace AES-based generator [1] running on output from /dev/random [1] https://golang.org/src/crypto/rand/rand_unix.go https://golang.org/src/crypto/rand/rand_unix.go
- tptacek 10y ago
  Unless you're on Plan 9, this isn't a concern.

niftich 10y ago

Some additional requests: - Rust (no out-of-the-box crypto lib AFAIK) - Erlang (ships-by-default 'crypto' uses openssl) - Java 7 and below are, unfortunately, still in common use, despite being end-of-life. There is a config file to be edited, or a switch to be passed to the java executable to switch the RNG source away from /dev/random, the pre-8 SecureRandom default. - Some obvious note about how client-side Javascript is a lost cause, in case anybody's tempted

indolering 10y ago
> Some obvious note about how client-side Javascript is a lost cause, in case anybody's tempted crypto.getRandomBytes is supported by everything that matters[0], won't be a problem in ~5 years. 0: http://caniuse.com/#feat=getrandomvalues http://caniuse.com/#feat=getrandomvalues
matt_wulfeck 10y ago
If on Linux it's almost always safe to read from /dev/urandom.
- vram22 10y ago
  > it's almost always safe When is it not?
  - matt_wulfeck 10y ago
    If you have a fresh VM without a seed file, that isn't connected to the internet and has had little in the way of devices, and you immediately begin generating keys before the normal file system operations stir the generator, then the deterministic nature of the generator says you'll get a predictable key, or more likely, end up generating a key that's simply not unique. Just by being online and running for a few seconds will cause enough entropy to put the generator into a safe state, so this rarely happens in practice. In fact Ubuntu comes prepackaged with pollinate[0] which makes this basically a non-issue for everyone that's not wearing a tinfoil hat. 1. http://manpages.ubuntu.com/manpages/trusty/man1/pollinate.1.html http://manpages.ubuntu.com/manpages/trusty/man1/pollinate.1....

hackcasual 10y ago

> Other Unix-like (including OS X): /dev/urandom Do other OSes ensure there's enough initial entropy to safely use urandom?

wahern 10y ago
1) Define "enough" in terms of each and every specific piece of hardware, combined with the mixing heuristics and functions in every kernel of interest. It's one thing to say you want N bits of entropy; it's quite another to accurately quantify it. (Count the number of times you see a word like "assume", "estimate", etc, in papers discussing kernel entropy estimation, and you'll never look at entropy estimation the same way.) 2) If you managed that and were able to conclude in the negative for a specific combination of kernel, kernel revision, and hardware models and revisions, what then? Are you going to recreate all that infrastructure in your app or through a gargantuan dependency along the lines of egd or havaged (which we've already learned from experience is a poor idea)? My point being, the question is arguably irrelevant. If available entropy at boot time (usually first boot, as the shutdown/startup process recycles entropy pools) is insufficient, better to pressure vendors (software and hardware) to address it than to add another layer of cruft or a complex mitigation. The fix will probably be out in the next year or so (indeed, with RDRAND has been out for years); the hack will linger for far longer, perhaps eventually becoming the weakest link in the chain. For similar reasons I think the article's as well as libsodium's choice to only use the most recent arc4random implementation as shipped in recent OpenBSD releases is misguided. Every other *BSD provides arc4random, and arguably applications should always prefer to use the libc's arc4random implementation. Eventually they'll be upgraded, at which point your application will have a useless dependency adding at best needless complexity and at worst vulnerabilities. Once you look behind the interface of arc4random, you're obliged to look behind the interface of /dev/urandom, etc, and I doubt it will look much better. Until the recent overhaul (last month), the PRNG infrastructure in Linux relied on SHA-1, which isn't much better than RC4, especially when considering that the RC4 used in the classic arc4random implementations discarded the first 1k of output and reseeded regularly. SHA-1 hasn't been recommended for cryptographic purposes for several years, and while you could make pragmatic arguments for why it was still acceptable, you could do the same for the particular use of RC4 in older arc4random implementations. I don't see libsodium skipping /dev/urandom for kernels before 4.9. At some point you have to stop bending over backwards to support platforms with poor QoS and leave things as you found them as long as they're not utterly broken. If experience with egd and, more generally, the boots-and-suspenders approach to security is any measure, classic arc4random is well within the bounds of reasonable. Simplicity is the rule of the day; experience has taught that complexity is invariably a net loss in security, especially when the weakest link remains unchanged.

atheg33 10y ago

Further advancement in secure random number generation across languages will probably lead to more "don't roll your own crypto" reminder posts in the near future. But it's still a great feature to have in every programming language imho.

alrs 10y ago

An example in Go: https://github.com/alrs/radseed https://github.com/alrs/radseed

sjclemmy 10y ago

A few years ago I worked for a gift card manufacturer and I was tasked with creating random PAN numbers for a gift card product for a large high street retailer (sadly no longer with us). I read up on it and couldn't find any decent guide as to how to produce CSPRNG - and to be fair, there was very little information in 'layman' terms about what CS meant in this context. As I couldn't be certain that any of the software routes available to me were secure I ended up purchasing this http://www.protego.se/ http://www.protego.se/ to generate the random numbers. I am not convinced that random numbers were required for the security element of the gift cards, but I don't believe there were any reports of any fraudulent activity based on unknown numbers. Developments in this arena over the last 10 years would have made my job a lot easier!

deleted 10y ago
[deleted]
koolba 10y ago
> I read up on it and couldn't find any decent guide as to how to produce CSPRNG - and to be fair, there was very little information in 'layman' terms about what CS meant in this context. When you say "produce CSPRNG" do you mean writing a generator or simply generating numbers? Nobody[1] should be writing a CSPRNG and app developers should be using language/OS functions (ex: /dev/urandom). > As I couldn't be certain that any of the software routes available to me were secure I ended up purchasing this http://www.protego.se/ http://www.protego.se/ to generate the random numbers. Wow did you really look at that page and say, "This is going to be easier and more secure than using /dev/urandom?". Reading the rest of the info on the page, http://www.protego.se/trnsup.htm http://www.protego.se/trnsup.htm, does not give me a lot of confidence in the product. The explanation page of how it works is just a bunch of scare info against using a software solution. [1]: Okay obviously someone has to do it but you all know what I mean...
- _asummers 10y ago
  Prefer to use getrandom instead of /dev/urandom now, if your OS supports it. http://man7.org/linux/man-pages/man2/getrandom.2.html http://man7.org/linux/man-pages/man2/getrandom.2.html

avodonosov 10y ago

In Common Lisp: (ql:quickload :secure-random) (secure-random:number 100) => 73 (secure-random:bytes 20 secure-random:*generator*) => <a simple array with 20 random bytes> (This library requires OpenSSL)

junke 10y ago
Or without OpenSSL (ql:quickload :isaac) ;; FIRST TIME (one of both) ;; using (random), which on sane implementations should rely on /dev/urandom (defvar *isaac* (isaac:init-common-lisp-random-seed)) ;; using /dev/urandom directly (defvar *isaac* (isaac:init-kernel-seed)) ;; Generate bits (isaac:rand-bits *isaac* 30) ISAAC is supposed to be secure, but I don't know enough about cryptography to judge it.
- avodonosov 10y ago
  The main reason to use OpenSSL in secure-random is to get a seed - not all platforms have /dev/[u]ramdon. OpenSSL has code to initialize a seed from platform-dependent APIs
  - CiPHPerCoder 10y ago
    We can't condone any solution that depends on userspace PRNGs like OpenSSL instead of the kernel's CSPRNG. There are probably worse examples of userspace PRNGs that attempt to be cryptographically secure than what OpenSSL provides, but we wrote this to encourage good habits. Reliance on OpenSSL's PRNG is definitely not a good habit to get into. The "General Requirements for Inclusion" section is our basic evaluation criteria. Cross-platform APIs that wrap the operating system's CSPRNG aren't that difficult. See https://github.com/cryptosphere/sysrandom/blob/master/ext/sysrandom/randombytes_sysrandom.c https://github.com/cryptosphere/sysrandom/blob/master/ext/sy...
    - avodonosov 10y ago
      Hm, why userspace PRNGs are bad? Does code executed in kernel mode become better automatically? If OpenSSL has bad RNG, then a lot of SSL traffic is in danger... Also, do you think Java's SecureRandom is not a user space implementation? I guess it only uses /dev/[u]random for seed.

deleted 10y ago

[deleted]

throwawayReply 10y ago

I can't help but feel this sums up the difference between these two ecosystems: Cryptographically Secure Randomness in .NET (C#) The generally accepted solution is to use System.Security.Cryptography.RNGCryptoServiceProvider Cryptographically Secure Randomness in Node.js Don't use crypto.randomBytes() An unfair cherry picking perhaps, but an illustrative example.

jonathanstrange 10y ago

It might be a good idea to additionally check the interface for failure of the PRNG, e.g. that not all random numbers returned are 0.

jrapdx3 10y ago

FWIW in Tcl it's fairly easy to generate random numbers from /dev/urandom on Unix-like systems[0]. It involves opening /dev/urandom, reading 8 bytes from the device, binary scan of the returned value, converting from the hex format and scaling the result to fall within a specified min to max range. The tcllib has other PRNG procedures, and there's the "Random" package ISAAC implementation. Whether output quality of any of these procedures is adequate probably depends on the purpose. [0] http://tcl.wiki/29163 http://tcl.wiki/29163

junke 10y ago
Reading this story, I was curious about ISAAC, so I posted a link to its webpage in another post (https://news.ycombinator.com/item?id=12441451 https://news.ycombinator.com/item?id=12441451), hoping people would give their opinion on it. Do you have experience with ISAAC?
- jrapdx3 10y ago
  That's the same link provided in the Tcl wiki. I haven't actually used the ISAAC generator since the urandom is considered reliable. However, urandom is only available on Linux/BSD systems, on Windows alternatives are necessary. From the comments on the ISAAC home page, the ISAAC and ISAAC-64 versions would appear adequate for most uses, but a reader more experienced in CSPRNG analysis might have more information.
pwg 10y ago
> reading 8 bytes from the device, binary scan of the returned value, converting from the hex format It is not necessary to round trip through a hex encoding. Tcl's binary scan is capable of converting an 8 byte value directly into a 64-bit signed or unsigned integer. You can even request big or little endian interpretation of the bytes. Check the "w" conversion flag in the man-page. For unsigned you'd add the "u" flag. For big-endian conversion use the "W" flag instead.
zAy0LfpBZLC8mAC 10y ago
> and scaling the result to fall within a specified min to max range. This is actually incorrect. Unless the size of the range is a power of two, this gives non-uniform probability. The bias might be acceptable if it's small enough, but it's something to be aware of, and, if you want to have something robust, to avoid. A general safe strategy is to check if the raw random value is below the maximum multiple of the size of the range that's no greater than the number of possible values for the raw random value, and if not, generate a new one and repeat. (In principle, any smaller multiple (>= 1) would work as well, but taking the maximum gives the smallest number of retries.)

gant 10y ago

PHP: WordPress, according to Automattic: https://api.wordpress.org/secret-key/1.1/salt https://api.wordpress.org/secret-key/1.1/salt

richardwhiuk 10y ago
Not sure what your point is here?
- CiPHPerCoder 10y ago
  As of WordPress 4.4.0, wp_rand() is fine. https://paragonie.com/blog/2015/10/coming-wordpress-4-4-csprng https://paragonie.com/blog/2015/10/coming-wordpress-4-4-cspr... I had already suggested (to Dion Hulse) that this antipattern of pinging wordpress.org to define the salts during setup be removed in favor of always generating them locally. I don't know if/when they'll implement that (no matter what benefits a security fix might provide, WordPress always prioritizes "no BC breaks even on 0.0001% of the installed base" first), but if you're looking for a more secure CMS I happen to be building one here: https://github.com/paragonie/airship https://github.com/paragonie/airship

Nursie 10y ago

-- edit -- In case it's not clear I really, really agree with this article, I was having a rant about people I've worked with who make things needlessly complex, not the link, which is great advice! Just use /dev/urandom I can't believe how many crappy engineers I've encountered who are resistant to this advice. The onpy coherent objection I've ever heard is that "It never blocs therefore it can provide random numbers with insufficient entropy", which has been adequately refuted many times now. But still I encounter experienced, senior guys doing all sorts of Heath-Robinson crap with reading tons of entropy from TPM chips, feeding it to openssl or other systems as a seed and using those instead. It's completely unnecessary adds complexity (which in turn decreases security) and just... What's really frustrating is people being convinced that random numbers are hard and therefore that they really need to do all this 'hard' stuff. I guess it flatters their egos.

x1798DE 10y ago
The article specifically mentions that it is about how to use urandom, i.e. what are the language-specific wrappers in each language.
- Nursie 10y ago
  Indeed, it's a great article :) Didn't mean to come across as negative about it, though I can see how people might get that impression!
richardwhiuk 10y ago
It's wrong to use during system startup as it may not have been seeded yet.
- Nursie 10y ago
  That's true, but likely only true during the first ever boot due to entropy recycling.
  - Dylan16807 10y ago

eveningcoffee 10y ago

As general rule, before you start using language provided secure random number generator, you should at least informally verify the full chain of the generator on multiple platforms.

Hello71 10y ago

on the whole, redefining 'int' in Python is not a great idea: >>> int <type 'int'> >>> int('10') 10 >>> int = 10 >>> int('10') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'int' object is not callable

0003 10y ago
Really hurts the article's credibility to intersect cryptographic domain knowledge with technical programming knowledge. That said, I did learn something from it, but I'll be doing a bit more research...

steaminghacker 10y ago

tangential question; does anyone know of a cheap hardware random number generator for PCs. thanks.

bomdo 10y ago
https://www.kickstarter.com/projects/moonbaseotago/onerng-an-open-source-entropy-generator https://www.kickstarter.com/projects/moonbaseotago/onerng-an... It's a ~$50 USB dongle, opensource and available for shipping. Anecdotally, I personally used it for an RF project that need uniform randomness and exhausted /dev/random on the target hardware. It worked fine, if a bit slow for my very specific usecase. That said, if you want to go even cheaper, seriously consider if you can just stick with /dev/urandom. See http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/ for an interesting read on why that might just work.
Sami_Lehtinen 10y ago
Crappy (free, obsolete) webcam, fm-receiver, tv-receiver + sha512? - Define cheap.
- chrisseaton 10y ago
  I'm not normally a paranoid person but EM radiation seems like it's something that it would be easy for someone outside your house in a van to manipulate.
  - MichaelRenor 10y ago
    Discussions about random seed generators usually devolve into theoretical issues about why <random idea> can be manipulated. If you're really paranoid, just do this: $ echo "<bash face against keyboard multiple times>" >> /dev/random
  - Dylan16807 10y ago
    It should be basically impossible for them to manipulate the low bits of the signal. Then you hash it, which makes any bias irrelevant as long as enough bits are random.

deleted 10y ago

[deleted]

mneemonic 10y ago

I would love to see this explained. TY. import time, sys, random def diorandom(m, n): start = time.time() for x in range(0, n): r = m.randint(0, 2**255) elapsed = time.time() - start return elapsed def diorandint(n): start = time.time() for x in range(0, n): r = random.randint(0, 2**255) elapsed = time.time() - start return elapsed if __name__ == '__main__': m = random.SystemRandom() modo1 = diorandom(m, 100000) modo2 = diorandint(100000) print('100,000 iterations') print('SystemRandom().randint: {}'.format(modo1)) print('random.randint: {}'.format(modo2)) I actually would know why you didn't simply encouraged random.randint.

koolba 10y ago
> I actually would know why you didn't simply encouraged random.randint. Because it's not secure. Read the big fat warning in pink: https://docs.python.org/2/library/random.html https://docs.python.org/2/library/random.html

happy-go-lucky 10y ago

In Python snippet of the blog post, urandom() was used without importing the os module :)

CiPHPerCoder 10y ago
Oops! How did I miss that? Thanks for letting me know. Fixed, and a bunch of suggested additions are coming later today. :)

leeter 10y ago

The recommendation for windows is hopelessly out of date. On Vista and up CryptGenRandom is exported directly from Advapi32. That said... just use the CNG functions (BCryptGenRandom[1]), which is what both RtlGenRandom and CryptGenRandom use under the covers. BCrypt.dll is going to get loaded either way. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa375458(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

FrozenVoid 10y ago

The best option imho is implementing a modern Xorshift variant in the language, not depending on runtime/OS. http://xoroshiro.di.unimi.it/xoroshiro128plus.c http://xoroshiro.di.unimi.it/xoroshiro128plus.c