6 ms·

Web server offers true random numbers via fluctuations of the quantum vacuum

73 points by rickyconnolly 15y ago

noodly 15y ago
Here's the original source: http://150.203.48.55/index.php http://150.203.48.55/index.php
- mapleoin 15y ago
  And here's a pretty stream of random colors: http://150.203.48.55/RainCol.php http://150.203.48.55/RainCol.php
  - moontear 15y ago
    I wanted to post this too. Makes me feel special knowing that these are truly random.
- acqq 15y ago
  What's wrong with the one in jwz's post? http://photonics.anu.edu.au/qoptics/Research/qrng.php http://photonics.anu.edu.au/qoptics/Research/qrng.php
mike-cardwell 15y ago
If you have a modernish machine running Linux, find out if it has a TPM (Trusted Platform Module) built in. If it does, you can use an application called "Trousers" to take advantage of its built in hardware random number generator. It will directly feed random data into /dev/random If you don't have a TPM, you can get one of these usb sticks: http://www.entropykey.co.uk/ http://www.entropykey.co.uk/ and it will also feed /dev/random with large amounts of real random data.
- skrebbel 15y ago
  How can a TPM provide as "real" random values as a quantum vacuum? (or cosmic rays, or anything else very much random in nature)
  - JoachimSchipper 15y ago
    According to http://www.outflux.net/blog/archives/2009/10/22/tpm-as-rng/#comment-793 http://www.outflux.net/blog/archives/2009/10/22/tpm-as-rng/#..., TPM may even be using PRNGs. So you're right that the quality of output need not be better than what a modern OS already provides. (I am not aware of any case where a properly-used /dev/random failed to produce sufficient entropy. Note that doing something sensible at boot is part of "properly-used" here.) It is possible to provide true randomness from a chip, but it's probably cheaper not to.
    - __alexs 15y ago
      The Intel TRNG design used for RdRand in Ivy Bridge has an impressively low transistor count so I don't think die-cost alone is the issue. http://spectrum.ieee.org/semiconductors/processors/behind-intels-new-randomnumber-generator/0 http://spectrum.ieee.org/semiconductors/processors/behind-in... http://en.wikipedia.orghwiki/RdRand http://en.wikipedia.orghwiki/RdRand I suspect they just didn't consider RNG quality as a big factor in the usefulness of the design. TPM pre-dates the recent advances in on-die TRNG design. Plus the chip tends to live on a low-speed bus on the system which makes it inherently a bit crappy as a place to keep a high-bandwidth application like an RNG.
  - mike-cardwell 15y ago
    When did I say that it does? The randomness of the data is entirely down to the implementation. Personally I'd trust the random data supplied by a third party web service less than the pseudo-random data that the software on my local machine produces. Especially if you mix in data from a TPM, especially if you mix in data from an Entropy Key as well.
    - hessenwolf 15y ago
      Yes, this is a real issue. There's no way I'd get that past a financial regulator.
- teamnyble 15y ago
  Can anyone explain how this works? The entropy key site says "The Entropy Key contains two high-quality noise generators, and an ARM Cortex CPU that actively measures, checks and confirms all generated random numbers, before encrypting them and sending them to the server"... but why can't we just write high-quality noise generators in software? How is the hardware better?
  - ajuc 15y ago
    Because you can't get really random number from deterministic hardware (and computers are deterministic, more or less :) ). No matter what you do, running the same algorithm on the same input gives you the same data on regular computer. And the randomness (measured in Kolmogorov complexity [1] of the resulting string) is by definition smaller than the length of the generator program. And truly random string is such string, that there's no possible program producing this string as output, that is shorter than the string. On deterministic computers you can only produce pseudorandom numbers, with varying quality of randomness, depending on generator program you use, but it's sometimes not enough (esp. in cryptography). [1] http://en.wikipedia.org/wiki/Kolmogorov_complexity http://en.wikipedia.org/wiki/Kolmogorov_complexity
  - silvestrov 15y ago
    Pure software-based random generators are deterministic and thus not random. When something is deterministic, it is reproducible and therefore not good for creating crypto keys. Most /dev/random implementations use data from the ethernet driver, the keyboard, mouse etc to get some input which isn't easy to reproduce.
    - deleted 15y ago
      [deleted]
  - mzl 15y ago
    Software is deterministic, so it is not really possible to generate noise. There are non-deterministic parts of software (for example, thread interaction in concurrent systems), but this is just observable behavior of hardware effects. The typical thing people do to get large amounts of randomness is to first generate some "real" random noise from some source (hardware thingamajigs, network timings, user interactions) and extrapolate these into longer sequences of random-looking numbers. The extrapolation can be fast and simple (in which case the randomness is not so good), or it can be slow and complex (to get decent looking randomness).
- Florin_Andrei 15y ago
  > If you have a modernish machine running Linux, find out if it has a TPM (Trusted Platform Module) built in. How do you determine that on Linux?
  - fleitz 15y ago
    https://wiki.archlinux.org/index.php/TPM https://wiki.archlinux.org/index.php/TPM
Core-TX 15y ago
oh noes. Not this poo again. Save money, and do the following. - Get a high res webcam - Open it up and remove the filters. - Hook up the device. - From time to time you'll see random white dots appear. There you have your ultra cheap random number generator. Works best in high radiation environments.
- mike-cardwell 15y ago
  Do you know of any guides to do this? Are you aware of any free software to convert the image data into a usable stream of random data?
  - rickyconnolly 15y ago
    "oh noes. Not this poo again. Save money, and do the following. - Get a high res webcam - Open it up and remove the filters. - Hook up the device. - From time to time you'll see random white dots appear. There you have your ultra cheap random number generator." It would seem that this method would be muchmore expensive and time consuming than a simple web query. How precisely would this save me money?
    - Core-TX 15y ago
      Pretty simple. You want random numbers for security. If you want security, you should not trust a third party. Only yourself. If you have to generate random numbers yourself; I just provided you a cheap solution.
    - DanBC 15y ago
      > How precisely would this save me money? It's under your control. There's no chance of an attacker serving you skewed numbers, or of MITM, or etc etc. That may be important. (Cryptography, for example.) Some people don't care, they just need some random numbers. People doing modelling like a lot of random numbers that are random enough, but they don't care if that same set of numbers is available elsewhere, or if it's easily repeatable.
  - Arjuna 15y ago
andrewcooke 15y ago
so is the paper that describes this available for free anywhere? the trail of links from that page ends at http://apl.aip.org/resource/1/applab/v98/i23/p231103_s1?isAuthorized=no http://apl.aip.org/resource/1/applab/v98/i23/p231103_s1?isAu... update - similar article (free) from same people http://www.opticsinfobase.org/view_article.cfm?gotourl=http%3A%2F%2Fwww%2Eopticsinfobase%2Eorg%2FDirectPDFAccess%2F7E9122D0-A0B4-6DFC-9DAF7CF4BF5D2F30_222934%2Epdf%3Fda%3D1%26id%3D222934%26seq%3D0%26mobile%3Dno&org= http://www.opticsinfobase.org/view_article.cfm?gotourl=http%... (source http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-19-21-20665 http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-19-21-2... ) [it seems like this is actually a big deal - they are getting huge throughput]
tete 15y ago
Hasn't Via done something similar? Since we have millions of transistors in our systems couldn't they (their collectors) be used to generate "truly" random numbers?
mtrimpe 15y ago
I've always wondered how many probabilistic algorithms would have been discovered which outperform deterministic ones if a high-bandwidth hardware RNG was available on every machine.
- patrickyeon 15y ago
  I understood that probabilistic algos only need something that "looks random enough" and are fine using PRNGs. True randomness doesn't buy you any improvement, as the algorithm is only counting on some of the properties of a random source.
  - mbq 15y ago
    The game is about speed -- probabilistic algorithm use PRNGs so heavily that their internal CPU demand may easily become the limiting factor.
eli_gottlieb 15y ago
I dunno, it could be entangled...
joblessjunkie 15y ago
FermiLab has been offering HotBits for a very long time: http://www.fourmilab.ch/hotbits/ http://www.fourmilab.ch/hotbits/ The site contains some interesting discussion around randomness and how their service works.
- ScottBurson 15y ago
  Yes, but, uh ... Fourmilab is not Fermilab :-)
  - joblessjunkie 15y ago
    Ouch, that's a bit embarrassing...
acqq 15y ago
People, if you need random numbers for some application of yours, you shouldn't get them from their server, period. See alternatives in the posts here. The most modern example, apparently the following hardware instruction will be available in all next Intel processors: http://en.wikipedia.org/wiki/RdRand http://en.wikipedia.org/wiki/RdRand The main article is just to appreciate the pure awesomeness of us being able to actually see something that's a result of quantum fluctuations! In vacuum!
Porter_423 15y ago
Their product may be really great, but the company seems unable to support it. I ordered my key in October, never received it, and have been mailing them periodically ever since, perhaps 20 times, trying to resolve it with them. To date I have received one response. Their response ignored everything I had said, and assured me it would arrive soon, even though I'd explained in my email that I'd changed addresses. I hope one day this will be resolved, because it really seems like a cool device.
EGreg 15y ago
Perhaps they are only truly random until they are observed. After which they become deterministic. :)
- jerf 15y ago
  That might sort of be funny if it weren't simply true, and not in any sort of funny-true way, either. Random numbers generated in the past are deterministic. Of course they are. What would non-deterministic-yet-already-generated numbers look like?
marshray 15y ago
I see a few people mentioning applications of this in cryptography and I feel that this is harmful. You do not need exotic hardware to collect entropy to generate crypto keys. You need a correctly designed cryptographically secure PRNG which has had time to accumulate 100 or so bits of entropy (i.e., unknowability to the attacker). It's nice if it can be freshened every now and then with more entropy, but that really only should make a difference if you lose a backup tape and by some miracle the private key wasn't on there already. After it's warmed up, it should never need to block due to 'depleted entropy'. No one has ever broken such a CSPRNG. This is how the OpenSSL, OpenBSD, and FreeBSD /dev/randoms work. On the other hand, what does tend to get broken regularly are overcomplicated and overengineered RNGs like this beast. The one on the new Intel chips is handy, but it's largely overkill. We only need 100 bits now and then, not a steady rate at Gb/s (which can shut off abruptly when an attacker sharing your cloud hardware node decides to starve you of them). It's also not something that we can review for backdoors, so we should be reluctant to use it as the sole source of entropy. A quantum vacuum random server is great like the way an internet-connected coffee pot is great. Fun, but not useful in production.
- tptacek 15y ago
  One wouldn't want to give the impression that CSPRNG design is a solved problem, but I think you're right: exotic "true random" entropy sources definitely don't target the real challenges.
  - marshray 15y ago
    I believe it is largely a solved problem. Or at least the unsolved parts are reduced to a single problem: entropy input estimation.
  - JoachimSchipper 15y ago
    Are you aware of any instances of the OS /dev/Xrandom failing in an exploitable way, assuming proper initialization? I'm sure that people still mess up, but there are a lot of problems that are solved in that sense.

73 points by rickyconnolly 15y ago

noodly 15y ago

Here's the original source: http://150.203.48.55/index.php http://150.203.48.55/index.php

mapleoin 15y ago
And here's a pretty stream of random colors: http://150.203.48.55/RainCol.php http://150.203.48.55/RainCol.php
- moontear 15y ago
  I wanted to post this too. Makes me feel special knowing that these are truly random.
acqq 15y ago
What's wrong with the one in jwz's post? http://photonics.anu.edu.au/qoptics/Research/qrng.php http://photonics.anu.edu.au/qoptics/Research/qrng.php

mike-cardwell 15y ago

If you have a modernish machine running Linux, find out if it has a TPM (Trusted Platform Module) built in. If it does, you can use an application called "Trousers" to take advantage of its built in hardware random number generator. It will directly feed random data into /dev/random If you don't have a TPM, you can get one of these usb sticks: http://www.entropykey.co.uk/ http://www.entropykey.co.uk/ and it will also feed /dev/random with large amounts of real random data.

skrebbel 15y ago
How can a TPM provide as "real" random values as a quantum vacuum? (or cosmic rays, or anything else very much random in nature)
- JoachimSchipper 15y ago
  According to http://www.outflux.net/blog/archives/2009/10/22/tpm-as-rng/#comment-793 http://www.outflux.net/blog/archives/2009/10/22/tpm-as-rng/#..., TPM may even be using PRNGs. So you're right that the quality of output need not be better than what a modern OS already provides. (I am not aware of any case where a properly-used /dev/random failed to produce sufficient entropy. Note that doing something sensible at boot is part of "properly-used" here.) It is possible to provide true randomness from a chip, but it's probably cheaper not to.
  - __alexs 15y ago
    The Intel TRNG design used for RdRand in Ivy Bridge has an impressively low transistor count so I don't think die-cost alone is the issue. http://spectrum.ieee.org/semiconductors/processors/behind-intels-new-randomnumber-generator/0 http://spectrum.ieee.org/semiconductors/processors/behind-in... http://en.wikipedia.orghwiki/RdRand http://en.wikipedia.orghwiki/RdRand I suspect they just didn't consider RNG quality as a big factor in the usefulness of the design. TPM pre-dates the recent advances in on-die TRNG design. Plus the chip tends to live on a low-speed bus on the system which makes it inherently a bit crappy as a place to keep a high-bandwidth application like an RNG.
- mike-cardwell 15y ago
  When did I say that it does? The randomness of the data is entirely down to the implementation. Personally I'd trust the random data supplied by a third party web service less than the pseudo-random data that the software on my local machine produces. Especially if you mix in data from a TPM, especially if you mix in data from an Entropy Key as well.
  - hessenwolf 15y ago
    Yes, this is a real issue. There's no way I'd get that past a financial regulator.
teamnyble 15y ago
Can anyone explain how this works? The entropy key site says "The Entropy Key contains two high-quality noise generators, and an ARM Cortex CPU that actively measures, checks and confirms all generated random numbers, before encrypting them and sending them to the server"... but why can't we just write high-quality noise generators in software? How is the hardware better?
- ajuc 15y ago
  Because you can't get really random number from deterministic hardware (and computers are deterministic, more or less :) ). No matter what you do, running the same algorithm on the same input gives you the same data on regular computer. And the randomness (measured in Kolmogorov complexity [1] of the resulting string) is by definition smaller than the length of the generator program. And truly random string is such string, that there's no possible program producing this string as output, that is shorter than the string. On deterministic computers you can only produce pseudorandom numbers, with varying quality of randomness, depending on generator program you use, but it's sometimes not enough (esp. in cryptography). [1] http://en.wikipedia.org/wiki/Kolmogorov_complexity http://en.wikipedia.org/wiki/Kolmogorov_complexity
- silvestrov 15y ago
  Pure software-based random generators are deterministic and thus not random. When something is deterministic, it is reproducible and therefore not good for creating crypto keys. Most /dev/random implementations use data from the ethernet driver, the keyboard, mouse etc to get some input which isn't easy to reproduce.
  - deleted 15y ago
    [deleted]
- mzl 15y ago
  Software is deterministic, so it is not really possible to generate noise. There are non-deterministic parts of software (for example, thread interaction in concurrent systems), but this is just observable behavior of hardware effects. The typical thing people do to get large amounts of randomness is to first generate some "real" random noise from some source (hardware thingamajigs, network timings, user interactions) and extrapolate these into longer sequences of random-looking numbers. The extrapolation can be fast and simple (in which case the randomness is not so good), or it can be slow and complex (to get decent looking randomness).
Florin_Andrei 15y ago
> If you have a modernish machine running Linux, find out if it has a TPM (Trusted Platform Module) built in. How do you determine that on Linux?
- fleitz 15y ago
  https://wiki.archlinux.org/index.php/TPM https://wiki.archlinux.org/index.php/TPM

Core-TX 15y ago

oh noes. Not this poo again. Save money, and do the following. - Get a high res webcam - Open it up and remove the filters. - Hook up the device. - From time to time you'll see random white dots appear. There you have your ultra cheap random number generator. Works best in high radiation environments.

mike-cardwell 15y ago
Do you know of any guides to do this? Are you aware of any free software to convert the image data into a usable stream of random data?
- rickyconnolly 15y ago
  "oh noes. Not this poo again. Save money, and do the following. - Get a high res webcam - Open it up and remove the filters. - Hook up the device. - From time to time you'll see random white dots appear. There you have your ultra cheap random number generator." It would seem that this method would be muchmore expensive and time consuming than a simple web query. How precisely would this save me money?
  - Core-TX 15y ago
    Pretty simple. You want random numbers for security. If you want security, you should not trust a third party. Only yourself. If you have to generate random numbers yourself; I just provided you a cheap solution.
  - DanBC 15y ago
    > How precisely would this save me money? It's under your control. There's no chance of an attacker serving you skewed numbers, or of MITM, or etc etc. That may be important. (Cryptography, for example.) Some people don't care, they just need some random numbers. People doing modelling like a lot of random numbers that are random enough, but they don't care if that same set of numbers is available elsewhere, or if it's easily repeatable.
- Arjuna 15y ago

andrewcooke 15y ago

so is the paper that describes this available for free anywhere? the trail of links from that page ends at http://apl.aip.org/resource/1/applab/v98/i23/p231103_s1?isAuthorized=no http://apl.aip.org/resource/1/applab/v98/i23/p231103_s1?isAu... update - similar article (free) from same people http://www.opticsinfobase.org/view_article.cfm?gotourl=http%3A%2F%2Fwww%2Eopticsinfobase%2Eorg%2FDirectPDFAccess%2F7E9122D0-A0B4-6DFC-9DAF7CF4BF5D2F30_222934%2Epdf%3Fda%3D1%26id%3D222934%26seq%3D0%26mobile%3Dno&org= http://www.opticsinfobase.org/view_article.cfm?gotourl=http%... (source http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-19-21-20665 http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-19-21-2... ) [it seems like this is actually a big deal - they are getting huge throughput]

tete 15y ago

Hasn't Via done something similar? Since we have millions of transistors in our systems couldn't they (their collectors) be used to generate "truly" random numbers?

mtrimpe 15y ago

I've always wondered how many probabilistic algorithms would have been discovered which outperform deterministic ones if a high-bandwidth hardware RNG was available on every machine.

patrickyeon 15y ago
I understood that probabilistic algos only need something that "looks random enough" and are fine using PRNGs. True randomness doesn't buy you any improvement, as the algorithm is only counting on some of the properties of a random source.
- mbq 15y ago
  The game is about speed -- probabilistic algorithm use PRNGs so heavily that their internal CPU demand may easily become the limiting factor.

eli_gottlieb 15y ago

I dunno, it could be entangled...

joblessjunkie 15y ago

FermiLab has been offering HotBits for a very long time: http://www.fourmilab.ch/hotbits/ http://www.fourmilab.ch/hotbits/ The site contains some interesting discussion around randomness and how their service works.

ScottBurson 15y ago
Yes, but, uh ... Fourmilab is not Fermilab :-)
- joblessjunkie 15y ago
  Ouch, that's a bit embarrassing...

acqq 15y ago

People, if you need random numbers for some application of yours, you shouldn't get them from their server, period. See alternatives in the posts here. The most modern example, apparently the following hardware instruction will be available in all next Intel processors: http://en.wikipedia.org/wiki/RdRand http://en.wikipedia.org/wiki/RdRand The main article is just to appreciate the pure awesomeness of us being able to actually see something that's a result of quantum fluctuations! In vacuum!

Porter_423 15y ago

Their product may be really great, but the company seems unable to support it. I ordered my key in October, never received it, and have been mailing them periodically ever since, perhaps 20 times, trying to resolve it with them. To date I have received one response. Their response ignored everything I had said, and assured me it would arrive soon, even though I'd explained in my email that I'd changed addresses. I hope one day this will be resolved, because it really seems like a cool device.

EGreg 15y ago

Perhaps they are only truly random until they are observed. After which they become deterministic. :)

jerf 15y ago
That might sort of be funny if it weren't simply true, and not in any sort of funny-true way, either. Random numbers generated in the past are deterministic. Of course they are. What would non-deterministic-yet-already-generated numbers look like?

marshray 15y ago

I see a few people mentioning applications of this in cryptography and I feel that this is harmful. You do not need exotic hardware to collect entropy to generate crypto keys. You need a correctly designed cryptographically secure PRNG which has had time to accumulate 100 or so bits of entropy (i.e., unknowability to the attacker). It's nice if it can be freshened every now and then with more entropy, but that really only should make a difference if you lose a backup tape and by some miracle the private key wasn't on there already. After it's warmed up, it should never need to block due to 'depleted entropy'. No one has ever broken such a CSPRNG. This is how the OpenSSL, OpenBSD, and FreeBSD /dev/randoms work. On the other hand, what does tend to get broken regularly are overcomplicated and overengineered RNGs like this beast. The one on the new Intel chips is handy, but it's largely overkill. We only need 100 bits now and then, not a steady rate at Gb/s (which can shut off abruptly when an attacker sharing your cloud hardware node decides to starve you of them). It's also not something that we can review for backdoors, so we should be reluctant to use it as the sole source of entropy. A quantum vacuum random server is great like the way an internet-connected coffee pot is great. Fun, but not useful in production.

tptacek 15y ago
One wouldn't want to give the impression that CSPRNG design is a solved problem, but I think you're right: exotic "true random" entropy sources definitely don't target the real challenges.
- marshray 15y ago
  I believe it is largely a solved problem. Or at least the unsolved parts are reduced to a single problem: entropy input estimation.
- JoachimSchipper 15y ago
  Are you aware of any instances of the OS /dev/Xrandom failing in an exploitable way, assuming proper initialization? I'm sure that people still mess up, but there are a lot of problems that are solved in that sense.