5 ms·

Usual caveat: Hardware RNGs don't really solve any practical problem, and are mostly based on urban legends about cryptography, e.g. the idea that you have to c

by hannob 4y ago

Usual caveat: Hardware RNGs don't really solve any practical problem, and are mostly based on urban legends about cryptography, e.g. the idea that you have to constantly refeed "lost" entropy in an RNG. The issues with RNGs usually come down to lack of use of a secure RNG, bugs within the RNG or early boot time entropy. Only the third may benefit in theory from a hardware RNG, but the devices that potentially have this problem usually don't come with an USB port, so it doesn't help either.

dale_glass 4y ago
I remember Linux running into a bunch of issues with boot time entropy. Some things were sometimes getting stuck on boot due to that. Also this "lost entropy" thing has long been a bane of VM servers, which could consume vast amounts of it. I think this changed fairly recently. And lots of things have USB ports these days, including things like commercial wifi access points, routers, switches, etc. Some of those have internal ones.
dragontamer 4y ago
All Intel and AMD chips come with a hardware RNG today: the RDRAND and RDSEED instructions. RDRAND is kinda-sorta like /dev/urandom, while RDSEED is /dev/random with stronger entropy guarantees. Linux didn't want to "trust" this hardware, for whatever reason. I guess there was that one time an AMD bug in the instruction returned all-zeros (or something similar). But as "long as the instruction is properly working", it is... well... working, lol. But its not like its really possible to test if its working correctly. -------- From that perspective, a hardware RNG needs to be "large" and "discrete" enough for humans to easily test and prove that its working, to create trust. Otherwise, hardware and/or firmware bugs could lead to a security problem. EDIT: if you are at a mass produced but don't verify kinda person, then you might as well use RDSEED instructions to get her entropy. Intel/AMD probably are good enough to prevent firmware / hardware bugs. I know that VMs also have an issue here: whether or not to make RDSEED deterministic/predictable or not. Should the future RDSEED instructions be saved in a VM snapshot?
- jnwatson 4y ago
  You can (and are required to in certain environments) test for basic failures in your hardware RNG. The AMD bug was bad not because of the bug, but because it was obvious they had no built-in-test functionality.
- garaetjjte 4y ago
  >RDRAND is kinda-sorta like /dev/urandom, while RDSEED is /dev/random with stronger entropy guarantees. /dev/urandom and /dev/random is almost the same nowadays, the only difference is that random will block after boot until enough entropy is collected, but it never "runs out" of entropy anymore.
- adrian_b 4y ago
  Any hardware RNG that is included in a CPU or any other complex integrated circuit cannot be trusted, because it is impossible to verify that it is truly a RNG and not a pseudo-RNG that implements backdoors. The only hardware RNGs that can be trusted are those that use an analog-digital converter (e.g. one of the ADCs included in any microcontroller, or the audio microphone input of a computer) and a discrete analog noise source, e.g. a diode or a resistor, because their components can be tested separately, to verify that they do what they claim to do. Nevertheless, the output of the hardware RNGs provided by the Intel or AMD CPUs can be used in a reasonably secure way if the supposedly random values are never used directly, but they are passed through a cryptographic hash, e.g. SHA-256. Even SHA-1 is completely adequate for this special purpose, where collision-resistance does not matter (and it is faster than any other secure hashes in the ARM, AMD or Intel CPUs with SHA-1 instructions). Even better, the supposedly random value could be hashed together with the timestamp read from the TSC, or just with the value of a counter incremented after generating a random number, for additional unpredictability. One can add to the hash a unique computer identifier, e.g. the MAC address of the first Ethernet interface. In this case, distinct sequences of uniformly-distributed pseudo-random numbers will continue to be produced even in computers where the hardware RNG remains stuck to a constant value, as it happened in some buggy AMD Zen models. The hashing (with a secure hash algorithm) of the RNG output destroys any hidden structure that might have been exploitable by the designers of the hardware RNG, while preserving the uniform distribution of the random numbers.
jnwatson 4y ago
The device itself isn't useful, but the design could be incorporated into systems that need a better source of early boot time entropy. Lots of embedded systems don't such a source, though it is getting better as more SoC vendors incorporate HW RNG.
Zababa 4y ago
For the full story, I was trying to reduce the boot time on my Linux system. One of the things that takes the most time is systemd-random-seed. From what I understand, it takes this long because my computer is not generating entropy fast enough, so it has to wait for a while. The solution to fix this is to find a way to generate more entropy. The system doc on random seeds has lots of information about what you can use: https://systemd.io/RANDOM_SEEDS/ https://systemd.io/RANDOM_SEEDS/. One of the options is RDRAND, which my CPU can do. But after reading a bit about it, it seems that some people are supicious of RDRAND. I have no idea if I should too, and since it was disabled by default on my distribution, I decided to keep it of for now, as I don't really understand this subject. At this point I wondered if any hardware RNG existed, and found the OneRNG, that seems like a cool project, so I posted it here. This is not a recommandation to use OneRNG instead of RDRAND, I don't know enough about this domain to suggest one or another.
riedel 4y ago
Most applications actually IMHO come from statistics and simulation. It seems PRNGs are sometimes problematic if you need true white noise...
Genbox 4y ago
You are correct. However, history teaches us important lessons. Back in the old days when cryptography was a weird concept nobody understood, we had the Linear Congruential Generator (LCG)[1] to generate pseudo random numbers. It looked random, so we used it. Then some egghead said "give me a few outputs of your LCG and I can reverse it back to a seed"[2]. Oh damn! What do we do? The obvious solution: Reseed the RNG before you use it. Then another egghead said "I've invented Mersenne Twister (MT). It is faster and more secure"[3]. So we switched to that. No need to reseed the RNG anymore! But then someone said "We have broken MT. Take some numbers. Give it to this app, and it will give you the seed"[4]. And so we started to reseed the RNG again. Today it is hash-chaining, XOR-shift and improved linear-feedback shift register algorithms. Do you want to put your money on that we have now reached bug-free well-enough PRNGs that we no longer have fiddle with reseeding? It is a dangerous gamble. More than anything I wish we were clever enough to create robust, correctness proven and high-performance PRNGs so we could stop all this nonsense - but alas, here we are. [1] https://academic.oup.com/comjnl/article/1/2/83/425243 https://academic.oup.com/comjnl/article/1/2/83/425243 [2] https://en.wikipedia.org/wiki/Marsaglia%27s_theorem https://en.wikipedia.org/wiki/Marsaglia%27s_theorem [3] http://www.math.sci.hiroshima-u.ac.jp/m-mat/MT/ARTICLES/mt.pdf http://www.math.sci.hiroshima-u.ac.jp/m-mat/MT/ARTICLES/mt.p... [4] https://github.com/altf4/untwister https://github.com/altf4/untwister
- eternityforest 4y ago
  Xorshift, lfsr, Mersenne, and LCG were never intended to be secure as far as I know, and were never cryptographically broken because they were never secure to begin with. They discovered flaws that could bias some large scientific simulations and such in some of them. But nobody is going to be directly using USB numbers to make billions a second for simulations, these dongles are mostly meant for cryptographically secure stuff. Their competition is whatever algorithms Linux urandom snd OpenSSL are using at the moment, and the builtin RNGs most chips have had for a decade. CSPRNGs are occasional broken, but it seems to happen about as often as other crypto primitives. They are based on hashes and ciphers usually, and these days those can go for decades without much progress on a break. Unless P==NP or something happens and ruins all the fun, I'm fine with current crypto prngs. I don't think I've ever handled data that was so critical that anything else was needed.
kevin_thibedeau 4y ago
They have applications beyond cryptography. One use case is to generate a unique random ID at startup. You can't do that without some sort of entropy source.
JoachimS 4y ago
In many embedded systems and constrained devices, there is a lack of proper randomness. And often the system environment is too stable to use as source of entropy (by connecting a sensor). This makes it hard to set up secure sessions. If fairness, more and more MCUs (esp based on ARM Cortex) that contain some sort of TRNG / entropy source. Some of them have fairly good quality too. But there are still tons of 8051-based MCUs that lack any source of randomness. Having a cheap, physically compact, low power source of entropy that can be attached is in these cases quite useful. Esp for securing systems already in use.

by hannob 4y ago

dale_glass 4y ago

I remember Linux running into a bunch of issues with boot time entropy. Some things were sometimes getting stuck on boot due to that. Also this "lost entropy" thing has long been a bane of VM servers, which could consume vast amounts of it. I think this changed fairly recently. And lots of things have USB ports these days, including things like commercial wifi access points, routers, switches, etc. Some of those have internal ones.

dragontamer 4y ago

All Intel and AMD chips come with a hardware RNG today: the RDRAND and RDSEED instructions. RDRAND is kinda-sorta like /dev/urandom, while RDSEED is /dev/random with stronger entropy guarantees. Linux didn't want to "trust" this hardware, for whatever reason. I guess there was that one time an AMD bug in the instruction returned all-zeros (or something similar). But as "long as the instruction is properly working", it is... well... working, lol. But its not like its really possible to test if its working correctly. -------- From that perspective, a hardware RNG needs to be "large" and "discrete" enough for humans to easily test and prove that its working, to create trust. Otherwise, hardware and/or firmware bugs could lead to a security problem. EDIT: if you are at a mass produced but don't verify kinda person, then you might as well use RDSEED instructions to get her entropy. Intel/AMD probably are good enough to prevent firmware / hardware bugs. I know that VMs also have an issue here: whether or not to make RDSEED deterministic/predictable or not. Should the future RDSEED instructions be saved in a VM snapshot?

jnwatson 4y ago
You can (and are required to in certain environments) test for basic failures in your hardware RNG. The AMD bug was bad not because of the bug, but because it was obvious they had no built-in-test functionality.
garaetjjte 4y ago
>RDRAND is kinda-sorta like /dev/urandom, while RDSEED is /dev/random with stronger entropy guarantees. /dev/urandom and /dev/random is almost the same nowadays, the only difference is that random will block after boot until enough entropy is collected, but it never "runs out" of entropy anymore.
adrian_b 4y ago
Any hardware RNG that is included in a CPU or any other complex integrated circuit cannot be trusted, because it is impossible to verify that it is truly a RNG and not a pseudo-RNG that implements backdoors. The only hardware RNGs that can be trusted are those that use an analog-digital converter (e.g. one of the ADCs included in any microcontroller, or the audio microphone input of a computer) and a discrete analog noise source, e.g. a diode or a resistor, because their components can be tested separately, to verify that they do what they claim to do. Nevertheless, the output of the hardware RNGs provided by the Intel or AMD CPUs can be used in a reasonably secure way if the supposedly random values are never used directly, but they are passed through a cryptographic hash, e.g. SHA-256. Even SHA-1 is completely adequate for this special purpose, where collision-resistance does not matter (and it is faster than any other secure hashes in the ARM, AMD or Intel CPUs with SHA-1 instructions). Even better, the supposedly random value could be hashed together with the timestamp read from the TSC, or just with the value of a counter incremented after generating a random number, for additional unpredictability. One can add to the hash a unique computer identifier, e.g. the MAC address of the first Ethernet interface. In this case, distinct sequences of uniformly-distributed pseudo-random numbers will continue to be produced even in computers where the hardware RNG remains stuck to a constant value, as it happened in some buggy AMD Zen models. The hashing (with a secure hash algorithm) of the RNG output destroys any hidden structure that might have been exploitable by the designers of the hardware RNG, while preserving the uniform distribution of the random numbers.

jnwatson 4y ago

The device itself isn't useful, but the design could be incorporated into systems that need a better source of early boot time entropy. Lots of embedded systems don't such a source, though it is getting better as more SoC vendors incorporate HW RNG.

Zababa 4y ago

For the full story, I was trying to reduce the boot time on my Linux system. One of the things that takes the most time is systemd-random-seed. From what I understand, it takes this long because my computer is not generating entropy fast enough, so it has to wait for a while. The solution to fix this is to find a way to generate more entropy. The system doc on random seeds has lots of information about what you can use: https://systemd.io/RANDOM_SEEDS/ https://systemd.io/RANDOM_SEEDS/. One of the options is RDRAND, which my CPU can do. But after reading a bit about it, it seems that some people are supicious of RDRAND. I have no idea if I should too, and since it was disabled by default on my distribution, I decided to keep it of for now, as I don't really understand this subject. At this point I wondered if any hardware RNG existed, and found the OneRNG, that seems like a cool project, so I posted it here. This is not a recommandation to use OneRNG instead of RDRAND, I don't know enough about this domain to suggest one or another.

riedel 4y ago

Most applications actually IMHO come from statistics and simulation. It seems PRNGs are sometimes problematic if you need true white noise...

Genbox 4y ago

You are correct. However, history teaches us important lessons. Back in the old days when cryptography was a weird concept nobody understood, we had the Linear Congruential Generator (LCG)[1] to generate pseudo random numbers. It looked random, so we used it. Then some egghead said "give me a few outputs of your LCG and I can reverse it back to a seed"[2]. Oh damn! What do we do? The obvious solution: Reseed the RNG before you use it. Then another egghead said "I've invented Mersenne Twister (MT). It is faster and more secure"[3]. So we switched to that. No need to reseed the RNG anymore! But then someone said "We have broken MT. Take some numbers. Give it to this app, and it will give you the seed"[4]. And so we started to reseed the RNG again. Today it is hash-chaining, XOR-shift and improved linear-feedback shift register algorithms. Do you want to put your money on that we have now reached bug-free well-enough PRNGs that we no longer have fiddle with reseeding? It is a dangerous gamble. More than anything I wish we were clever enough to create robust, correctness proven and high-performance PRNGs so we could stop all this nonsense - but alas, here we are. [1] https://academic.oup.com/comjnl/article/1/2/83/425243 https://academic.oup.com/comjnl/article/1/2/83/425243 [2] https://en.wikipedia.org/wiki/Marsaglia%27s_theorem https://en.wikipedia.org/wiki/Marsaglia%27s_theorem [3] http://www.math.sci.hiroshima-u.ac.jp/m-mat/MT/ARTICLES/mt.pdf http://www.math.sci.hiroshima-u.ac.jp/m-mat/MT/ARTICLES/mt.p... [4] https://github.com/altf4/untwister https://github.com/altf4/untwister

eternityforest 4y ago
Xorshift, lfsr, Mersenne, and LCG were never intended to be secure as far as I know, and were never cryptographically broken because they were never secure to begin with. They discovered flaws that could bias some large scientific simulations and such in some of them. But nobody is going to be directly using USB numbers to make billions a second for simulations, these dongles are mostly meant for cryptographically secure stuff. Their competition is whatever algorithms Linux urandom snd OpenSSL are using at the moment, and the builtin RNGs most chips have had for a decade. CSPRNGs are occasional broken, but it seems to happen about as often as other crypto primitives. They are based on hashes and ciphers usually, and these days those can go for decades without much progress on a break. Unless P==NP or something happens and ruins all the fun, I'm fine with current crypto prngs. I don't think I've ever handled data that was so critical that anything else was needed.

kevin_thibedeau 4y ago

They have applications beyond cryptography. One use case is to generate a unique random ID at startup. You can't do that without some sort of entropy source.

JoachimS 4y ago

In many embedded systems and constrained devices, there is a lack of proper randomness. And often the system environment is too stable to use as source of entropy (by connecting a sensor). This makes it hard to set up secure sessions. If fairness, more and more MCUs (esp based on ARM Cortex) that contain some sort of TRNG / entropy source. Some of them have fairly good quality too. But there are still tons of 8051-based MCUs that lack any source of randomness. Having a cheap, physically compact, low power source of entropy that can be attached is in these cases quite useful. Esp for securing systems already in use.