4 ms·

(disclaimer I sell a HW random number generator) I think both the diagrams here are simplistic - in reality pre 4.8 there were 3 pools (an input pool and 2 out

(disclaimer I sell a HW random number generator) I think both the diagrams here are simplistic - in reality pre 4.8 there were 3 pools (an input pool and 2 output pools, one of which blocked) and post 4.8 where there are 2 pools, an input pool and a blocking pool (urandom now pulls from the input pool thru a CSPRNG). One big downside of the new (post 4.8) architecture is that urandom_min_reseed_secs is ignored - pre-4.8 you could ask the kernel to periodically pull a little bit of real entropy from the input to refresh the urandom output (non-blocking) pool, by default that was once every 60 seconds, but in our case with lots of entropy available we'd set it to happen once a second - now post 4.8 it looks like urandom_min_reseed_secs is ignored and the input pool (where /dev/urandom gets its data from) is not regularly updated unless data is also pulled from /dev/random - which may make sense if entropy is rare, but is not so useful when back pressure from /dev/random is left to control the rate at which external entropy flows into the system - I suspect it's time to present a patch that makes urandom_min_reseed_secs work again Nevertheless the conclusion "use /dev/urandom" is still valid

Taniwha 8y ago
BTW - a tiny bit of python to keep /dev/urandom fed post 4.8 - useful if you have a hardware RNG (either an external one or the one in your CPU): #! /usr/bin/python # # OneRNG /dev/urandom tickler # (C) Copyright Paul Campbell Moonbase Otago 2018 # Released under a GPL 3.0 license # import time with open("/dev/random", 'rb') as r: while True: with open("/proc/sys/kernel/random/entropy_avail", 'r') as e: iv = e.read() print(iv) if iv > 1500: x= r.read(1000) e.close() time.sleep(1)
- canhascodez 8y ago
  Ruby version: #!/usr/bin/env ruby source = File.open('/dev/random', 'rb') loop do source.read(1000) unless File.read('/proc/sys/kernel/random/entropy_avail').to_i < 1500 sleep 1 end
- cuspycode 8y ago
  Do you have a gist or similar link for this code? Copy & paste doesn't work due to indentation issues.
- acqq 8y ago
  GPL 3.0 for a few obvious lines seems to be really an unfitting license. Non-licensed English language version, containing the same information: Once per second, read /proc/sys/kernel/random/entropy_avail and if the resulting number is more than 1500 perform a read of 1000 bytes from /dev/random.
  - nothrabannosir 8y ago
    (this is more of a tangential licensing PSA, since we're on the subject) Not licensing something is, ironically, restrictive. If you want to release to public domain (or equivalent), you have to explicitly mention that. E.g. by licensing under CC0. Otherwise, default copyright and "all rights reserved" applies. https://creativecommons.org/share-your-work/public-domain/cc0/ https://creativecommons.org/share-your-work/public-domain/cc... (But, IMO, nothing wrong with publicizing use of GPL 3! :) thanks either way to both)
    - semi-extrinsic 8y ago
      I think GP is getting at something else. As the length/complexity of the code goes to zero, the copyright and "all rights reserved" vanishes; this is well established, or else we'd all have to license "Hello world" examples from Kernighan and Ritchie's book. If someone writes the same one-sentence pseudocode as GP in a book or blogpost, nobody could reasonably claim that an implementation would fall under the copyright of that book. And I think you'd be very hard pressed to argue it's copyrightable just because someone typed it out into real code.
  - Taniwha 8y ago
    sorry, just grabbed my standard disclaimer, can't release code without copyrighting it first. As others have pointed out it's trivial to reimplement in the language of your choice go for it
- paavoova 8y ago
  entropy_avail returns bits of entropy to a maximum of 4096(?), but this reads 1000 bytes. Why check if it's greater than 1500 if you're going to use it all anyway? Am I missing something here?
  - Taniwha 8y ago
    remember that this is for people with external entropy sources, the kernel entropy will refill fast. 1500 is more a measure of "is /dev/random busy at the moment" if it is then it's pulling data from from the external entropy source and /dev/urandom is essentially already being fed with more entropy The missing mechanism we're trying to emulate used to essentially read N bytes (whatever user space happened to want at the time) every M seconds our experience has been that that can be way more than 1000 bytes, but reduce it if you like
- tptacek 8y ago
  What problem does this solve?
ChrisSD 8y ago
Does increasing the rate of external entropy risk swamping the system generated entropy? I.e. if the external source is compromised could this also compromise urandom such that the results could become predictable?
- tptacek 8y ago
  No! This is the thing everyone needs to understand: once you have achieved "entropy" with a modern CSPRNG (including T'so's), you never lose that property. Think of the RNG like a stream cipher like AES-CTR. How much data do you have to encrypt with AES-CTR to "lose" or "leak" the underlying AES key?
  - Taniwha 8y ago
    Here I have to disagree (in the small) essentially the problem of 'breaking' a RNG is discovering that hidden state (the entropy pool) behind the CSPRNG - if you believe that the NSA, and their competitors, don't each have a team of PhDs pouring over the linux random code looking for ways to do that you're dreaming. After all if you know its state you can predict the keys it will make in the future My intuition is that every operation we do (start TCP connections, generate keys, backoff after a failed login, ....) reveals a tiny fraction of a bit about that hidden state, web assembly calling random over and over again probably reveals much much more, over time it builds up. (anyone want to guess if there's a quantum computer attack here, it kind of looks like the sort of problem where there might be) So I think that with a sophisticated enough attacker entropy does degrade, the trick is to replace that entropy at a faster rate that the rate at which you guess the worst case attacker might be able to guess. For most people it probably doesn't matter, you're exposing so little state over time you're likely effectively as safe as the parent poster claims, it kind of depends on whether, in this age when we know that nation state actors are watching every packet that passes thru their hands, watching every thing we're doing online, are you interesting enough for them to spend the time and effort? does the US/5Is think you're not a Trump supporter? are the Chinese interested in your trade secrets? whi knows what the Russians think As I mentioned above using /dev/urandom doesn't have a mechanism you can use to signal this guess about how much entropy it needs over time to be safe, prior to 4.8 we used to have a mechanism that said "give it some more entropy every N seconds, 60 by default" but that's now broken (provided here a python applet that periodically drains a little bit of /dev/random entropy if it has plenty available to stir the input pool that /dev/urandom uses for its entropy source)
brennebeck 8y ago
While I agree with what you’re saying, I think it should be pointed out that the author clearly states that it’s a simplistic model and even mentions the multiple pools briefly (though I’m unsure if that was add post-comment, as author is on hn also).
- Tomte 8y ago
  No, the page hasn‘t been changed in quite some time. The last real change was the 4.8 change, long after 4.8 was out.

Taniwha 8y ago

BTW - a tiny bit of python to keep /dev/urandom fed post 4.8 - useful if you have a hardware RNG (either an external one or the one in your CPU): #! /usr/bin/python # # OneRNG /dev/urandom tickler # (C) Copyright Paul Campbell Moonbase Otago 2018 # Released under a GPL 3.0 license # import time with open("/dev/random", 'rb') as r: while True: with open("/proc/sys/kernel/random/entropy_avail", 'r') as e: iv = e.read() print(iv) if iv > 1500: x= r.read(1000) e.close() time.sleep(1)

canhascodez 8y ago
Ruby version: #!/usr/bin/env ruby source = File.open('/dev/random', 'rb') loop do source.read(1000) unless File.read('/proc/sys/kernel/random/entropy_avail').to_i < 1500 sleep 1 end
cuspycode 8y ago
Do you have a gist or similar link for this code? Copy & paste doesn't work due to indentation issues.
acqq 8y ago
GPL 3.0 for a few obvious lines seems to be really an unfitting license. Non-licensed English language version, containing the same information: Once per second, read /proc/sys/kernel/random/entropy_avail and if the resulting number is more than 1500 perform a read of 1000 bytes from /dev/random.
- nothrabannosir 8y ago
  (this is more of a tangential licensing PSA, since we're on the subject) Not licensing something is, ironically, restrictive. If you want to release to public domain (or equivalent), you have to explicitly mention that. E.g. by licensing under CC0. Otherwise, default copyright and "all rights reserved" applies. https://creativecommons.org/share-your-work/public-domain/cc0/ https://creativecommons.org/share-your-work/public-domain/cc... (But, IMO, nothing wrong with publicizing use of GPL 3! :) thanks either way to both)
  - semi-extrinsic 8y ago
    I think GP is getting at something else. As the length/complexity of the code goes to zero, the copyright and "all rights reserved" vanishes; this is well established, or else we'd all have to license "Hello world" examples from Kernighan and Ritchie's book. If someone writes the same one-sentence pseudocode as GP in a book or blogpost, nobody could reasonably claim that an implementation would fall under the copyright of that book. And I think you'd be very hard pressed to argue it's copyrightable just because someone typed it out into real code.
- Taniwha 8y ago
  sorry, just grabbed my standard disclaimer, can't release code without copyrighting it first. As others have pointed out it's trivial to reimplement in the language of your choice go for it
paavoova 8y ago
entropy_avail returns bits of entropy to a maximum of 4096(?), but this reads 1000 bytes. Why check if it's greater than 1500 if you're going to use it all anyway? Am I missing something here?
- Taniwha 8y ago
  remember that this is for people with external entropy sources, the kernel entropy will refill fast. 1500 is more a measure of "is /dev/random busy at the moment" if it is then it's pulling data from from the external entropy source and /dev/urandom is essentially already being fed with more entropy The missing mechanism we're trying to emulate used to essentially read N bytes (whatever user space happened to want at the time) every M seconds our experience has been that that can be way more than 1000 bytes, but reduce it if you like
tptacek 8y ago
What problem does this solve?

ChrisSD 8y ago

Does increasing the rate of external entropy risk swamping the system generated entropy? I.e. if the external source is compromised could this also compromise urandom such that the results could become predictable?

tptacek 8y ago
No! This is the thing everyone needs to understand: once you have achieved "entropy" with a modern CSPRNG (including T'so's), you never lose that property. Think of the RNG like a stream cipher like AES-CTR. How much data do you have to encrypt with AES-CTR to "lose" or "leak" the underlying AES key?
- Taniwha 8y ago
  Here I have to disagree (in the small) essentially the problem of 'breaking' a RNG is discovering that hidden state (the entropy pool) behind the CSPRNG - if you believe that the NSA, and their competitors, don't each have a team of PhDs pouring over the linux random code looking for ways to do that you're dreaming. After all if you know its state you can predict the keys it will make in the future My intuition is that every operation we do (start TCP connections, generate keys, backoff after a failed login, ....) reveals a tiny fraction of a bit about that hidden state, web assembly calling random over and over again probably reveals much much more, over time it builds up. (anyone want to guess if there's a quantum computer attack here, it kind of looks like the sort of problem where there might be) So I think that with a sophisticated enough attacker entropy does degrade, the trick is to replace that entropy at a faster rate that the rate at which you guess the worst case attacker might be able to guess. For most people it probably doesn't matter, you're exposing so little state over time you're likely effectively as safe as the parent poster claims, it kind of depends on whether, in this age when we know that nation state actors are watching every packet that passes thru their hands, watching every thing we're doing online, are you interesting enough for them to spend the time and effort? does the US/5Is think you're not a Trump supporter? are the Chinese interested in your trade secrets? whi knows what the Russians think As I mentioned above using /dev/urandom doesn't have a mechanism you can use to signal this guess about how much entropy it needs over time to be safe, prior to 4.8 we used to have a mechanism that said "give it some more entropy every N seconds, 60 by default" but that's now broken (provided here a python applet that periodically drains a little bit of /dev/random entropy if it has plenty available to stir the input pool that /dev/urandom uses for its entropy source)

brennebeck 8y ago

While I agree with what you’re saying, I think it should be pointed out that the author clearly states that it’s a simplistic model and even mentions the multiple pools briefly (though I’m unsure if that was add post-comment, as author is on hn also).

Tomte 8y ago
No, the page hasn‘t been changed in quite some time. The last real change was the 4.8 change, long after 4.8 was out.