4 ms·

Someone please ELI5 this?

valarauca1 11y ago
Basically thread scheduling is an imperfect art. So when you tell a thread to sleep for 4ms, it may sleep for 1-16ms depending on other threads work load, resource contention, etc. This is a huge problem if you work in very precise time sensitive data (I do ISO17025 flow calibration systems), and when you start needing >100 data points per second, of dynamic IO you will run into millisecond timing issue standard time shared OS's will introduce. The solution to this is Real-Time OS's (not Real-Time webdevelopment) that handle scheduling differently, and prioritize IO tasks even higher. But most of these suck. :.:.: This random noise (from the scheduler) is a pretty good source of entropy (and the only assumption is your Unix box has other processes, that are using resources). Because your process, generally has no clue that hundreds of other processes are inflight and also being executed at the same time. So their interference, is effectively magic data from nowhere. Dan Kaminsky did a talk at Defcon last year where he mentioned the need for this type of CSRNG [1] I even tooled up a very basic (and flawed) implementation a few months ago to play with the notion [2] [1] https://www.youtube.com/watch?v=xneBjc8z0DE https://www.youtube.com/watch?v=xneBjc8z0DE [2] https://github.com/valarauca/Simple-CSRNG https://github.com/valarauca/Simple-CSRNG
- hurin 11y ago
  Is there a significant advantage? Either-way you just need to fill /dev/random enough for /urandom to be well seeded.
  - valarauca1 11y ago
    Yes. Interprocess scheduling isn't currently a source of entropy for /dev/random or /dev/urandom. Edit: I have random/urandom backwards. Still doesn't change my core point. Sorry for the confusion. /dev/random is a PRNG, and predictable. You shouldn't use it for security applications but only for specific state/algorithm randomness. /dev/urandom requires hardware noise/key events, etc. to generate its entropy. These become hard to find when your dealing with purely virtualized installations. :.:.: The key focus for this is webapps, or should be. Far to many use PRNG to give session cookies, and these are very very easy to hyjack especially if cookies can be issued whenever a user logs in/out. Its pretty trivial to generate 1,000 -> 5,000 session cookies (from login/logout) and attempt to find a PRNG pattern.
    - garrettr_ 11y ago
      /dev/random is a PRNG, and predictable. You shouldn't use it for security applications but only for specific state/algorithm randomness. /dev/urandom requires hardware noise/key events, etc. to generate its entropy. These become hard to find when your dealing with purely virtualized installations. This is not true, at least on Linux. /dev/random is actually closer to what you describe as /dev/urandom. It is a cryptographically strong randomness source that blocks depending on the state of its internal "entropy pool". /dev/urandom is also a cryptographically strong randomness source (seeded from /dev/random), but it does not block. Here's a great blog post with more info: http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/
      - tptacek 11y ago
        On the BSDs, there's no real distinction between the two, and the names are just there for compat.
    - hurin 11y ago
      > /dev/random is a PRNG, and predictable. You shouldn't use it for security applications but only for specific state/algorithm randomness. > /dev/urandom requires hardware noise/key events, etc. to generate its entropy. These become hard to find when your dealing with purely virtualized installations. You have it backwards. http://en.wikipedia.org/?title=/dev/random http://en.wikipedia.org/?title=/dev/random
    - bitwize 11y ago
      Actually /dev/random is truly random and requires entropy inputs; once the current entropy pool is exhausted, reads from /dev/random block. /dev/urandom is similar but reuses the entropy pool to produce PR values until more entropy comes in, so it's good for games and such.
      - tptacek 11y ago
        As the comments above point out, this isn't correct.
    - 11y ago
- xenadu02 11y ago
  That's not what the paper is doing. It's exploiting randomness from pipeline stalls, memory access latencies, CPU frequency scaling, etc. Even disabling interrupts and cache doesn't reduce the entropy enough to make it non-random.
  - valarauca1 11y ago
    Yes it is. Just your looking at the system at a different level. Resource contention is the stalls caused by context switching, cache missing, system latencies, and branch prediction. Ultimately these 4 sources of latency build on top of each other to generate all your stalls, and slow downs in the system. What you see as CPU frequency scaling, I just see as my state switching in my program happen slower. What you see as pipeline stalls are just cache/ram misses, hard disk IO latency, and lots of context switches compounding on each other. TL;DR You say "a starchy, tuberous crop from the perennial nightshade Solanum tuberosum L.", I say potato.

Basically thread scheduling is an imperfect art. So when you tell a thread to sleep for 4ms, it may sleep for 1-16ms depending on other threads work load, resource contention, etc. This is a huge problem if you work in very precise time sensitive data (I do ISO17025 flow calibration systems), and when you start needing >100 data points per second, of dynamic IO you will run into millisecond timing issue standard time shared OS's will introduce. The solution to this is Real-Time OS's (not Real-Time webdevelopment) that handle scheduling differently, and prioritize IO tasks even higher. But most of these suck. :.:.: This random noise (from the scheduler) is a pretty good source of entropy (and the only assumption is your Unix box has other processes, that are using resources). Because your process, generally has no clue that hundreds of other processes are inflight and also being executed at the same time. So their interference, is effectively magic data from nowhere. Dan Kaminsky did a talk at Defcon last year where he mentioned the need for this type of CSRNG [1] I even tooled up a very basic (and flawed) implementation a few months ago to play with the notion [2] [1] https://www.youtube.com/watch?v=xneBjc8z0DE https://www.youtube.com/watch?v=xneBjc8z0DE [2] https://github.com/valarauca/Simple-CSRNG https://github.com/valarauca/Simple-CSRNG

hurin 11y ago
Is there a significant advantage? Either-way you just need to fill /dev/random enough for /urandom to be well seeded.
- valarauca1 11y ago
  Yes. Interprocess scheduling isn't currently a source of entropy for /dev/random or /dev/urandom. Edit: I have random/urandom backwards. Still doesn't change my core point. Sorry for the confusion. /dev/random is a PRNG, and predictable. You shouldn't use it for security applications but only for specific state/algorithm randomness. /dev/urandom requires hardware noise/key events, etc. to generate its entropy. These become hard to find when your dealing with purely virtualized installations. :.:.: The key focus for this is webapps, or should be. Far to many use PRNG to give session cookies, and these are very very easy to hyjack especially if cookies can be issued whenever a user logs in/out. Its pretty trivial to generate 1,000 -> 5,000 session cookies (from login/logout) and attempt to find a PRNG pattern.
  - garrettr_ 11y ago
    /dev/random is a PRNG, and predictable. You shouldn't use it for security applications but only for specific state/algorithm randomness. /dev/urandom requires hardware noise/key events, etc. to generate its entropy. These become hard to find when your dealing with purely virtualized installations. This is not true, at least on Linux. /dev/random is actually closer to what you describe as /dev/urandom. It is a cryptographically strong randomness source that blocks depending on the state of its internal "entropy pool". /dev/urandom is also a cryptographically strong randomness source (seeded from /dev/random), but it does not block. Here's a great blog post with more info: http://www.2uo.de/myths-about-urandom/ http://www.2uo.de/myths-about-urandom/
    - tptacek 11y ago
      On the BSDs, there's no real distinction between the two, and the names are just there for compat.
  - hurin 11y ago
    > /dev/random is a PRNG, and predictable. You shouldn't use it for security applications but only for specific state/algorithm randomness. > /dev/urandom requires hardware noise/key events, etc. to generate its entropy. These become hard to find when your dealing with purely virtualized installations. You have it backwards. http://en.wikipedia.org/?title=/dev/random http://en.wikipedia.org/?title=/dev/random
  - bitwize 11y ago
    Actually /dev/random is truly random and requires entropy inputs; once the current entropy pool is exhausted, reads from /dev/random block. /dev/urandom is similar but reuses the entropy pool to produce PR values until more entropy comes in, so it's good for games and such.
    - tptacek 11y ago
      As the comments above point out, this isn't correct.
  - 11y ago
xenadu02 11y ago
That's not what the paper is doing. It's exploiting randomness from pipeline stalls, memory access latencies, CPU frequency scaling, etc. Even disabling interrupts and cache doesn't reduce the entropy enough to make it non-random.
- valarauca1 11y ago
  Yes it is. Just your looking at the system at a different level. Resource contention is the stalls caused by context switching, cache missing, system latencies, and branch prediction. Ultimately these 4 sources of latency build on top of each other to generate all your stalls, and slow downs in the system. What you see as CPU frequency scaling, I just see as my state switching in my program happen slower. What you see as pipeline stalls are just cache/ram misses, hard disk IO latency, and lots of context switches compounding on each other. TL;DR You say "a starchy, tuberous crop from the perennial nightshade Solanum tuberosum L.", I say potato.