5 ms·

Arc4random – randomization for all occasions

80 points by edwintorok 12y ago

edwintorok 12y ago
Shouldn't glibc provide an arc4random? Apparently the new syscall is not for direct use [1], and you might not want to link with LibreSSL just to get arc4random [2] [1] http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00041.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000... [2] http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00040.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000...
- throwaway2048 12y ago
  The chances of glibc importing openbsd derived functions are practically nil unfortunately, as we saw with the strlcat saga. With the departure of Drepper, things might have changed, but strlcat is still not imported.
  - sarciszewski 12y ago
    What is the reason for that? :(
    - pdw 12y ago
      Not specified in any standard, not much of an improvement according to the glibc maintainers, and already provided in libbsd.
      - edwintorok 12y ago
        Speaking of libbsd it should be updated to use ChaCha20 too: https://bugs.freedesktop.org/show_bug.cgi?id=85827 https://bugs.freedesktop.org/show_bug.cgi?id=85827
    - IvyMike 12y ago
      Drepper's caustic style aside, his point ultimately was that truncation is still a problem. Whether or not you think glibc should include strlcat, this is an important point to understand. More discussion here: http://stackoverflow.com/q/2114896/67591 http://stackoverflow.com/q/2114896/67591
      - tedunangst 12y ago
        Note that you can mechanically translate strcat(dst, src) to strlcat(dst, src, SIZE_MAX) with no loss of functionality or change in behavior. No worries about truncation, either. Exactly the same thing. Now, if reading a piece of code that does strlcat(dst, src, SIZE_MAX) would make you feel uncomfortable, the question is how should you feel reading code that uses strcat?
  - justincormack 12y ago
    Things have changed enormously since Drepper left. It is a nice project to work with and many things that were vetoed are getting in.
  - fafner 12y ago
    I don't think that's true. Drepper was against strlcat for specific reasons. And in the meanwhile the C11 standard had adopted an alternative (but imo worse) API for it, based on the Microsoft stuff. Framing this as a general rejection of APIs because they are from OpenBSD is highly disingenuous.
- justincormack 12y ago
  There is a proposal to make a Posix interface that is essentially similar.[1] [1] http://austingroupbugs.net/view.php?id=859 http://austingroupbugs.net/view.php?id=859
  - tedunangst 12y ago
    And now I have more work to do... :)
    - joehillen 12y ago
      So is this going in? Are you using arc4random()?
      - tedunangst 12y ago
        That's certainly possible. I don't control what goes where, I only made the proposal.
protomyth 12y ago
Link to Hackfest 2014: Theo de Raadt presented "arc4random - randomization for all occasions" video https://www.youtube.com/watch?v=aWmLWx8ut20 https://www.youtube.com/watch?v=aWmLWx8ut20 via http://www.bsdnow.tv/episodes/ http://www.bsdnow.tv/episodes/ a great podcast with ep 64 talking about the presentation and whole lot more.
101914 12y ago
All occasions, even shell scripts. For example to choose a pseudorandom entry from a (reasonably short) list and set it as an environmental variable: echo "main(){ arc4random();}" |gcc -x c -pipe -o arc4random -static - b1=$(sed -n '$!D;=' .list); c1=$((b1/255)); arc4random; d1=$(($?*c1)); VARIABLE=$( sed ''"$d1"'!d;'"$d1"'q' .list ); echo $VARIABLE
- acveilleux 12y ago
  It's simpler to dd from /dev/[u]random however and in OpenBSD, you'll get exactly the same quality result. Most of the benefits of arc4random() are nullified by invoking gcc...
  - tedunangst 12y ago
    On OpenBSD, it's even simpler to just echo $RANDOM, but that only gets you 15 bits because sh. sysctl -n kern.arandom can be used in a pinch, too.
    - eridius 12y ago
      Theoretically couldn't you do something like echo $(( $RANDOM << 15 | $RANDOM )) to get 30 bits?
  - 101914 12y ago
    OK, I feel stupid. How would I get a random number using dd and /dev/urandom? I am not sure I understand the gcc comment. Can you explain? Assuming I cannot get a random number from dd'ing /dev/urandom then I needed a utility to do it where the system only has a small base and does not have gcc. Unangst's suggestion to use sysctl is better (although it means I have to crunch the sysctl utility). I will be using that from now on. Thank you for the tip! As for $RANDOM, that may be OpenBSD-specific. For example, does NetBSD have it?
    - Sanddancer 12y ago
      $RANDOM is shell-specific. bash has $RANDOM, ksh has $RANDOM, and zsh has $RANDOM, standard sh doesn't have it.
acqq 12y ago
The two (for me) important slides: http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00016.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000... rc4random cannot fail Look at the function definitions: uint32_t arc4random(void); void arc4random_buf(void *buf, size_t nbytes); uint32_t arc4random_uniform(uint32_t upper_bound); All 3 functions guarantee a result. They will not fail, and cannot return an error. http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00041.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000... Linux? We have encouraged Ted Ts'o to create a getentropy-like interface for Linux; getrandom() Too many options, too easy to misuse Interactions with signal handlers (EINTR) Hope applications do not directly call getrandom()... Bit worrying Really feels like there is pressure against easy-access random.. "Use /dev/random" meme continues damaging effects Linux's getrandom really has a bunch of error codes: http://lwn.net/Articles/606202/ http://lwn.net/Articles/606202/ EINVAL An invalid flag was passed to getrandom(2) EFAULT buf is outside the accessible address space. EAGAIN The requested entropy was not available, and the getentropy(2) would have blocked if GRND_BLOCK flag was set. EINTR While blocked waiting for entropy, the call was interrupted by a signal handler; see the description of how interrupted read(2) calls on "slow" devices are handled with and without the SA_RESTART flag in the signal(7) man page. And a few flags: GRND_NONBLOCK Don't block and return EAGAIN instead GRND_RANDOM Use the /dev/random pool instead of /dev/urandom I agree that the best interface is the one that can't fail.
- shawn-butler 12y ago
  arc4random has a larger range as well up to 0x100000000, whereas rand/random top out at RAND_MAX of 0x7fffffff?
- ushi 12y ago
  getrandom() is a generic interface you can use to implement getentropy(), which is used to seed arc4random(). It is not an alternative arc4random implementation... From the OpenBSD Manpage: getentropy() will succeed unless: [EFAULT] The buf parameter points to an invalid address. [EIO] Too many bytes requested, or some other fatal error occurred. See: http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2 http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/...

80 points by edwintorok 12y ago

Shouldn't glibc provide an arc4random? Apparently the new syscall is not for direct use [1], and you might not want to link with LibreSSL just to get arc4random [2] [1] http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00041.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000... [2] http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00040.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000...

throwaway2048 12y ago
The chances of glibc importing openbsd derived functions are practically nil unfortunately, as we saw with the strlcat saga. With the departure of Drepper, things might have changed, but strlcat is still not imported.
- sarciszewski 12y ago
  What is the reason for that? :(
  - pdw 12y ago
    Not specified in any standard, not much of an improvement according to the glibc maintainers, and already provided in libbsd.
    - edwintorok 12y ago
      Speaking of libbsd it should be updated to use ChaCha20 too: https://bugs.freedesktop.org/show_bug.cgi?id=85827 https://bugs.freedesktop.org/show_bug.cgi?id=85827
  - IvyMike 12y ago
    Drepper's caustic style aside, his point ultimately was that truncation is still a problem. Whether or not you think glibc should include strlcat, this is an important point to understand. More discussion here: http://stackoverflow.com/q/2114896/67591 http://stackoverflow.com/q/2114896/67591
    - tedunangst 12y ago
      Note that you can mechanically translate strcat(dst, src) to strlcat(dst, src, SIZE_MAX) with no loss of functionality or change in behavior. No worries about truncation, either. Exactly the same thing. Now, if reading a piece of code that does strlcat(dst, src, SIZE_MAX) would make you feel uncomfortable, the question is how should you feel reading code that uses strcat?
- justincormack 12y ago
  Things have changed enormously since Drepper left. It is a nice project to work with and many things that were vetoed are getting in.
- fafner 12y ago
  I don't think that's true. Drepper was against strlcat for specific reasons. And in the meanwhile the C11 standard had adopted an alternative (but imo worse) API for it, based on the Microsoft stuff. Framing this as a general rejection of APIs because they are from OpenBSD is highly disingenuous.
justincormack 12y ago
There is a proposal to make a Posix interface that is essentially similar.[1] [1] http://austingroupbugs.net/view.php?id=859 http://austingroupbugs.net/view.php?id=859
- tedunangst 12y ago
  And now I have more work to do... :)
  - joehillen 12y ago
    So is this going in? Are you using arc4random()?
    - tedunangst 12y ago
      That's certainly possible. I don't control what goes where, I only made the proposal.

protomyth 12y ago

Link to Hackfest 2014: Theo de Raadt presented "arc4random - randomization for all occasions" video https://www.youtube.com/watch?v=aWmLWx8ut20 https://www.youtube.com/watch?v=aWmLWx8ut20 via http://www.bsdnow.tv/episodes/ http://www.bsdnow.tv/episodes/ a great podcast with ep 64 talking about the presentation and whole lot more.

101914 12y ago

All occasions, even shell scripts. For example to choose a pseudorandom entry from a (reasonably short) list and set it as an environmental variable: echo "main(){ arc4random();}" |gcc -x c -pipe -o arc4random -static - b1=$(sed -n '$!D;=' .list); c1=$((b1/255)); arc4random; d1=$(($?*c1)); VARIABLE=$( sed ''"$d1"'!d;'"$d1"'q' .list ); echo $VARIABLE

acveilleux 12y ago
It's simpler to dd from /dev/[u]random however and in OpenBSD, you'll get exactly the same quality result. Most of the benefits of arc4random() are nullified by invoking gcc...
- tedunangst 12y ago
  On OpenBSD, it's even simpler to just echo $RANDOM, but that only gets you 15 bits because sh. sysctl -n kern.arandom can be used in a pinch, too.
  - eridius 12y ago
    Theoretically couldn't you do something like echo $(( $RANDOM << 15 | $RANDOM )) to get 30 bits?
- 101914 12y ago
  OK, I feel stupid. How would I get a random number using dd and /dev/urandom? I am not sure I understand the gcc comment. Can you explain? Assuming I cannot get a random number from dd'ing /dev/urandom then I needed a utility to do it where the system only has a small base and does not have gcc. Unangst's suggestion to use sysctl is better (although it means I have to crunch the sysctl utility). I will be using that from now on. Thank you for the tip! As for $RANDOM, that may be OpenBSD-specific. For example, does NetBSD have it?
  - Sanddancer 12y ago
    $RANDOM is shell-specific. bash has $RANDOM, ksh has $RANDOM, and zsh has $RANDOM, standard sh doesn't have it.

acqq 12y ago

The two (for me) important slides: http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00016.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000... rc4random cannot fail Look at the function definitions: uint32_t arc4random(void); void arc4random_buf(void *buf, size_t nbytes); uint32_t arc4random_uniform(uint32_t upper_bound); All 3 functions guarantee a result. They will not fail, and cannot return an error. http://www.openbsd.org/papers/hackfest2014-arc4random/mgp00041.html http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000... Linux? We have encouraged Ted Ts'o to create a getentropy-like interface for Linux; getrandom() Too many options, too easy to misuse Interactions with signal handlers (EINTR) Hope applications do not directly call getrandom()... Bit worrying Really feels like there is pressure against easy-access random.. "Use /dev/random" meme continues damaging effects Linux's getrandom really has a bunch of error codes: http://lwn.net/Articles/606202/ http://lwn.net/Articles/606202/ EINVAL An invalid flag was passed to getrandom(2) EFAULT buf is outside the accessible address space. EAGAIN The requested entropy was not available, and the getentropy(2) would have blocked if GRND_BLOCK flag was set. EINTR While blocked waiting for entropy, the call was interrupted by a signal handler; see the description of how interrupted read(2) calls on "slow" devices are handled with and without the SA_RESTART flag in the signal(7) man page. And a few flags: GRND_NONBLOCK Don't block and return EAGAIN instead GRND_RANDOM Use the /dev/random pool instead of /dev/urandom I agree that the best interface is the one that can't fail.

shawn-butler 12y ago
arc4random has a larger range as well up to 0x100000000, whereas rand/random top out at RAND_MAX of 0x7fffffff?
ushi 12y ago
getrandom() is a generic interface you can use to implement getentropy(), which is used to seed arc4random(). It is not an alternative arc4random implementation... From the OpenBSD Manpage: getentropy() will succeed unless: [EFAULT] The buf parameter points to an invalid address. [EIO] Too many bytes requested, or some other fatal error occurred. See: http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2 http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/...