3 ms·

It seems there isn't even anything written about FD_CLOEXEC and its associated race conditions either, as far as I can tell. Basically it's impossible to portab

It seems there isn't even anything written about FD_CLOEXEC and its associated race conditions either, as far as I can tell. Basically it's impossible to portably spawn a subprocess in a safe manner if you don't have sufficient control over all the code running in your process, because you might duplicate file descriptors into the child that you might not have intended, and that can break things in the parent.

rwmj 4y ago
AIUI the problem there is not FD_CLOEXEC/SOCK_CLOEXEC but the possibility that some library might not be using it? (That is to say, *_CLOEXEC if used does not have race conditions) However we usually cope with that by closing all unknown/unexpected file descriptors after fork and before exec. Linux even has a system call to make that easier: https://man7.org/linux/man-pages/man2/close_range.2.html https://man7.org/linux/man-pages/man2/close_range.2.html
- dataflow 4y ago
  > AIUI the problem there is not FD_CLOEXEC/SOCK_CLOEXEC but the possibility that some library might not be using it? Not exactly. The problem is that you have to be able to set it atomically from the creation of the file descriptor. Setting it after creation is subject to a race condition where a fork occurs in the interim. There's no portable way to do that, and people often ignore O_CLOEXEC even when there's a platform-dependent way to pass it. (How often do you see dup3() called, for example? And how often do you see higher-level languages and libraries expose this and force callers to make a conscious decision?) > However we usually cope with that by closing all unknown/unexpected file descriptors after fork and before exec. You can't really do that portably (well, maybe unless you want to call close() billions of times). And even if/when you can do that, you run into the reverse problem, where you might close descriptors that were supposed to be duplicated into the subprocess but that you didn't know about. (One example is when a user performs redirect inside a shell like 2>&3 and wants it to work inside a descendant process - you don't want to just randomly close FDs you don't recognize.)
  - mike_hock 4y ago
    TFA talks specifically about Linux (despite the title). AFAIK most if not all Linux APIs provide a way to set CLOEXEC on creation (as in, there is at least one alternative that does). So that's a solution, but of course, every library must adhere to it. Or you have to accept that when spawning a new process, you have to know which FDs you want to leak. But doing it portably? Lol yeah, forget it. Posix is a farce.
    - wahern 4y ago
      > Posix is a farce. accept4, dup3, pipe2, F_DUPFD_CLOEXEC, MSG_CMSG_CLOEXEC, and SOCK_CLOEXEC are already in the draft for POSIX Issue 8. Most of these have already been in the BSDs and Solaris for a few years, now, at least. macOS is the major exception. Also in the draft are close-on-fork analogs: F_DUPFD_CLOFORK, FD_CLOFORK, MSG_CMSG_CLOFORK, O_CLOFORK, and SOCK_CLOFORK.
      - mike_hock 4y ago
        Cool, better late than never.
  - rwmj 4y ago
    Oh for sure (and I meant O_CLOEXEC, not FD_CLOEXEC) ... We are fairly thorough with using dup3, pipe2 etc in our own code, on platforms that support it. Of course hard to control other people's code.
jclulow 4y ago
> you might duplicate file descriptors into the child that you might not have intended, and that can break things in the parent. This is not true on all systems. For example, on ilumos we have fdwalk(3C) and in particular closefrom(3C). These allow you to dup2() your intended descriptors to a contiguous block starting at fd 3 (assuming you keep the stdio descriptors at 0-2) and then close everything from one past the last descriptor you intend the child to inherit. https://illumos.org/man/3C/closefrom https://illumos.org/man/3C/closefrom
- dataflow 4y ago
  Interesting, but how would the subprocess understand the resulting FDs? Doesn't that mean you can't really launch a subprocess that doesn't cooperate? In particular, A launches B, and B launches C, then C would have no idea what the FD numbers represent unless B somehow actively receives that information from A through some custom mechanism and communicates that information to C... right? So if B is a random general-purpose process (say, a Bash or Python process) then you're out of luck? Unless I'm missing something.
  - jclulow 4y ago
    I think there are two kinds of transitions: - on fork, you are by definition implicitly cooperating with the parent as you are the same software image as the parent; in the case of transitive forks you can be expected, I think, to continue cooperating. - on exec, you move from implicit cooperation to the need for explicit cooperation, through some kind of interface contract; e.g., the BASH_XTRACEFD environment variable. It is incumbent upon you to hide the good silverware before making the exec transition, as exec is a trust boundary; like when you sell your house, you don't get to control what the new owner of the process does with it, except through kernel-enforced mechanisms like privileges and resource controls and so on.
    - dataflow 4y ago
      I think you're introducing complexity into the discussion that isn't relevant. All I'm saying is if you do something like (rm -f /run/lock/temp && exec 5<> /run/lock/temp && echo test123 > /run/lock/temp && perl -e 'system("bash -c \"cat /dev/fd/5\"")') it needs to work fine, and this shouldn't require any particular cooperation between Perl and Bash. FDs need to be inherited correctly... just like environment variables already are.

rwmj 4y ago

AIUI the problem there is not FD_CLOEXEC/SOCK_CLOEXEC but the possibility that some library might not be using it? (That is to say, *_CLOEXEC if used does not have race conditions) However we usually cope with that by closing all unknown/unexpected file descriptors after fork and before exec. Linux even has a system call to make that easier: https://man7.org/linux/man-pages/man2/close_range.2.html https://man7.org/linux/man-pages/man2/close_range.2.html

dataflow 4y ago
> AIUI the problem there is not FD_CLOEXEC/SOCK_CLOEXEC but the possibility that some library might not be using it? Not exactly. The problem is that you have to be able to set it atomically from the creation of the file descriptor. Setting it after creation is subject to a race condition where a fork occurs in the interim. There's no portable way to do that, and people often ignore O_CLOEXEC even when there's a platform-dependent way to pass it. (How often do you see dup3() called, for example? And how often do you see higher-level languages and libraries expose this and force callers to make a conscious decision?) > However we usually cope with that by closing all unknown/unexpected file descriptors after fork and before exec. You can't really do that portably (well, maybe unless you want to call close() billions of times). And even if/when you can do that, you run into the reverse problem, where you might close descriptors that were supposed to be duplicated into the subprocess but that you didn't know about. (One example is when a user performs redirect inside a shell like 2>&3 and wants it to work inside a descendant process - you don't want to just randomly close FDs you don't recognize.)
- mike_hock 4y ago
  TFA talks specifically about Linux (despite the title). AFAIK most if not all Linux APIs provide a way to set CLOEXEC on creation (as in, there is at least one alternative that does). So that's a solution, but of course, every library must adhere to it. Or you have to accept that when spawning a new process, you have to know which FDs you want to leak. But doing it portably? Lol yeah, forget it. Posix is a farce.
  - wahern 4y ago
    > Posix is a farce. accept4, dup3, pipe2, F_DUPFD_CLOEXEC, MSG_CMSG_CLOEXEC, and SOCK_CLOEXEC are already in the draft for POSIX Issue 8. Most of these have already been in the BSDs and Solaris for a few years, now, at least. macOS is the major exception. Also in the draft are close-on-fork analogs: F_DUPFD_CLOFORK, FD_CLOFORK, MSG_CMSG_CLOFORK, O_CLOFORK, and SOCK_CLOFORK.
    - mike_hock 4y ago
      Cool, better late than never.
- rwmj 4y ago
  Oh for sure (and I meant O_CLOEXEC, not FD_CLOEXEC) ... We are fairly thorough with using dup3, pipe2 etc in our own code, on platforms that support it. Of course hard to control other people's code.

jclulow 4y ago

> you might duplicate file descriptors into the child that you might not have intended, and that can break things in the parent. This is not true on all systems. For example, on ilumos we have fdwalk(3C) and in particular closefrom(3C). These allow you to dup2() your intended descriptors to a contiguous block starting at fd 3 (assuming you keep the stdio descriptors at 0-2) and then close everything from one past the last descriptor you intend the child to inherit. https://illumos.org/man/3C/closefrom https://illumos.org/man/3C/closefrom

dataflow 4y ago
Interesting, but how would the subprocess understand the resulting FDs? Doesn't that mean you can't really launch a subprocess that doesn't cooperate? In particular, A launches B, and B launches C, then C would have no idea what the FD numbers represent unless B somehow actively receives that information from A through some custom mechanism and communicates that information to C... right? So if B is a random general-purpose process (say, a Bash or Python process) then you're out of luck? Unless I'm missing something.
- jclulow 4y ago
  I think there are two kinds of transitions: - on fork, you are by definition implicitly cooperating with the parent as you are the same software image as the parent; in the case of transitive forks you can be expected, I think, to continue cooperating. - on exec, you move from implicit cooperation to the need for explicit cooperation, through some kind of interface contract; e.g., the BASH_XTRACEFD environment variable. It is incumbent upon you to hide the good silverware before making the exec transition, as exec is a trust boundary; like when you sell your house, you don't get to control what the new owner of the process does with it, except through kernel-enforced mechanisms like privileges and resource controls and so on.
  - dataflow 4y ago
    I think you're introducing complexity into the discussion that isn't relevant. All I'm saying is if you do something like (rm -f /run/lock/temp && exec 5<> /run/lock/temp && echo test123 > /run/lock/temp && perl -e 'system("bash -c \"cat /dev/fd/5\"")') it needs to work fine, and this shouldn't require any particular cooperation between Perl and Bash. FDs need to be inherited correctly... just like environment variables already are.