3 ms·

In Ninja, which needs to spawn a lot of subprocesses but it otherwise not especially large in memory and which doesn't use threads, we moved from fork to posix_

by evmar 5y ago

In Ninja, which needs to spawn a lot of subprocesses but it otherwise not especially large in memory and which doesn't use threads, we moved from fork to posix_spawn (which is the "I want fork+exec immediately, please do the smartest thing you can" wrapper) because it performed better on OS X and Solaris: https://github.com/ninja-build/ninja/commit/89587196705f54afb904c8f4572e65de7274dd81 https://github.com/ninja-build/ninja/commit/89587196705f54af...

ridiculous_fish 5y ago
posix_spawn also outperforms fork on Linux under more recent glibc and musl, which can use vfork under the hood. https://twitter.com/ridiculous_fish/status/1232889390763933698 https://twitter.com/ridiculous_fish/status/12328893907639336...
xroche 5y ago
The issue with posix_spawn is that you can't close all descriptors before exec. This is especially an issue as most libraries are still unaware they need to open every single handle with the close-on-exec flag set.
- cryptonector 5y ago
  Solaris/Illumos has an extension[0] for that. [0] http://src.illumos.org/source/search?project=illumos-gate&full=posix_spawn_file_actions_addclosefrom_np&defs=&refs=&path=&hist=&type=&xrd=&nn=1 [1] https://docs.oracle.com/cd/E36784_01/html/E36874/posix-spawn-file-actions-addclosefrom-np-3c.html
  - kazinator 5y ago
    For implementations which don't have it, you can stuff, into the file_actions, say, 4093 close action entries into the file_actions, targeting descriptors 3 to 4095. This big file_actions object can be cached and re-used for multiple calls to posix_spawn. It won't close descriptor 4096, but that's probably beyond giving a darn in most cases. If you have an application that opens high descriptor numbers, you probably know.
    - cryptonector 5y ago
      A better approach is to exec an intermediate helper program that will do it and then exec the actual intended program. One can also use this approach to do things like reset signal dispositions to SIG_IGN.
      - kazinator 5y ago
        ... add another option to /usr/bin/env and you got it!
- kazinator 5y ago
  Closing all descriptors is next to useless; you usually need to inherit at least standard in/out/error. What you need is an operation like "close all descriptors >= N", as posix_spawn opcode.
  - cryptonector 5y ago
    Indeed, it's very common to want to close all FDs other than 0, 1, and 2, of course, as well as a few other exceptions (e.g., a pipe a parent might read from, FDs on which flocks are held). The reason one often wants to close all open FDs besides those is simple: too many FDs that should be made O_CLOEXEC often aren't, and even when they are, too often there is a race to use fcntl() to do so on one thread while another one forks. Yes, there are new system calls that allow race-free setting of O_CLOEXEC on new FDs, but they will take a long time to be widely used. I've implemented closefrom() type APIs more than once. Of course, I happen to know about Illumos', so there's that.

by evmar 5y ago

ridiculous_fish 5y ago

posix_spawn also outperforms fork on Linux under more recent glibc and musl, which can use vfork under the hood. https://twitter.com/ridiculous_fish/status/1232889390763933698 https://twitter.com/ridiculous_fish/status/12328893907639336...

xroche 5y ago

The issue with posix_spawn is that you can't close all descriptors before exec. This is especially an issue as most libraries are still unaware they need to open every single handle with the close-on-exec flag set.

cryptonector 5y ago
Solaris/Illumos has an extension[0] for that. [0] http://src.illumos.org/source/search?project=illumos-gate&full=posix_spawn_file_actions_addclosefrom_np&defs=&refs=&path=&hist=&type=&xrd=&nn=1 [1] https://docs.oracle.com/cd/E36784_01/html/E36874/posix-spawn-file-actions-addclosefrom-np-3c.html
- kazinator 5y ago
  For implementations which don't have it, you can stuff, into the file_actions, say, 4093 close action entries into the file_actions, targeting descriptors 3 to 4095. This big file_actions object can be cached and re-used for multiple calls to posix_spawn. It won't close descriptor 4096, but that's probably beyond giving a darn in most cases. If you have an application that opens high descriptor numbers, you probably know.
  - cryptonector 5y ago
    A better approach is to exec an intermediate helper program that will do it and then exec the actual intended program. One can also use this approach to do things like reset signal dispositions to SIG_IGN.
    - kazinator 5y ago
      ... add another option to /usr/bin/env and you got it!
kazinator 5y ago
Closing all descriptors is next to useless; you usually need to inherit at least standard in/out/error. What you need is an operation like "close all descriptors >= N", as posix_spawn opcode.
- cryptonector 5y ago
  Indeed, it's very common to want to close all FDs other than 0, 1, and 2, of course, as well as a few other exceptions (e.g., a pipe a parent might read from, FDs on which flocks are held). The reason one often wants to close all open FDs besides those is simple: too many FDs that should be made O_CLOEXEC often aren't, and even when they are, too often there is a race to use fcntl() to do so on one thread while another one forks. Yes, there are new system calls that allow race-free setting of O_CLOEXEC on new FDs, but they will take a long time to be widely used. I've implemented closefrom() type APIs more than once. Of course, I happen to know about Illumos', so there's that.