4 ms·

The code is more complicated than it needs to be. It spawns N threads, then each thread forkexecs child processes. These threads communicate through an atomic i

The code is more complicated than it needs to be. It spawns N threads, then each thread forkexecs child processes. These threads communicate through an atomic int. There is no need for threads. Just spawn background processes: echo 1 & echo 2 & wait echo 3 & wait echo 4 & ... The key is that the wait() system call will hang until any child process finishes.

the_mitsuhiko 10y ago
I doubt that will make it measurable more efficient. Threads are cheap.
- bmalehorn 10y ago
  My point wasn't optimization. Just that the author was making it more complicated than was needed. It's like discovering that "ls" is spawning 5 threads for internal communication.
  - the_mitsuhiko 10y ago
    In a language with good threading support, internal threads are typically making things easier rather than harder. I tend to spawn lots of threads in Rust just because I can and it simplifies the code a lot over having to do some async callback mess. In particular there is no sane way to async waitpid() on POSIX.
    - geofft 10y ago
      pselect / self-pipe / etc. on SIGCHLD, and on receipt of SIGCHLD, loop through all known child processes with wait(WNOHANG). ... I'm not sure I can disagree with "no sane way".
      - the_mitsuhiko 10y ago
        The only thing you can do from a signal handler is flipping a static and only one signal handler can be set. Good luck making this reusable. In fact, I challenge you to solve the problem "spawn process; waitpid for 15 seconds; otherwise kill hard" in Rust (or C++ if you feel like) on POSIX once with threads and once without threads by sticking to what's permitted in the standard and so that multiple processes can be waited for. Then also measure CPU impact :)
        gpderetta 10y ago
        > The only thing you can do from a signal handler is flipping a static and only one signal handler can be set. This is false, you can call any async signal safe function. Incidentally write is one of them. Another trick is the close-on-exit pipe.
        geofft 10y ago
        "Async-signal-safe" is a C concept (from the POSIX world where C is your interface to the system, and library calls vs. system calls are behind the abstraction layer), so it doesn't directly apply to Rust. But the underlying semantics of signals are simple to describe: you get interrupted at some instruction pointer and jump into a new function. You can do whatever you want provided you uphold safety, correctness, liveness, etc. If you change a variable, it has to be one that isn't prone to being cached in a register or the stack by the main program. POSIX's sig_atomic_t does this; in Rust you can use the normal atomic types. They are a tiny bit too careful if this is thread-local, but an ordinary thread-local variable is permitted to be cached within the same thread, and signal handlers break that. If you take a lock, you have to do something reasonable if the lock is already held, including by the code you interrupted. So you probably shouldn't lock at all. The biggest reason for a POSIX function not to be async-signal-safe is because it wants to call malloc, which takes out a lock (at least a per-thread or per-CPU lock) on the heap. If you get signaled during a malloc, and the signal handler tries to malloc, you deadlock. But anything that does not risk liveness or correctness problems is fair game. In particular, basically all system calls are fair game, since they're just sending a message to the kernel. C's fprintf() will want to buffer in userspace, which involves an allocation, but write() will at most buffer in the kernel, and the kernel-side code doesn't have the problem of having flow control interrupted while you're in a signal handler. Even if you were previously in a blocking write() when you received a signal, the kernel will return from its implementation of write before delivering the signal back to userspace, so there isn't a re-entrant call to the kernel-side write code. libc's fprintf() doesn't have that luxury. (And yes, the concept of Rust on POSIX is a bit ill-defined, because POSIX is a set of C-language APIs, which can be implemented in any valid way in C, including header macros. Rust threads use pthreads, yes, but inter-thread communication doesn't involve whatever sig_atomic_t is typedef'd or #defined to.)

the_mitsuhiko 10y ago

I doubt that will make it measurable more efficient. Threads are cheap.

bmalehorn 10y ago
My point wasn't optimization. Just that the author was making it more complicated than was needed. It's like discovering that "ls" is spawning 5 threads for internal communication.
- the_mitsuhiko 10y ago
  In a language with good threading support, internal threads are typically making things easier rather than harder. I tend to spawn lots of threads in Rust just because I can and it simplifies the code a lot over having to do some async callback mess. In particular there is no sane way to async waitpid() on POSIX.
  - geofft 10y ago
    pselect / self-pipe / etc. on SIGCHLD, and on receipt of SIGCHLD, loop through all known child processes with wait(WNOHANG). ... I'm not sure I can disagree with "no sane way".
    - the_mitsuhiko 10y ago
      The only thing you can do from a signal handler is flipping a static and only one signal handler can be set. Good luck making this reusable. In fact, I challenge you to solve the problem "spawn process; waitpid for 15 seconds; otherwise kill hard" in Rust (or C++ if you feel like) on POSIX once with threads and once without threads by sticking to what's permitted in the standard and so that multiple processes can be waited for. Then also measure CPU impact :)
      - gpderetta 10y ago
        > The only thing you can do from a signal handler is flipping a static and only one signal handler can be set. This is false, you can call any async signal safe function. Incidentally write is one of them. Another trick is the close-on-exit pipe.
        geofft 10y ago
        "Async-signal-safe" is a C concept (from the POSIX world where C is your interface to the system, and library calls vs. system calls are behind the abstraction layer), so it doesn't directly apply to Rust. But the underlying semantics of signals are simple to describe: you get interrupted at some instruction pointer and jump into a new function. You can do whatever you want provided you uphold safety, correctness, liveness, etc. If you change a variable, it has to be one that isn't prone to being cached in a register or the stack by the main program. POSIX's sig_atomic_t does this; in Rust you can use the normal atomic types. They are a tiny bit too careful if this is thread-local, but an ordinary thread-local variable is permitted to be cached within the same thread, and signal handlers break that. If you take a lock, you have to do something reasonable if the lock is already held, including by the code you interrupted. So you probably shouldn't lock at all. The biggest reason for a POSIX function not to be async-signal-safe is because it wants to call malloc, which takes out a lock (at least a per-thread or per-CPU lock) on the heap. If you get signaled during a malloc, and the signal handler tries to malloc, you deadlock. But anything that does not risk liveness or correctness problems is fair game. In particular, basically all system calls are fair game, since they're just sending a message to the kernel. C's fprintf() will want to buffer in userspace, which involves an allocation, but write() will at most buffer in the kernel, and the kernel-side code doesn't have the problem of having flow control interrupted while you're in a signal handler. Even if you were previously in a blocking write() when you received a signal, the kernel will return from its implementation of write before delivering the signal back to userspace, so there isn't a re-entrant call to the kernel-side write code. libc's fprintf() doesn't have that luxury. (And yes, the concept of Rust on POSIX is a bit ill-defined, because POSIX is a set of C-language APIs, which can be implemented in any valid way in C, including header macros. Rust threads use pthreads, yes, but inter-thread communication doesn't involve whatever sig_atomic_t is typedef'd or #defined to.)