4 ms·

Interesting discussion. If these were user-space threads, like FreeBSD ~20 years ago, there'd be no problem. When fork() is called, the whole user-space threa

Interesting discussion. If these were user-space threads, like FreeBSD ~20 years ago, there'd be no problem. When fork() is called, the whole user-space threads package would be forked, along with all the threads. So the obvious question is whether it's fundamental that with kernel threads the fork() system call doesn't clone all the other threads in the process? Yes, that's not how it's done, but could Apple choose to implement a new fork_all() system call? I imagine it wouldn't be easy - you'd need to pause all the running threads while you copied state, but is there a reason it's actually not possible?

mhandley 9y ago
Thinking about it some more, one problem would be what to do with threads that have just called a system call and are waiting for a response. To have both parent and child process system calls complete without errors would require cloning the kernel state, and that's not trivial. It some cases, there's probably no good answer, short of causing one of the two system calls to return an error, and that in itself would confuse a lot of user-space software. I suppose you could return something similar to EAGAIN, but you'd be dependent on user-space handling that cleanly. Maybe you could hide this under libc?
- wruza 9y ago
  EINTR? I suspect some calls are defined to never return it, but it may be just de facto, not strict rule. Edit: simply signalling all threads with SIGFORK, waiting for a handler to return and freezing a thread at SA_RESTART point feels right way to do it. Why did POSIX decide to fork() only one thread by default at all?
  - crest 9y ago
    Because the alternatives are worse.
- ScottBurson 9y ago
  So this is a worse-is-better design decision driven by a previous worse-is-better design decision. If system calls were interruptible, then the problem of threads in system calls would go away.
geocar 9y ago
Is this what you want most of the time? If you're just going fork() && exec() then why would you copy all that state just to run some subprogram? Is that what you want any of the time? This prefork implementation is silly. Either do prefork after initialisation so you can take the benefits of COW, or don't bother: Just use SO_REUSEPORT and run 10 copies of your server. This distributes the TCP traffic and provides you an excellent way to upgrade-in-place (just roll up the new version as you roll-down old ones).
- mhandley 9y ago
  I agree - most of the time, the current fork() implementation is what you want, so it's the right choice. The conventional wisdom is that you shouldn't spawn threads and then fork(), and so long as you stick to this, you're fine. Today though, everyone loves threads. It's hard to know precisely what happens in some low-level library, so it's hard to be sure that you don't have any threads left running after initialization. This is the subject of the OP. My question though is broader - if we chose, could we add a version of fork() that did clone all the threads? I'm not entirely sure what it would be used for, but I'm sure there would be uses. Likely some of those would be for increased security, as processes provide stronger isolation.
  - geocar 9y ago
    > I'm not entirely sure what it would be used for, but I'm sure there would be uses. I suspect strongly there aren't. One of the biggest problems you have to contend with is around mutexes (and the like) and their state. If you copy them, we double-book an external-resource (like storage or something), and if you don't we almost certainly deadlock. Another is any thread waiting on a resource (like reading a file descriptor) or writing to the disk (do you write twice?) and so on.
- gcb0 9y ago
  because that what everyone that uses fork want. memory access. otherwise they would use posix_spawn or something
4ad 9y ago
Solaris has forkall(2) and forkallx(2) system calls.

mhandley 9y ago

Thinking about it some more, one problem would be what to do with threads that have just called a system call and are waiting for a response. To have both parent and child process system calls complete without errors would require cloning the kernel state, and that's not trivial. It some cases, there's probably no good answer, short of causing one of the two system calls to return an error, and that in itself would confuse a lot of user-space software. I suppose you could return something similar to EAGAIN, but you'd be dependent on user-space handling that cleanly. Maybe you could hide this under libc?

wruza 9y ago
EINTR? I suspect some calls are defined to never return it, but it may be just de facto, not strict rule. Edit: simply signalling all threads with SIGFORK, waiting for a handler to return and freezing a thread at SA_RESTART point feels right way to do it. Why did POSIX decide to fork() only one thread by default at all?
- crest 9y ago
  Because the alternatives are worse.
ScottBurson 9y ago
So this is a worse-is-better design decision driven by a previous worse-is-better design decision. If system calls were interruptible, then the problem of threads in system calls would go away.

geocar 9y ago

Is this what you want most of the time? If you're just going fork() && exec() then why would you copy all that state just to run some subprogram? Is that what you want any of the time? This prefork implementation is silly. Either do prefork after initialisation so you can take the benefits of COW, or don't bother: Just use SO_REUSEPORT and run 10 copies of your server. This distributes the TCP traffic and provides you an excellent way to upgrade-in-place (just roll up the new version as you roll-down old ones).

mhandley 9y ago
I agree - most of the time, the current fork() implementation is what you want, so it's the right choice. The conventional wisdom is that you shouldn't spawn threads and then fork(), and so long as you stick to this, you're fine. Today though, everyone loves threads. It's hard to know precisely what happens in some low-level library, so it's hard to be sure that you don't have any threads left running after initialization. This is the subject of the OP. My question though is broader - if we chose, could we add a version of fork() that did clone all the threads? I'm not entirely sure what it would be used for, but I'm sure there would be uses. Likely some of those would be for increased security, as processes provide stronger isolation.
- geocar 9y ago
  > I'm not entirely sure what it would be used for, but I'm sure there would be uses. I suspect strongly there aren't. One of the biggest problems you have to contend with is around mutexes (and the like) and their state. If you copy them, we double-book an external-resource (like storage or something), and if you don't we almost certainly deadlock. Another is any thread waiting on a resource (like reading a file descriptor) or writing to the disk (do you write twice?) and so on.
gcb0 9y ago
because that what everyone that uses fork want. memory access. otherwise they would use posix_spawn or something

4ad 9y ago

Solaris has forkall(2) and forkallx(2) system calls.