8 ms·

Why threads can't fork

147 points by akerl_ 12y ago

djcapelis 12y ago
Really? Writing a wrapper around fork with a mutex doesn't seem like that huge of an issue. This combined with pthread_atfork() should provide a way to make this work, no?
- asveikau 12y ago
  A mutex around fork() buys you nothing. The problem is other locks, which often are not even part of your code. The article has perhaps the most likely example. Imagine malloc() has a lock. Another thread happens to be inside malloc() at the moment that you fork(), and therefore owns the lock and might be in the middle of manipulating shared data structures. Now suddenly the child cannot malloc(), because that thread [suspended in the middle of its execution] isn't going to be carried over to it, and will never be able to clean up its intermediate state and release the lock.
  - djcapelis 12y ago
    It buys you plenty if you prevent other threads from entering until you all meet in a rendezvous lock.
    - asveikau 12y ago
      The worker thread that was kicked off by some random library dependency (acquiring a lock you didn't know about) isn't going to care about your rendezvous lock. Even if you do control all the threads getting them to do what you're suggesting may be nontrivial and costly. Edit: IMO it's better to just admit the programming model is thorny and move on. I feel similarly about signals. Restrict what you do after fork() and generally be cautious, the same way you'd be cautious reacting to a signal.
      - djcapelis 12y ago
        Wrap the thread create calls and make them care. :)
- pjmlp 12y ago
  Threads don't work the same way across all UNIX systems, there are little differences on how signals are handled, also POSIX functions tend to have implementation specific semantics.
  - djcapelis 12y ago
    Yes cross platform is hard. You can still use whatever primitives are available to cobble this together. pthreads support or equiv isn't exactly crazy to expect. Otherwise you wouldn't offer locks in languages either! This is a feature that can be implemented.
    - pjmlp 12y ago
      Sometimes those primitives have so different set of corner cases and implementation specific semantics, that they could just have different names across "compatible" OS. I used to do cross platform across Aix, HP-UX, Solaris, GNU/Linux, FreeBSD and Windows NT/2000 back in the .COM days.
      - djcapelis 12y ago
        Yes of course. When you code the implementation for each platform you always have to keep that in mind. Same is true for locks or exposing any low level systems features. See also: high performance event handling, etc. But does anyone suggest you just can't ever use lots of sockets? (It is a bad idea sometimes, to be fair!) We write the code necessary to do it. (Since we're apparently going into personal history, I used to be a platform manager for a ~million line cross platform codebase that ran on most of the things in your list plus IRIX and OS X. Most of my responsibilities were really about keeping the build environment working enough to run one binary across all the Linux distributions, but I was also involved in issues that specific to my platform. That is one of many projects where I've dealt with platform specific issues on a variety of codebases, several of which I've written in their entirety. Anyway. I have a passing familiarity with some of these issues.)
        12y ago
Dylan16807 12y ago
That's quite a copout with forkall. Presumably you have some idea what your threads are doing if you want to fork the entire process, so don't do it in the middle of writing to a file.
- swartkrans 12y ago
  How could you know what your threads are doing at the time you call fork? That doesn't sound safe. I write such code in a way that threads communicate with each other when they need to, but if I start assuming what those threads are doing that seems like it could get really complicated really fast.
  - thetrb 12y ago
    Sure it can get complicated, but there are also some safe ways to fork. Imagine you fork at the very beginning of your program before starting any threads. No danger in that. So the language could at least allow it (but provide the proper warnings around it).
    - Vendan 12y ago
      the issue arises when the programming language doesn't let you run code before the threads split out, like golang.
  - Dylan16807 12y ago
    I don't know exactly what they're doing, but being able to reason that they are not performing file IO is a vastly easier problem than figuring out a completely safe point to kill them all.
- dietrichepp 12y ago
  How many libraries do you use? Are you sure they don't create their own threads? Also, just imagine something like this: file.write(blah); if (has_forked) thread_exit(); file.write(blah2); if (has_forked) thread_exit(); file.write(blah3) if (has_forked) thread_exit(); Note the race conditions in the above code... forkall() is a disaster. Presumably you'd combine it with pthread_atfork(), but it's still a total mess.
aidenn0 12y ago
This is an issue in common lisp as well; you generally have the choice of either fork or threads. In fact ClozureCL, a popular lisp implementation launches an extra thread at startup (For I/O IIRC), and for a while someone maintained a fork of it that did not do so to allow usage of the fork syscall.
- 616c 12y ago
  Interesting. I noticed that with RESTAS (a CL web server/API server library) they make a big deal of daemonization and not requiring the usual hacks to daemonize a CL server (tmux/screen/dtach/etc). Was this a common (no pun intended) problem among CL implementations and why server daemonization is an issue? I am just learning CL, and noticed RESTAS only really supports this daemonization feature in SBCL, if I believe the documentation. I guess I am going to have to dive into the source this weekend and check it out.
  - aidenn0 12y ago
    I use daemontools to monitor my daemons, so I've never tried to daemonize a lisp daemon. With SBCL you can still fork so long as you do it before you spawn any threads. I'm guessing that RESTAS uses the sb-posix:fork function to daemonize, which would explain why it only works on sbcl.
    - 616c 12y ago
      Neat. Thanks for the explanation. I had installed RESTAS inside of a Clozure CL image but had not gotten as far as daemonization and forking. Judging from the one sentence on the landing page, I supposed it would blow up. Haha.
Animats 12y ago
Sun did try multi-thread fork semantics, but that didn't help. UNIX "fork" started as a hack. The reason UNIX originally used a fork/exec approach to program launch was to conserve memory on the PDP-11. "fork" originally worked by swapping the process out to disk. Then, at the moment there was a good copy in both memory and on disk, the process table entry was duplicated, with one copy pointing to the swapped-out process image and one pointed to the in-memory copy. The regular swapping system took it from there. Then, as machines got bigger, the Berkeley BSD crowd, rather than simply introducing a "run" primitive, hacked on a number of variants of "fork" to make program launch more efficient. That's what we mostly have today. Plan 9 supported more variants in a more rational way; you could choose whether to share or copy code, data, stack, and I think file opens. The Plan 9 paper says that most of the variants proved useful. But that approach ended with Plan 9. UCLA Locus just had a "run" primitive; Locus ran on multiple machines without shared memory, so "fork" wasn't too helpful there. Threads came long after "fork" in the UNIX world. (Threads originated with UNIVAC 1108 Exec 8 (called OS 2200 today), first released in 1967, where they were called "activities"). Exec 8 had "run", and an activity could fork off another activity, but not process-level fork. Activities were available both inside and outside the OS kernel, decades before UNIX. That's why UNIX thread semantics have always been troublesome, especially where signals are involved. They were added on, not designed in.
- gonzo 12y ago
  "fork()" didn't start as a hack. There is no documentation to suggest that one return from fork() was a copy now on-disk while the other remained in ram. http://cm.bell-labs.com/cm/cs/who/dmr/man21.pdf http://cm.bell-labs.com/cm/cs/who/dmr/man21.pdf http://www3.alcatel-lucent.com/bstj/vol57-1978/articles/bstj57-6-1905.pdf http://www3.alcatel-lucent.com/bstj/vol57-1978/articles/bstj...
  - Animats 12y ago
    Yes, there is. John Lion's "A commentary on the Sixth Edition UNIX Operating System", goes through the PDP-11 UNIX kernel line by line. http://www.lemis.com/grog/Documentation/Lions/book.pdf http://www.lemis.com/grog/Documentation/Lions/book.pdf Page 37, Section 7.13, "newproc": 1906: Call "xswap" (4368) to copy the data segments into the disk swap area. Because the second parameter is zero, the main memory area will not be released. 1907: Mark the new process as "swapped out". 1908: Return the current process to its normal state.
lmm 12y ago
fork() is a dumb interface, and non-portable anyway. I've yet to see a use case that couldn't be handled with either threads, or spawning another process - after all, those are the only APIs you get elsewhere. If you need to use a language with a runtime (not just Go by any means, the likes of Python also suffer from this issue) from two processes that need to be separate but communicate with each other, do the fork first, then start the language runtime (i.e. embed the language in your parent program).
- ridiculous_fish 12y ago
  "Spawning another process" is a very complicated task. There's tons of process state that you need to set before the child starts: process group, file descriptors, signal mask, foreground status, etc. You can try to bundle it all up into a single API like posix_spawn, but the API becomes large and it's hard to cover everything. posix_spawn sure doesn't. fork is an elegant solution to this problem: you can run code as the child, before the child gets to start. I am not aware of any alternative that's as flexible.
- agwa 12y ago
  > I've yet to see a use case that couldn't be handled with either threads, or spawning another process fork() allows you to set up file descriptors such as stdin and stdout before execing another process. This is essential for pipelines.
- mdwrigh2 12y ago
  > I've yet to see a use case that couldn't be handled with either threads, or spawning another process - after all, those are the only APIs you get elsewhere. Android uses this to pre-load framework resources and code in a way that lets all applications share the backing memory. And when applications crash, they don't bring down the initial process that preloaded everything (so it can continue spawning new apps). How would you handle that with only threads or spawning another process?
  - ANTSANTS 12y ago
    Uh, dynamic linking?
rubiquity 12y ago
You give people an amazing implementation of M:N concurrency and the people want to fork. You give people concurrency via forking, and they want N:M concurrency.
- qznc 12y ago
  In other words: Whenever people get a nice tool for something, they want to use the tool for other things as well.
- wahern 12y ago
  Did Go ever fix M:N threading? From what I've read there's significant performance degradation when you use M:N threading as opposed to 1:N. If Go figured out how to efficiently detect data dependencies and relationships, and then automatically move goroutines around, that would be exceptionally noteworthy. Everybody starts out thinking they can do this, which is why Solaris, NetBSD, Linux, Java, et al all started with out M:N threading. But then when they figure out that it's a Really Hard(tm) problem, they invariably shift to 1:1. I've found that it's better to leave it to the developer to choose whether to run an OS thread or coroutine, just as the developer chooses between a process and OS thread. So in my project[1] I don't spend much time trying to automate that. [1] http://25thandclement.com/~william/projects/cqueues.html http://25thandclement.com/~william/projects/cqueues.html
  - justincormack 12y ago
    Google are working on some kernel help for userspace threads there was an article on lwn.net a while ago. Someone told me yesterday he was seeing a lot of spurious wakeups but hadn't debugged them yet. So I think there is room for improvement.
ridiculous_fish 12y ago
The fish shell http://fishshell.com http://fishshell.com is multithreaded and calls fork, so it can be done. But it is difficult, even if you just call execve(). An example of a problem we encountered: what if execve() fails? Then we want to print an error message, based on the value of errno. perror() is the preferred way to do this. But on Linux, perror() calls into gettext to show a localized message, and gettext takes a lock, and then you deadlock. This is a hard problem to solve because it requires knowing the status of all locks. This breaks the abstractions that library authors present, where locks are internal and not exposed.
- ben0x539 12y ago
  Isn't it easier to have the parent print the error message by passing yet another pipe to the child that closes on exec and can be used to pass up errno?
  - ridiculous_fish 12y ago
    That's a good idea! I'm not sure if it's easier but it would effectively sidestep this class of issues.
  - bdonlan 12y ago
    You could also use an inherited shared memory segment (an anonymous mmap with MAP_SHARED) to report failure codes. Since you only need to set this up once, it could result in lower overhead than creating pipes for every exec.
preillyme 12y ago
The pthread_atfork() function shall declare fork handlers to be called before and after fork(), in the context of the thread that called fork(). The prepare fork handler shall be called before fork() processing commences. The parent fork handle shall be called after fork() processing completes in the parent process. The child fork handler shall be called after fork() processing completes in the child process. If no handling is desired at one or more of these three points, the corresponding fork handler address(es) may be set to NULL. The order of calls to pthread_atfork() is significant. The parent and child fork handlers shall be called in the order in which they were established by calls to pthread_atfork(). The prepare fork handlers shall be called in the opposite order. I'm not sure if that's the best approach, but it's an attempt at least.
- pjmlp 12y ago
  I just had a look at IEEE Std 1003.1, 2013 Edition for pthread_atfork(). It has a few corner cases across POSIX systems. I wouldn't bet it works 100% the same way in all UNIXes.
  - djcapelis 12y ago
    Of course it doesn't, very few things do. Cross platform is hard, but that doesn't mean you don't use features.
    - pjmlp 12y ago
      Of course, but sometimes it is a huge pain. I used to do cross platform across Aix, HP-UX, Solaris, GNU/Linux, FreeBSD and Windows NT/2000 back in the .COM days.
      - wahern 12y ago
        It's much easier in 2014 than it was in 2004. POSIX has evolved, and POSIX conformance has substantially improved. Most systems are, in practice, nearly 100% conformant to POSIX-2001. Excepting Windows, I rarely run into difficult portability problems except when I deliberately use non-POSIX functionality or newer POSIX functionality. I target Linux, OS X, OpenBSD, NetBSD, FreeBSD, Solaris, and AIX. The biggest laggard was OpenBSD, particularly wrt to threading, signal handling, and real-time extensions. But in the past couple of years that's been substantially addressed. One of my biggest headaches now is OS X. They appear to have stopped trying to track POSIX, so while everybody else is busily implementing POSIX-2004, POSIX-2008, and tentative POSIX features, OS X is nearly at a stand-still. OS X hasn't fixed any significant conformance issues, adopted real-time extensions, nor adopted any POSIX-2008 features for several years, now.
ChuckMcM 12y ago
In my opinion this is fundamentally a problem of mixing coprocessing metaphors. The whole thread vs process vs container differences are tied up in leakage between the permissions model (process) and computation model (threads and containers). The thread equivalent of 'fork' would be some form of promotion to a 'root' thread (which is to say one where instance data about computation limits can be changed independently) Processes, which were the traditional collection point of resources under an identity, ideally sit apart from computation constraints. And if you follow that path you realize that resource allocation (which is one of the three key parts of OS management) then need to both be computation aware, and identity aware. In the example of the article, malloc would break the lock such that its validity would be related to the identity it was associated with, so if you promoted a thread to the 'identity' level you would invalidate any locks visible to it that were attached to identity. There have been discussions about this in OS design for almost forever.
deleted 12y ago
[deleted]
robmccoll 12y ago
In the motivating Go example, implementing a fork doesn't seem like it would be particularly challenging since the runtime can pause all executing Go routines and reschedule them in the same place on the other side. Bit easier than pthreads.

147 points by akerl_ 12y ago

Really? Writing a wrapper around fork with a mutex doesn't seem like that huge of an issue. This combined with pthread_atfork() should provide a way to make this work, no?

asveikau 12y ago
A mutex around fork() buys you nothing. The problem is other locks, which often are not even part of your code. The article has perhaps the most likely example. Imagine malloc() has a lock. Another thread happens to be inside malloc() at the moment that you fork(), and therefore owns the lock and might be in the middle of manipulating shared data structures. Now suddenly the child cannot malloc(), because that thread [suspended in the middle of its execution] isn't going to be carried over to it, and will never be able to clean up its intermediate state and release the lock.
- djcapelis 12y ago
  It buys you plenty if you prevent other threads from entering until you all meet in a rendezvous lock.
  - asveikau 12y ago
    The worker thread that was kicked off by some random library dependency (acquiring a lock you didn't know about) isn't going to care about your rendezvous lock. Even if you do control all the threads getting them to do what you're suggesting may be nontrivial and costly. Edit: IMO it's better to just admit the programming model is thorny and move on. I feel similarly about signals. Restrict what you do after fork() and generally be cautious, the same way you'd be cautious reacting to a signal.
    - djcapelis 12y ago
      Wrap the thread create calls and make them care. :)
pjmlp 12y ago
Threads don't work the same way across all UNIX systems, there are little differences on how signals are handled, also POSIX functions tend to have implementation specific semantics.
- djcapelis 12y ago
  Yes cross platform is hard. You can still use whatever primitives are available to cobble this together. pthreads support or equiv isn't exactly crazy to expect. Otherwise you wouldn't offer locks in languages either! This is a feature that can be implemented.
  - pjmlp 12y ago
    Sometimes those primitives have so different set of corner cases and implementation specific semantics, that they could just have different names across "compatible" OS. I used to do cross platform across Aix, HP-UX, Solaris, GNU/Linux, FreeBSD and Windows NT/2000 back in the .COM days.
    - djcapelis 12y ago
      Yes of course. When you code the implementation for each platform you always have to keep that in mind. Same is true for locks or exposing any low level systems features. See also: high performance event handling, etc. But does anyone suggest you just can't ever use lots of sockets? (It is a bad idea sometimes, to be fair!) We write the code necessary to do it. (Since we're apparently going into personal history, I used to be a platform manager for a ~million line cross platform codebase that ran on most of the things in your list plus IRIX and OS X. Most of my responsibilities were really about keeping the build environment working enough to run one binary across all the Linux distributions, but I was also involved in issues that specific to my platform. That is one of many projects where I've dealt with platform specific issues on a variety of codebases, several of which I've written in their entirety. Anyway. I have a passing familiarity with some of these issues.)
      - 12y ago

Dylan16807 12y ago

That's quite a copout with forkall. Presumably you have some idea what your threads are doing if you want to fork the entire process, so don't do it in the middle of writing to a file.

swartkrans 12y ago
How could you know what your threads are doing at the time you call fork? That doesn't sound safe. I write such code in a way that threads communicate with each other when they need to, but if I start assuming what those threads are doing that seems like it could get really complicated really fast.
- thetrb 12y ago
  Sure it can get complicated, but there are also some safe ways to fork. Imagine you fork at the very beginning of your program before starting any threads. No danger in that. So the language could at least allow it (but provide the proper warnings around it).
  - Vendan 12y ago
    the issue arises when the programming language doesn't let you run code before the threads split out, like golang.
- Dylan16807 12y ago
  I don't know exactly what they're doing, but being able to reason that they are not performing file IO is a vastly easier problem than figuring out a completely safe point to kill them all.
dietrichepp 12y ago
How many libraries do you use? Are you sure they don't create their own threads? Also, just imagine something like this: file.write(blah); if (has_forked) thread_exit(); file.write(blah2); if (has_forked) thread_exit(); file.write(blah3) if (has_forked) thread_exit(); Note the race conditions in the above code... forkall() is a disaster. Presumably you'd combine it with pthread_atfork(), but it's still a total mess.

aidenn0 12y ago

This is an issue in common lisp as well; you generally have the choice of either fork or threads. In fact ClozureCL, a popular lisp implementation launches an extra thread at startup (For I/O IIRC), and for a while someone maintained a fork of it that did not do so to allow usage of the fork syscall.

616c 12y ago
Interesting. I noticed that with RESTAS (a CL web server/API server library) they make a big deal of daemonization and not requiring the usual hacks to daemonize a CL server (tmux/screen/dtach/etc). Was this a common (no pun intended) problem among CL implementations and why server daemonization is an issue? I am just learning CL, and noticed RESTAS only really supports this daemonization feature in SBCL, if I believe the documentation. I guess I am going to have to dive into the source this weekend and check it out.
- aidenn0 12y ago
  I use daemontools to monitor my daemons, so I've never tried to daemonize a lisp daemon. With SBCL you can still fork so long as you do it before you spawn any threads. I'm guessing that RESTAS uses the sb-posix:fork function to daemonize, which would explain why it only works on sbcl.
  - 616c 12y ago
    Neat. Thanks for the explanation. I had installed RESTAS inside of a Clozure CL image but had not gotten as far as daemonization and forking. Judging from the one sentence on the landing page, I supposed it would blow up. Haha.

Animats 12y ago

Sun did try multi-thread fork semantics, but that didn't help. UNIX "fork" started as a hack. The reason UNIX originally used a fork/exec approach to program launch was to conserve memory on the PDP-11. "fork" originally worked by swapping the process out to disk. Then, at the moment there was a good copy in both memory and on disk, the process table entry was duplicated, with one copy pointing to the swapped-out process image and one pointed to the in-memory copy. The regular swapping system took it from there. Then, as machines got bigger, the Berkeley BSD crowd, rather than simply introducing a "run" primitive, hacked on a number of variants of "fork" to make program launch more efficient. That's what we mostly have today. Plan 9 supported more variants in a more rational way; you could choose whether to share or copy code, data, stack, and I think file opens. The Plan 9 paper says that most of the variants proved useful. But that approach ended with Plan 9. UCLA Locus just had a "run" primitive; Locus ran on multiple machines without shared memory, so "fork" wasn't too helpful there. Threads came long after "fork" in the UNIX world. (Threads originated with UNIVAC 1108 Exec 8 (called OS 2200 today), first released in 1967, where they were called "activities"). Exec 8 had "run", and an activity could fork off another activity, but not process-level fork. Activities were available both inside and outside the OS kernel, decades before UNIX. That's why UNIX thread semantics have always been troublesome, especially where signals are involved. They were added on, not designed in.

gonzo 12y ago
"fork()" didn't start as a hack. There is no documentation to suggest that one return from fork() was a copy now on-disk while the other remained in ram. http://cm.bell-labs.com/cm/cs/who/dmr/man21.pdf http://cm.bell-labs.com/cm/cs/who/dmr/man21.pdf http://www3.alcatel-lucent.com/bstj/vol57-1978/articles/bstj57-6-1905.pdf http://www3.alcatel-lucent.com/bstj/vol57-1978/articles/bstj...
- Animats 12y ago
  Yes, there is. John Lion's "A commentary on the Sixth Edition UNIX Operating System", goes through the PDP-11 UNIX kernel line by line. http://www.lemis.com/grog/Documentation/Lions/book.pdf http://www.lemis.com/grog/Documentation/Lions/book.pdf Page 37, Section 7.13, "newproc": 1906: Call "xswap" (4368) to copy the data segments into the disk swap area. Because the second parameter is zero, the main memory area will not be released. 1907: Mark the new process as "swapped out". 1908: Return the current process to its normal state.

lmm 12y ago

fork() is a dumb interface, and non-portable anyway. I've yet to see a use case that couldn't be handled with either threads, or spawning another process - after all, those are the only APIs you get elsewhere. If you need to use a language with a runtime (not just Go by any means, the likes of Python also suffer from this issue) from two processes that need to be separate but communicate with each other, do the fork first, then start the language runtime (i.e. embed the language in your parent program).

ridiculous_fish 12y ago
"Spawning another process" is a very complicated task. There's tons of process state that you need to set before the child starts: process group, file descriptors, signal mask, foreground status, etc. You can try to bundle it all up into a single API like posix_spawn, but the API becomes large and it's hard to cover everything. posix_spawn sure doesn't. fork is an elegant solution to this problem: you can run code as the child, before the child gets to start. I am not aware of any alternative that's as flexible.
agwa 12y ago
> I've yet to see a use case that couldn't be handled with either threads, or spawning another process fork() allows you to set up file descriptors such as stdin and stdout before execing another process. This is essential for pipelines.
mdwrigh2 12y ago
> I've yet to see a use case that couldn't be handled with either threads, or spawning another process - after all, those are the only APIs you get elsewhere. Android uses this to pre-load framework resources and code in a way that lets all applications share the backing memory. And when applications crash, they don't bring down the initial process that preloaded everything (so it can continue spawning new apps). How would you handle that with only threads or spawning another process?
- ANTSANTS 12y ago
  Uh, dynamic linking?

rubiquity 12y ago

You give people an amazing implementation of M:N concurrency and the people want to fork. You give people concurrency via forking, and they want N:M concurrency.

qznc 12y ago
In other words: Whenever people get a nice tool for something, they want to use the tool for other things as well.
wahern 12y ago
Did Go ever fix M:N threading? From what I've read there's significant performance degradation when you use M:N threading as opposed to 1:N. If Go figured out how to efficiently detect data dependencies and relationships, and then automatically move goroutines around, that would be exceptionally noteworthy. Everybody starts out thinking they can do this, which is why Solaris, NetBSD, Linux, Java, et al all started with out M:N threading. But then when they figure out that it's a Really Hard(tm) problem, they invariably shift to 1:1. I've found that it's better to leave it to the developer to choose whether to run an OS thread or coroutine, just as the developer chooses between a process and OS thread. So in my project[1] I don't spend much time trying to automate that. [1] http://25thandclement.com/~william/projects/cqueues.html http://25thandclement.com/~william/projects/cqueues.html
- justincormack 12y ago
  Google are working on some kernel help for userspace threads there was an article on lwn.net a while ago. Someone told me yesterday he was seeing a lot of spurious wakeups but hadn't debugged them yet. So I think there is room for improvement.

ridiculous_fish 12y ago

The fish shell http://fishshell.com http://fishshell.com is multithreaded and calls fork, so it can be done. But it is difficult, even if you just call execve(). An example of a problem we encountered: what if execve() fails? Then we want to print an error message, based on the value of errno. perror() is the preferred way to do this. But on Linux, perror() calls into gettext to show a localized message, and gettext takes a lock, and then you deadlock. This is a hard problem to solve because it requires knowing the status of all locks. This breaks the abstractions that library authors present, where locks are internal and not exposed.

ben0x539 12y ago
Isn't it easier to have the parent print the error message by passing yet another pipe to the child that closes on exec and can be used to pass up errno?
- ridiculous_fish 12y ago
  That's a good idea! I'm not sure if it's easier but it would effectively sidestep this class of issues.
- bdonlan 12y ago
  You could also use an inherited shared memory segment (an anonymous mmap with MAP_SHARED) to report failure codes. Since you only need to set this up once, it could result in lower overhead than creating pipes for every exec.

preillyme 12y ago

The pthread_atfork() function shall declare fork handlers to be called before and after fork(), in the context of the thread that called fork(). The prepare fork handler shall be called before fork() processing commences. The parent fork handle shall be called after fork() processing completes in the parent process. The child fork handler shall be called after fork() processing completes in the child process. If no handling is desired at one or more of these three points, the corresponding fork handler address(es) may be set to NULL. The order of calls to pthread_atfork() is significant. The parent and child fork handlers shall be called in the order in which they were established by calls to pthread_atfork(). The prepare fork handlers shall be called in the opposite order. I'm not sure if that's the best approach, but it's an attempt at least.

pjmlp 12y ago
I just had a look at IEEE Std 1003.1, 2013 Edition for pthread_atfork(). It has a few corner cases across POSIX systems. I wouldn't bet it works 100% the same way in all UNIXes.
- djcapelis 12y ago
  Of course it doesn't, very few things do. Cross platform is hard, but that doesn't mean you don't use features.
  - pjmlp 12y ago
    Of course, but sometimes it is a huge pain. I used to do cross platform across Aix, HP-UX, Solaris, GNU/Linux, FreeBSD and Windows NT/2000 back in the .COM days.
    - wahern 12y ago
      It's much easier in 2014 than it was in 2004. POSIX has evolved, and POSIX conformance has substantially improved. Most systems are, in practice, nearly 100% conformant to POSIX-2001. Excepting Windows, I rarely run into difficult portability problems except when I deliberately use non-POSIX functionality or newer POSIX functionality. I target Linux, OS X, OpenBSD, NetBSD, FreeBSD, Solaris, and AIX. The biggest laggard was OpenBSD, particularly wrt to threading, signal handling, and real-time extensions. But in the past couple of years that's been substantially addressed. One of my biggest headaches now is OS X. They appear to have stopped trying to track POSIX, so while everybody else is busily implementing POSIX-2004, POSIX-2008, and tentative POSIX features, OS X is nearly at a stand-still. OS X hasn't fixed any significant conformance issues, adopted real-time extensions, nor adopted any POSIX-2008 features for several years, now.

ChuckMcM 12y ago

In my opinion this is fundamentally a problem of mixing coprocessing metaphors. The whole thread vs process vs container differences are tied up in leakage between the permissions model (process) and computation model (threads and containers). The thread equivalent of 'fork' would be some form of promotion to a 'root' thread (which is to say one where instance data about computation limits can be changed independently) Processes, which were the traditional collection point of resources under an identity, ideally sit apart from computation constraints. And if you follow that path you realize that resource allocation (which is one of the three key parts of OS management) then need to both be computation aware, and identity aware. In the example of the article, malloc would break the lock such that its validity would be related to the identity it was associated with, so if you promoted a thread to the 'identity' level you would invalidate any locks visible to it that were attached to identity. There have been discussions about this in OS design for almost forever.

deleted 12y ago

[deleted]

robmccoll 12y ago

In the motivating Go example, implementing a fork doesn't seem like it would be particularly challenging since the runtime can pause all executing Go routines and reschedule them in the same place on the other side. Bit easier than pthreads.