5 ms·
He misses the main problem with fork which is poor interaction with threads. Amongst the things you cannot do between the fork and the exec is malloc: another
by rwmj 9y ago
He misses the main problem with fork which is poor interaction with threads. Amongst the things you cannot do between the fork and the exec is malloc: another thread in the parent might be in the middle of malloc, might have grabbed a lock needed by malloc, then that thread disappears after fork but the lock is still held by the new child process which deadlocks in malloc.
This and other things are a massive cause of trouble for libraries that need to run subprocesses but need to also link to main programs that might use threads (such as libvirt).
- nh2 9y agoThis sounds incredibly bad. For most more complex programs and libraries, you have no control over when they malloc(). If you spawn a thread and run any library function in it, it may malloc(). Doesn't this mean that any program written in any programming language that supports multiple threads and spawns subprocesses is broken (it is racy for deadlocks, if the spawned process also wants to malloc)? (EDIT: I misunderstood what the parent said, so it's not so bad. See my comment below.) Is this problem discussed anywhere? Are there known ways how people handle this problem?
- FooBarWidget 9y ago> Doesn't this mean that any program written in any programming language that supports multiple threads and spawns subprocesses is broken (it is racy for deadlocks, if the spawned process also wants to malloc)? Yes. > Are there known ways how people handle this problem? The usual way is to ensure that after forking you only execute async-signal-safe code. Which includes exec(). So if you have more complicated logic which cannot be made async-signal-safe, you exec() an executable which performs that logic. This is a massive pain though. For example setenv() is not async-signal-safe so if I want to exec() something with specific env vars then I have to go through /usr/bin/env. Sigh.
- mcguire 9y agoOr you carefully ensure all of the threads are in a safe state before you fork().
- nh2 9y ago> The usual way is to ensure that after forking you only execute async-signal-safe code. Reading your comment I realised I misunderstood what the parent said. This misunderstanding is what made it a suprise for me. Critically, I missed "... between the fork and the exec ..." and only considered the statement "the lock is still held by the new child process which deadlocks in malloc", assuming this was somehow the case even after the exec(), which would be a huge problem if even the exec()ed child wouldn't be allowed to malloc. Of course on second thought this wrong assumption is nonsensical: When you exec(), you get a new process space, so any lock potentially thought up by libc or anything else, will be gone. So all is fine: For the purpose of starting another executable / implementing a popen(), one just must assure to not malloc (and as you say, in general write signal-handler-safe code) between fork and exec. This is feasible, and I suppose most programming languages do ensure that.
- ridiculous_fish 9y agoRight, however even this is harder than it looks. For example, what happens if exec fails? How do you print an error? Answer: very carefully - certainly not with printf!
- cryptonector 9y agoYou can write() to fildes 2, but I'd not recommend it. Instead what I'd recommend is that you have a pipe() for passing an error message (one byte will do) back to the parent, and then _exit().
- FooBarWidget 9y agoEven this is harder than it should be. What if the exec succeeds? The pipe is never closed. If you mark the pipe as O_CLOEXEC and rely on the parent treating EOF as success: what if the child crashes prematurely before exec? E.g. killed by sysadmin, or by OOM killer. What if a write() to the pipe fails?
- arghwhat 9y agoWell, the rule appears to be: - Don't thread before you fork. - Don't lock before you fork. - Don't open before you fork. - Don't do anything that is not meant to be a shared resource before you fork. Locks and threads is the worst. Locks can end up in weird states.
- JdeBP 9y agoUnless you are trying to implement systemd's "forking" readiness protocol for dæmons, in which case you have to adopt all of these problematic program design items. * http://jdebp.eu./FGA/unix-daemon-readiness-protocol-problems.html#NooneSpeaksForking http://jdebp.eu./FGA/unix-daemon-readiness-protocol-problems...
- rcxdude 9y agothe "forking" 'readiness protocol' exists for backwards compatibility and interop, because it represents the behavior of many daemons designed for sysvinit style daemon management, or at least the behaviour they should have in order to avoid races between services in that model. It was not designed by systemd nor is recommended by them.
- JdeBP 9y agoNo, it does not. Read the answer that I hyperlinked.
- arghwhat 9y agoYes it does. To quote the article you linked, "What they are doing is trying to let system administrators start dæmons the 1980s way..." It is a compatibility mode, and it is broken beyond repair because the technique itself is broken beyond repair.
- JdeBP 9y agoTrying. Now read the explanation of why it does not succeed, because no it does not represent the behaviours of these dæmons.
- AdieuToLogic 9y agoif I want to exec() something with specific env vars then I have to go through /usr/bin/env. Sigh. Why not use execve[0] to set up a specific environment for child processes? 0 - https://www.freebsd.org/cgi/man.cgi?query=execve&apropos=0&sektion=0&manpath=FreeBSD+11.1-RELEASE+and+Ports&arch=default&format=html https://www.freebsd.org/cgi/man.cgi?query=execve&apropos=0&s...
- 0xcde4c3db 9y agoMaybe a tangent, but I never really understood how the memory pointed to by envp is supposed to be managed. Does fork() copy the allocator state such that you can just do malloc(), fork(), then free() in the parent process, then execve() blows away the whole memory map in the child?
- AdieuToLogic 9y agoYes, the flow you describe is pretty much how it goes. The OS will "overlay" the fork'ed child process if exec succeeds in loading the program specified. If not, errno is set and typically the child process reports the problem (using something like strerror_r[0]) then exits. In either case, the parent process can free the memory allocated for the execve call once it returns. 0 - https://www.freebsd.org/cgi/man.cgi?query=strerror_r&apropos=0&sektion=3&manpath=FreeBSD+11.1-RELEASE+and+Ports&arch=default&format=html https://www.freebsd.org/cgi/man.cgi?query=strerror_r&apropos...
- cryptonector 9y agoNothing special is needed here. execve() does blow away the address space of the process and replaces it. As to fork(), it copies the address space. Of course, you can't putenv() on the child-side of fork() if the parent was multi-threaded, and you may not even be able to getenv() in that case either (check the standards and your C library's fork() man page).
- JdeBP 9y agoThis is also the principle underpinning Bernstein's pathexec. * https://cr.yp.to/lib/pathexec.html https://cr.yp.to/lib/pathexec.html
- cryptonector 9y agoUse posix_spawn(). Or construct an envp for execve() before fork()ing.
- tedunangst 9y agohttp://pubs.opengroup.org/onlinepubs/9699919799/functions/fork.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/fo...
- ridiculous_fish 9y agofish shell forks and spawns threads. It handles this issue by ensuring that the children do not call malloc, which is indeed hard to enforce.
- wnoise 9y ago> He misses the main problem with fork which is poor interaction with threads. That's a problem with threads, not fork(). It generalizes to all sorts of things needing to know about internals of threads and locks.
- arghwhat 9y agoIt sounds to me like fork() is unable to handle threads, unlike clone(). I would therefore say that fork() is to blame. Of course, it is a bit of nitpicking, as the problem is that they do not work well when combined.
- cryptonector 9y agoclone() can't either, if you call it like fork(). It's the semantics of fork() that are the problem.
- arghwhat 9y agoThe difference being that the semantics of fork() are optional with clone(), not with fork(). Thus, I would say that clone() can handle the situation.
- cryptonector 9y agoThat's the point of clone(), but the point is that the problems of fork() are with its semantics. Even with clone(), if you want to spawn a new process then your best bet is to go with vfork() semantics -- in principle you could create a thread-like process sharing the parent's address space that can then exec() without affecting the parent, but in practice you can only do this inside a C library because the protocol for setting up TLS (for errno) is underdocumented and requires knowledge of and access to your C library's internals.
- cryptonector 9y agoThat's relatively non-problematic: setup everything you need, fork(), exec() or _exit(). But there's lots of other issues. Things like PID number reuse, libraries in single-threaded programs that may need to reinitialize on fork() (via pthread_atfork()? no because fork() might have been called directly via syscall() or by trapping from asm code, bypassing the C library altogether), and so on. Just use posix_spawn(). EDIT: Also, ultimately, the worst thing about fork() is that it can't possibly be high-performance. Either fork() copies the writable parts of the address-space, or sets up COW, or copies the RSS and sets up COW for the rest. Well, on modern CPUs COW is really expensive. And copying a JVM is no fun either.
- deleted 9y ago[deleted]