3 ms·

> What makes fork() slow for large programs like Node is that fork() needs to lock and hold every single mutex that exists in a process while it happens It rea

by o11c 2y ago

> What makes fork() slow for large programs like Node is that fork() needs to lock and hold every single mutex that exists in a process while it happens It really doesn't; if it does that's a bug to fix. It only requires that the child process does not call any function that is not async-signal-safe if threads exist, and end with a call to exec or _exit. In C, this is usually pretty easy to arrange; at most the difficulty is if you want to support arbitrary file-descriptor mappings. However, it is impossible from languages like Javascript (or Python, etc.). pthread_atfork() is really only useful to call if you know you're using multiple processes and not using threads at all. _Fork() is supposed to be a thing now (to avoid any historical atfork handlers) but can't be relied on to exist. posix_spawn() has the unfortunate problem of being buggy and version-dependent when various options are used. It's probably fine if you don't need the options (for example, if you exec a shim first), or if you know how recent a platform you can use. vfork() on Linux is mostly an optimization (for scheduling or for when overcommit is disabled), but compiler optimizations can make it buggy (though given that /bin/sh etc. use it, compiler probably won't break it too much if you call it in the same way). For best results, it's advisable to either call clone (with a separate stack directly), or use architecture-specific assembly to call `vfork` in a way the compiler can't mess up. vfork() on many other platforms is basically just an alias for fork() (or maybe _Fork() ?), but I haven't investigated the details much. fork() is fine if your parent process isn't big enough that you're worried about memory accounting though, and you haven't added braindead atfork handlers.

jart 2y ago
You absolutely need to lock all the locks, because fork() kills all the other threads, so if any one of them was in a critical section while fork was called, then the new process will deadlock if it tries to acquire that lock afterwards. Yes, I know the POSIX standard has turned against fork() and considers vfork() beyond the pale. I don't know how they can forbid UNIX behavior that's essential to the workings of /bin/sh. No it's not realistic for the OS to define its most important function to be as useless as a signal handler, but here we are. That advice will be never be accepted. People on the best OSes like Linux and FreeBSD will always use vfork() regardless of what POSIX says. The de facto standard is that only vfork() must guarantee a swift execve() call. You learn to take the official standard with a grain of salt when you realize it's bankrolled by regulated enterprises who hire cheap engineers to work on life or death machinery and therefore can't be trusted with powerful footguns.
- o11c 2y ago
  > then the new process will deadlock if it tries to acquire that lock afterwards which is not async-signal safe and thus illegal if other threads ever existed (or if we're called from a signal handler). All semi-new programs (which might use threads) are responsible for not introducing such bugs. And if we're the only thread (and not in a signal handler), which all sufficiently-old programs (including shells) will be, it's the caller's responsibility to not call fork() while holding a mutex. Thus, one way or another, there's really no need to do this. From an application perspective, it's unnecessary, and from a libc perspective it is at most a QoI feature (one which should be limited to a very small number of locks at most, and frankly we've strongly learned should just detect the error (which is a simple matter of 2 thread-local counters - one for "number of mutexes held" and one for "number of async-signal contexts entered" - as well as knowledge of whether other threads exist) rather than trying to continue in the face of UB).
  - zbentley 2y ago
    Illegal it may be, but as jart (who has probably seen some shit, to put it mildly) points out, that doesn’t stop a very wide variety of programs from depending on this behavior, intentionally or inadvertently. Heck, runtimes like Python (and Node, if jart is correct in GGP) ship with atfork handlers specifically for the purpose of massaging runtime-created mutexes/condvars/etc. I don’t think those are there because the language maintainers like breaking the rules; they’re probably there because users kept depending on this behavior in spite of cautionary statements.
    - saagarjha 2y ago
      > I don’t think those are there because the language maintainers like breaking the rules You think wrong.
      - zbentley 2y ago
        Say more? Is this just get-gud anti scripting language elitism, or a reference to some specific history that the Python/Node/others(?) maintainers have regarding decisions made in this area?

by o11c 2y ago

jart 2y ago

You absolutely need to lock all the locks, because fork() kills all the other threads, so if any one of them was in a critical section while fork was called, then the new process will deadlock if it tries to acquire that lock afterwards. Yes, I know the POSIX standard has turned against fork() and considers vfork() beyond the pale. I don't know how they can forbid UNIX behavior that's essential to the workings of /bin/sh. No it's not realistic for the OS to define its most important function to be as useless as a signal handler, but here we are. That advice will be never be accepted. People on the best OSes like Linux and FreeBSD will always use vfork() regardless of what POSIX says. The de facto standard is that only vfork() must guarantee a swift execve() call. You learn to take the official standard with a grain of salt when you realize it's bankrolled by regulated enterprises who hire cheap engineers to work on life or death machinery and therefore can't be trusted with powerful footguns.

o11c 2y ago
> then the new process will deadlock if it tries to acquire that lock afterwards which is not async-signal safe and thus illegal if other threads ever existed (or if we're called from a signal handler). All semi-new programs (which might use threads) are responsible for not introducing such bugs. And if we're the only thread (and not in a signal handler), which all sufficiently-old programs (including shells) will be, it's the caller's responsibility to not call fork() while holding a mutex. Thus, one way or another, there's really no need to do this. From an application perspective, it's unnecessary, and from a libc perspective it is at most a QoI feature (one which should be limited to a very small number of locks at most, and frankly we've strongly learned should just detect the error (which is a simple matter of 2 thread-local counters - one for "number of mutexes held" and one for "number of async-signal contexts entered" - as well as knowledge of whether other threads exist) rather than trying to continue in the face of UB).
- zbentley 2y ago
  Illegal it may be, but as jart (who has probably seen some shit, to put it mildly) points out, that doesn’t stop a very wide variety of programs from depending on this behavior, intentionally or inadvertently. Heck, runtimes like Python (and Node, if jart is correct in GGP) ship with atfork handlers specifically for the purpose of massaging runtime-created mutexes/condvars/etc. I don’t think those are there because the language maintainers like breaking the rules; they’re probably there because users kept depending on this behavior in spite of cautionary statements.
  - saagarjha 2y ago
    > I don’t think those are there because the language maintainers like breaking the rules You think wrong.
    - zbentley 2y ago
      Say more? Is this just get-gud anti scripting language elitism, or a reference to some specific history that the Python/Node/others(?) maintainers have regarding decisions made in this area?