3 ms·

> The shell actually uses copy-on-write usefully. (And yes I'm aware that there's a good argument that the shell is almost the ONLY program that needs fork() !

> The shell actually uses copy-on-write usefully. (And yes I'm aware that there's a good argument that the shell is almost the ONLY program that needs fork() !) It's been a while since I looked at it, but I believe Android uses fork for it's copy-on-write sementics to optimize app startup. On boot it initializes a single instance of the app runtime environment. Then when you launch apps that initial process is forked. As a result you do not need to reinitialize the runtime for every app launch.

chubot 4y ago
Yes I think the argument is that Android (and Chrome) could use something like vfork or posix_spawn(). I'm not sure which, if any; I'd like to see an analysis of that... The issue is what kernel state is preserved/shared across the process creation call. Every process sort of has a "mirror" in kernel memory. The user memory is CoW, and I suppose you also have to choose whether to copy or reference every kernel data structure as well --- open files in FD tables which point to disk/pipes/sockets, locks which seem to be nonsensical, etc. But probably you can get the "warmup" property without the full semantics of fork(). That is the CoW of user memory is a somewhat separate choice from the kernel data structures. ---- As far as the shell .... In the recent linked thread, Ninja uses posix_spawn because it has a simple use of subprocesses: https://news.ycombinator.com/item?id=30503382 https://news.ycombinator.com/item?id=30503382 So there are definitely cases where a shell uses fork/exec like Ninja, so you could imagine optimizing it. But the subshell/subinterpreter case is probably the most general -- the language semantics depend on it. And it's actually useful, e.g. this "alternative shell challenge": https://www.oilshell.org/blog/2020/02/good-parts-sketch.html#the-alternative-shell-challenge https://www.oilshell.org/blog/2020/02/good-parts-sketch.html... and I use that pattern literally yesterday and this morning, etc. ---- edit: looks like fish already addresses this, i.e. where you can use posix_spawn in a shell, and where you can't! https://news.ycombinator.com/item?id=31743230 https://news.ycombinator.com/item?id=31743230
epmos 4y ago
This is moderately common for environments where you are pushing a lot of startup work into the dynamic linker and will be launching processes frequently. Loading shared libraries for example. You have a parent process which uses dlopen() to load all the libraries you want to avoid re-linking. When you want to spawn a child, rather than exec() you dlopen() an object with your child's main() and call it. For the case where you have enough libraries this is much faster than an exec(), saving tens of seconds on every application launch if you have a really bad case of C++. There some small surprises which become obvious with a little thought. You are responsible for everything that normally happens in your process before main() is called. ASLR is only done once per session. People rarely think to fix-up argv[] for ps and friends in the first version.

chubot 4y ago

Yes I think the argument is that Android (and Chrome) could use something like vfork or posix_spawn(). I'm not sure which, if any; I'd like to see an analysis of that... The issue is what kernel state is preserved/shared across the process creation call. Every process sort of has a "mirror" in kernel memory. The user memory is CoW, and I suppose you also have to choose whether to copy or reference every kernel data structure as well --- open files in FD tables which point to disk/pipes/sockets, locks which seem to be nonsensical, etc. But probably you can get the "warmup" property without the full semantics of fork(). That is the CoW of user memory is a somewhat separate choice from the kernel data structures. ---- As far as the shell .... In the recent linked thread, Ninja uses posix_spawn because it has a simple use of subprocesses: https://news.ycombinator.com/item?id=30503382 https://news.ycombinator.com/item?id=30503382 So there are definitely cases where a shell uses fork/exec like Ninja, so you could imagine optimizing it. But the subshell/subinterpreter case is probably the most general -- the language semantics depend on it. And it's actually useful, e.g. this "alternative shell challenge": https://www.oilshell.org/blog/2020/02/good-parts-sketch.html#the-alternative-shell-challenge https://www.oilshell.org/blog/2020/02/good-parts-sketch.html... and I use that pattern literally yesterday and this morning, etc. ---- edit: looks like fish already addresses this, i.e. where you can use posix_spawn in a shell, and where you can't! https://news.ycombinator.com/item?id=31743230 https://news.ycombinator.com/item?id=31743230

epmos 4y ago

This is moderately common for environments where you are pushing a lot of startup work into the dynamic linker and will be launching processes frequently. Loading shared libraries for example. You have a parent process which uses dlopen() to load all the libraries you want to avoid re-linking. When you want to spawn a child, rather than exec() you dlopen() an object with your child's main() and call it. For the case where you have enough libraries this is much faster than an exec(), saving tens of seconds on every application launch if you have a really bad case of C++. There some small surprises which become obvious with a little thought. You are responsible for everything that normally happens in your process before main() is called. ASLR is only done once per session. People rarely think to fix-up argv[] for ps and friends in the first version.