3 ms·
> Spawning processes generally takes much less than 1 ms on Unix It depends on whether one uses clone, fork, posix_spawn etc. Fork can take a while depending
by charleshn 1y ago
> Spawning processes generally takes much less than 1 ms on Unix
It depends on whether one uses clone, fork, posix_spawn etc.
Fork can take a while depending on the size of the address space, number of VMAs etc.
- crackez 1y agoFork on Linux should use copy-on-write vmpages now, so if you fork inside python it should be cheap. If you launch a new Python process from let's say the shell, and it's already in the buffer cache, then you should only have to pay the startup CPU cost of the interpreter, since the IO should be satisfied from buffer cache...
- charleshn 1y ago> Fork on Linux should use copy-on-write vmpages now, so if you fork inside python it should be cheap. No, that's exactly the point I'm making, copying PTEs is not cheap on a large address space, woth many VMAs. You can run a simple python script allocating a large list and see how it affects fork time.
- charleshn 1y agoSee e.g. https://www.alibabacloud.com/blog/async-fork-mitigating-query-latency-spikes-incurred-by-the-fork-based-snapshot-mechanism_600205 https://www.alibabacloud.com/blog/async-fork-mitigating-quer...
- knome 1y agofor glibc and linux, fork just calls clone. as does posix_spawn, using the flag CLONE_VFORK.