4 ms·
Yap. Erlang 19 (latest version) switched to using a smaller spawner executable and spawns all OS processes from there by forking. So it forks something restrict
by rdtsc 10y ago
Yap. Erlang 19 (latest version) switched to using a smaller spawner executable and spawns all OS processes from there by forking. So it forks something restricted and small not the whole VM.
Here are the details of how it works:
https://github.com/erlang/otp/blob/a5256e5221aff30f6d2cc7fab4875914ae734217/erts/emulator/sys/unix/erl_child_setup.c#L21-L49 https://github.com/erlang/otp/blob/a5256e5221aff30f6d2cc7fab...
They also claim a 3-5x speedup for launching external commands because of it. So there is a nice performance boost as well.
So basically can add 4th strategy -- fork once a small program at the start, then fork from there from then on.
- xroche 10y agoThis is not a bad strategy, especially because between the moment you fork() and the child calls exec(), you have a lot of memory regions shared in copy-on-write (because it would be Too costly to actually duplicate the universelle). Which means that busy threads writing in memory in the parent area will trigger minor page faults, possibly impacting performances...
- ithkuil 10y agoyeah, that's why vfork has been introduced back in the days. See http://man7.org/linux/man-pages/man2/vfork.2.html http://man7.org/linux/man-pages/man2/vfork.2.html
- the_why_of_y 10y agoThe "historic description" in the man page you cite disagrees: vfork(2) was added before BSD had copy-on-write, so the memory was actually copied and vfork(2) is essentially a crude hack to avoid that. Unfortunately once copy-on-write was implemented, the crude hack was retained for backwards compatibility....
- evanj 10y agoThanks for pointing this out! Interesting that the code states that part of the concern is increasing memory usage. I don't quite understand why it would "duplicate the memory usage" since fork uses copy-on-write? Anyway, I didn't explain it well, but this was what I was trying to get at with "fork a worker at the beginning of your program, before there can be other threads."
- the_why_of_y 10y agoCopy-on-write doesn't work the same on every OS... Linux uses over-commit so it's not a big deal, but Solaris (and NT) don't over-commit so you actually need enough VM at the time when you call fork or fork will fail, so you may need to provide a lot of swap space on those systems to successfully call fork in large processes.
- evanj 10y agoAh thanks, that makes sense. Fun trade-offs: Either run out of memory when you call fork, or fight with the out-of-memory killer at a later time :)