3 ms·

> Otherwise, fork/exec would stop working in any process using more than half of the system memory. Somehow Solaris manages just fine. And don't forget that s

by wahern 5y ago

> Otherwise, fork/exec would stop working in any process using more than half of the system memory. Somehow Solaris manages just fine. And don't forget that swap memory exists. Ironically, using overcommit without swap is asking for trouble on Linux. Overcommit or no overcommit, the Linux VM and page buffer systems are designed with the expectation of swap.

pjc50 5y ago
How does Solaris handle this? Or Darwin?
- wahern 5y ago
  Solaris has strict memory accounting; fork will fail if the system can't guarantee space for all non-shared anonymous memory. macOS has overcommit (all BSDs do to some extent, at least for fork), but it also automatically creates swap space so you rarely encounter issues one way or another in practice. fork and malloc can also fail in Linux even with overcommit enabled (rlimits, but also OOM killer racing with I/O page dirtying triggering best-effort timeout), so Linux buys you a little convenience at the cost of making it impossibly difficult to actually guarantee behavior when it matters most.
- scottlamb 5y ago
  I don't use Solaris, but I see they have vfork [1] which should be unaffected. Likewise posix_spawn{,p}. (I think those are often implemented in userspace via vfork; unsure if there's a separate system call on Solaris.) [1] https://docs.oracle.com/cd/E26505_01/html/816-5167/vfork-2.html https://docs.oracle.com/cd/E26505_01/html/816-5167/vfork-2.h...
joosters 5y ago
Solaris does have this problem! If you have a huge program running, fork()img it can fail on Solaris, even if you only want to just exec a tiny program. The key way of avoiding this is to ensure you have lots and lots of swap space.
_3u10 5y ago
Even without overcommit swap makes the system work better as unused pages can be written to disk and that memory used for disk cache and I think can also help with defragmentation of memory not sure how that process actually works.

by wahern 5y ago

pjc50 5y ago

How does Solaris handle this? Or Darwin?

wahern 5y ago
Solaris has strict memory accounting; fork will fail if the system can't guarantee space for all non-shared anonymous memory. macOS has overcommit (all BSDs do to some extent, at least for fork), but it also automatically creates swap space so you rarely encounter issues one way or another in practice. fork and malloc can also fail in Linux even with overcommit enabled (rlimits, but also OOM killer racing with I/O page dirtying triggering best-effort timeout), so Linux buys you a little convenience at the cost of making it impossibly difficult to actually guarantee behavior when it matters most.
scottlamb 5y ago
I don't use Solaris, but I see they have vfork [1] which should be unaffected. Likewise posix_spawn{,p}. (I think those are often implemented in userspace via vfork; unsure if there's a separate system call on Solaris.) [1] https://docs.oracle.com/cd/E26505_01/html/816-5167/vfork-2.html https://docs.oracle.com/cd/E26505_01/html/816-5167/vfork-2.h...

joosters 5y ago

Solaris does have this problem! If you have a huge program running, fork()img it can fail on Solaris, even if you only want to just exec a tiny program. The key way of avoiding this is to ensure you have lots and lots of swap space.

_3u10 5y ago

Even without overcommit swap makes the system work better as unused pages can be written to disk and that memory used for disk cache and I think can also help with defragmentation of memory not sure how that process actually works.