4 ms·

The problem with `fork()` is that its usage requires over commit. Consider the scenario of a process with 51% of memory usage, what happens when it forks? The

by ayende 5y ago

The problem with `fork()` is that its usage requires over commit. Consider the scenario of a process with 51% of memory usage, what happens when it forks? The system pretty much _has_ to allow overcommit at this point, otherwise fork() will fail. Now, you can change the accounting to say that when the process modify a memory page (thus creating a copy), we'll check that there is enough memory _then_. That would move the "danger" of memory being yanked from underneath you. But that means that a process that calls fork() may then run into a problem some indeterministic time later. That is kind of a killer.

zinekeller 5y ago
ERRORS The fork() function will fail if: [EAGAIN] The system lacked the necessary resources to create another process, or the system-imposed limit on the total number of processes under execution system-wide or by a single user {CHILD_MAX} would be exceeded. The fork() function may fail if: [ENOMEM] Insufficient storage space is available. The POSIX specification is clear on this: you can hard fail. Also, as an aside malloc() can also fail and the program must be able to handle that. Unfortunately for some programmers (mainly coming from GC-aware languages), they just assume that it won't fail. (Also, most rely on fork-exec but there's posix_spawn() for de novo execution). Again, NT does indeed have a fork function, however the Windows API doesn't expose that (and this is the hack Cygwin used when forking), but it's designed to fail instead of overcommit when all memory is exhausted.
- rbanffy 5y ago
  > Unfortunately for some programmers (mainly coming from GC-aware languages), they just assume that it won't fail. In most GC languages I've used, if malloc fails for some reason (we rarely call it ourselves directly) an error happens and, unless handled, the whole app crashes. Which is fine, BTW, because I much prefer an app that crashes and dies to one that does things with the assumption something that didn't work did.
- cozzyd 5y ago
  You CAN hard fail, but not overcommitting can severely limit the system in the very common case where you fork and exec. See e.g. https://lwn.net/Articles/785430/ https://lwn.net/Articles/785430/

by ayende 5y ago

zinekeller 5y ago

ERRORS The fork() function will fail if: [EAGAIN] The system lacked the necessary resources to create another process, or the system-imposed limit on the total number of processes under execution system-wide or by a single user {CHILD_MAX} would be exceeded. The fork() function may fail if: [ENOMEM] Insufficient storage space is available. The POSIX specification is clear on this: you can hard fail. Also, as an aside malloc() can also fail and the program must be able to handle that. Unfortunately for some programmers (mainly coming from GC-aware languages), they just assume that it won't fail. (Also, most rely on fork-exec but there's posix_spawn() for de novo execution). Again, NT does indeed have a fork function, however the Windows API doesn't expose that (and this is the hack Cygwin used when forking), but it's designed to fail instead of overcommit when all memory is exhausted.

rbanffy 5y ago
> Unfortunately for some programmers (mainly coming from GC-aware languages), they just assume that it won't fail. In most GC languages I've used, if malloc fails for some reason (we rarely call it ourselves directly) an error happens and, unless handled, the whole app crashes. Which is fine, BTW, because I much prefer an app that crashes and dies to one that does things with the assumption something that didn't work did.
cozzyd 5y ago
You CAN hard fail, but not overcommitting can severely limit the system in the very common case where you fork and exec. See e.g. https://lwn.net/Articles/785430/ https://lwn.net/Articles/785430/