3 ms·
"Fork is the direct cause of the stupidest component I’ve ever heard of in an operating system: the out-of-memory (aka OOM) killer." Well, technically, no. Or
by mcguire 9y ago
"Fork is the direct cause of the stupidest component I’ve ever heard of in an operating system: the out-of-memory (aka OOM) killer."
Well, technically, no.
Originally (1st Edition?), IIUC, fork() would fail if enough memory to copy the process wasn't free. Then, when fork() was changed (4th Edition?) to read-only share the address space, that wouldn't happen, but the exec() might fail if there wasn't enough space to set up the child. But those were all doing segmented memory so they're completely irrelevant to modern systems.
The OOM-killer comes with (IIRC) BSD's paged virtual memory. Some bright spark (yes, I have hated this design since I first tripped over it in the early- to mid-'90s) had the brilliant idea of supporting sparse matrices by not requiring allocated memory to be backed by valid pages, either RAM or disk.
If you don't require allocated memory to be backed, you can allocate a giant block of memory; if you never touch most of it, everything will be copacetic. It's good, I was told ca. 1992, for scientific programs in, like, FORTRAN. Yay. Unfortunately, lots of programs started using the feature, including the X server.
(Why do I hate with the fiery passion of a thousand suns the perverted genius who came up with this? AIX 3.2.5/4.1. Running the machine out of memory (remember, allocations don't fail, everything chugs along until some proggie touches an address and the OS can't realize the page) would kick off the OOM-killer, which would immediately hit inetd in the head with its little silver hammer, resulting in a machine that would still be working, -ish, sort-of, but unless you were logged in somewhere already to reboot it, you would have to go visit the physical console.)
Turning off the OOM-killer and overallocation policy was possible, but not advised, since many programs, including the X server, would suddenly use a truly atrocious amount of memory.
TL;DR: Memory overcommitment doesn't have anything to do with fork()/exec(), but rather is a deliberate design decision made with the addition of paged virtual memory, to support a limited class of applications, that probably didn't work very well even for that class.
- sitkack 9y agoThe proper selfish thing to do is lock all of your pages immediately after allocation. https://linux.die.net/man/2/mlock https://linux.die.net/man/2/mlock