6 ms·

> Or, why does an out-of-memory system hang instead of just telling the next process "nope, we don't have 300KB to allocate to you anymore" Blame UNIX for that

> Or, why does an out-of-memory system hang instead of just telling the next process "nope, we don't have 300KB to allocate to you anymore" Blame UNIX for that, and the fork() system call. It's a design quirk. fork() duplicates the process. So suppose your web browser consumes 10GB RAM out of the 16GB total on the system, and wants to run a process for anything. Like it just wants to exec something tiny, like `uname`. 1. 10GB process does fork(). 2. Instantly, you have two 10GB processes 3. A microsecond later, the child calls exec(), completely destroying its own state and replacing it with a 36K binary, freeing 10GB RAM. So there's two ways to go there: 1. You could require step 2 to be a full copy. Which means either you need more RAM, a huge chunk of which would always sit idle, or you need a lot of swap, for the same purpose. 2. We could overlook the memory usage increase and pretend that we have enough memory, and only really panic if the second process truly needs its own 10GB RAM that we don't have. That's what Linux does. The problem with #2 is that dealing with this happens completely in the background, at times completely unpredictable to the code. The OS allocates memory when the child changes memory, like does "a=1" somewhere. A program can't handle memory allocations failures there because as far as it knows, it's not allocating anything. So what you get is this fragile fiction that sometimes breaks and requires the kernel to kill something to maintain the system in some sort of working state. Windows doesn't have this issue at all because it has no fork(). New processes aren't children and start from scratch, so firefox never gets another 10GB sized clone. It just starts a new, 36K sized process.

peheje 4y ago
I find your writing style really pleasant and understandable! Much more so than the StackExchange answer. I really like the breakdown into steps, then what could happen steps and the follow ups. Where can I read more (from you?) in this style about OS and memory management?
jacquesm 4y ago
> 2. Instantly, you have two 10GB processes No, that's not how it works. The process table gets duplicated and copy-on-write takes care of the pages. As long as they are identical they will be shared, there is no way that 10GB of RAM will be allocated to the forked process and that all of the data will be copied.
- worthless-trash 4y ago
  OP is referring to what would happen in the naive implementation, not what actually happens.
- reisse 4y ago
  The data won't be copied, but kernel has to reserve the memory for both processes.
  - l33tman 4y ago
    No, it doesn't. See "overcommit"
    - enedil 4y ago
      In a sense it does. Page tables would need to be copied.
      - jacquesm 4y ago
        Yes, but they all point to the same pages. The tables take up a small fraction of the memory of the pages themselves.
        enedil 4y ago
        But large fraction, if all you do afterwards is an exec call. Given 8 bytes per page table entry and 4k pages, it's 1/512 memory wasted. So if your process uses 8GB, it's 16MB. Still takes noticeable time if you spawn often.
        jacquesm 4y ago
        I've never had the page tables be the cause of out of memory issues. Besides the fact that they are usually pre-allocated to avoid recursive page faults, but nothing would stop you from making the page tables themselves also copy-on-write during a fork.
RcouF1uZ4gsC 4y ago
In retrospect, fork was big mistake. It complicates a bunch of other things and introduces suboptimal patterns.
- t0suj4 4y ago
  A small reminder: In the age of Unix multiuser systems were very common. Fork was the optimal solution to be able to serve as much concurrent users or programs as possible while keeping the implementation simple. Today's RAM is cheap.
  - moonchrome 4y ago
    So much of design constraints in our base abstractions are not relevant today but we're still cobbling together solutions built on legacy technical decisions.
    - archgoon 4y ago
      [dead]
reisse 4y ago
> Blame UNIX for that, and the fork() system call. At least that design failure of UNIX has been fixed long ago. There are posix_spawn(3) and various clone(2) flavours which allow to spawn new process without copying the old one. And a lot of memory-intensive software actually use them, so modern Linux distros can be used without memory overprovisioning. I'd rather blame people who are still using fork(2) for anything that can consume more than 100MB of memory.
- AnIdiotOnTheNet 4y ago
  Can a modern distro really be used without over provisioning? Because the last time I tried it either the DE or display server hard locked immediately and I had to reboot the system. Having this ridiculous setting as the default has basically ensured that we can never turn it off because developers expect things to work this way. They have no idea what to do if malloc errors on them. They like being able to make 1TB allocs without worrying about the consequences and just letting the kernel shoot processes in the head randomly when it all goes south. Hell, the last time this came up many swore that there was literally nothing a programmer could do in the event of OOM. Learned helplessness. It's a goddamned mess and like many of Linux's goddamned messes not only are we still dealing with it in 2023, but every effort to do anything about it faces angry ranty backlash.
  - lxgr 4y ago
    Almost everything in life is overprovisioned, if you think about it: Your ISP, the phone network, hospitals, bank reserves (and deposit insurance)... What makes the approach uniquely unsuitable for memory management? The entire idea of swapping goes out of the window without overprovisioning as well, for better or worse.
    - AnIdiotOnTheNet 4y ago
      Perhaps there is some confusion because I used "overprovision" when the appropriate term here is "overcommit", but Windows manages to work fine without unix-style overcommit. I suspect most OSs in history do not use unix's style of overcommit. > What makes the approach uniquely unsuitable for memory management? The fact that something like OOM killer even needs to exist. Killing random processes to free up memory you blindly promised but couldn't deliver is not a reasonable way to do things. Edit: https://lwn.net/Articles/627725/ https://lwn.net/Articles/627725/
eru 4y ago
Keep in mind that copy-on-write makes analysing the situation a bit more complicated.
- nnntriplesec 4y ago
  How so?
  - pdpi 4y ago
    CoW is a strategy where you don't actually copy memory until you write to it. So, when the 10GB process spawns a child process, that child process also has 10GB of virtual memory, but both processes are backed by the same pages. It's only when one of them writes to a page that a copy happens. When you fork+exec you never actually touch most of those pages, so you never actually pay for them. (Obviously, that's the super-simplified version, and I don't fully understand the subtleties involved, but that's exactly what GP means: it's harder to analyse)
    - eru 4y ago
      Thanks for writing the explanation. To make it slightly more complicated: you don't pay for the 10 GB directly, but you still pay for setting up the metadata, and that scales with the amount of virtual memory used.
l33tman 4y ago
The Windows example is a non-sequitur as in both cases, you end up with the 36K sized process both in Windows and Linux if you want to spawn a sub-process that exec's. The fork() => exec() path is not the issue (if there is an issue at all here), and if you use threading the memory is not forked like this to start with (on either of the OSes). I guess the case you want to highlight is more if you for example mmap() 10 GB of RAM on that 16 GB machine that only has 5 GB unused swap space left and where all of the physical RAM is filled with dirty pages already. Should the mmap() succeed, and then the process is killed if it eventually tries to use more pages than will fit in RAM or the backing swap? This is the overcommit option which is selectable on Linux. I think the defaults seem pretty good and accept that a process can get killed long after the "explicit" memory mapping call is done.
josefx 4y ago
> blame UNIX for that, and the fork() system call. Given that most code I have seen would not be able to handle an allocation failure gracefully I wouldn't call it "blame", if the OS just silently failed memory allocations on whatever program tried to allocate next you would basically end up with a system where random applications crash, which is similar to what the OOM killer does, just with no attempt to be smart about it. Even better, it is outright impossible to gracefully handle allocation failures in some languages, see for example variable length arrays in C.
- tjoff 4y ago
  That is a very weak argument. Also, why would you bother to handle it gracefully when the OS won't allow you to do it? Also, outright impossible in some languages? Just don't use VLAs if then? "Problem" solved.
  - josefx 4y ago
    > Also, why would you bother to handle it gracefully when the OS won't allow you to do it? There are many situations where you can get an allocation failure even with over provisioning enabled. > Just don't use VLAs if then? "Problem" solved. Yes, just don't use that language feature that is visually identical to a normal array. Then make sure that your standard library implementation doesn't have random malloc calls hidden in functions that cannot communicate an error and abort instead https://www.thingsquare.com/blog/articles/rand-may-call-malloc/ https://www.thingsquare.com/blog/articles/rand-may-call-mall.... Then ensure that your dependencies follow the same standards of handling allocation failures ... . I concede that it might be possible, but you are working against an ecosystem that is actively trying to sabotage you.
    - tjoff 4y ago
      VLAs are barely used and frowned upon by most. It is not relevant enough to discuss. Yes, mallocs in standard library is a problem. But this is rather the result of a mindset where over provision exist than anything else.
- 4y ago
0x0 4y ago
You don't HAVE to use fork() + exec() in Linux as far as I know. There is for example posix_swawn() which does the double whammy combo for you. Also, if you DO use fork() without immediately doing an exec(), and start writing all over your existing allocations... just don't?
- the8472 4y ago
  posix_spawn isn't powerful enough since it only supports a limited set of process setup operations. So if you need to do some specific syscalls before exec that aren't covered by it then the fork/exec dance is still necessary. In principle one can use vfork+exec instead but that very hard to get right. What's really needed is io_uring_spawn but that's still a WIP. https://lwn.net/Articles/908268/ https://lwn.net/Articles/908268/
KMag 4y ago
Option 3: vfork() has existed for a long time. The child process temporarily borrows all of the parent's address space. The calling process is frozen until the child exits or calls a flavor of exec. Granted, it's pretty brittle and any modification of non-stack address space other than changing a variable of type pid_t is undefined behavior before exec is called. However, it gets around the disadvantages of fork() while maintaining all of the flexibility of Unix's separation of process creation (fork/vfork) and process initialization (exec*). vfork followed immediately by exec gives you Windows-like process creation, and last I checked, despite having the overhead of a second syscall, was still faster than process creation on Windows.
whartung 4y ago
We actually ran into this a long time ago with Solaris and Java. Java has JSP, Java Server Pages. JSP processing translates a JSP file into Java source code, compiles it, then caches, loads, and executes the resulting class file. Back then, the server would invoke the javac compiler through a standard fork and exec. That’s all well and good, save when you have a large app server image sucking up the majority of the machine. As far as we could tell, it was a copy on write kind of process, it didn’t actually try to do the actual work when forking the app server. Rather it tried to do the allocation, found it didn’t have the space or swap, and just failed with a system OOM error (which differs from a Java out of memory/heap error). As I recall adding swap was the short term fix (once we convinced the ops guy that, yes it was possible to “run out of memory”). Long term we made sure all of our JSPs were pre-compiled. Later, this became a non issue for a variety of reasons, including being able to run the compiler natively within a running JVM.

peheje 4y ago

I find your writing style really pleasant and understandable! Much more so than the StackExchange answer. I really like the breakdown into steps, then what could happen steps and the follow ups. Where can I read more (from you?) in this style about OS and memory management?

jacquesm 4y ago

> 2. Instantly, you have two 10GB processes No, that's not how it works. The process table gets duplicated and copy-on-write takes care of the pages. As long as they are identical they will be shared, there is no way that 10GB of RAM will be allocated to the forked process and that all of the data will be copied.

worthless-trash 4y ago
OP is referring to what would happen in the naive implementation, not what actually happens.
reisse 4y ago
The data won't be copied, but kernel has to reserve the memory for both processes.
- l33tman 4y ago
  No, it doesn't. See "overcommit"
  - enedil 4y ago
    In a sense it does. Page tables would need to be copied.
    - jacquesm 4y ago
      Yes, but they all point to the same pages. The tables take up a small fraction of the memory of the pages themselves.
      - enedil 4y ago
        But large fraction, if all you do afterwards is an exec call. Given 8 bytes per page table entry and 4k pages, it's 1/512 memory wasted. So if your process uses 8GB, it's 16MB. Still takes noticeable time if you spawn often.
        jacquesm 4y ago
        I've never had the page tables be the cause of out of memory issues. Besides the fact that they are usually pre-allocated to avoid recursive page faults, but nothing would stop you from making the page tables themselves also copy-on-write during a fork.

RcouF1uZ4gsC 4y ago

In retrospect, fork was big mistake. It complicates a bunch of other things and introduces suboptimal patterns.

t0suj4 4y ago
A small reminder: In the age of Unix multiuser systems were very common. Fork was the optimal solution to be able to serve as much concurrent users or programs as possible while keeping the implementation simple. Today's RAM is cheap.
- moonchrome 4y ago
  So much of design constraints in our base abstractions are not relevant today but we're still cobbling together solutions built on legacy technical decisions.
  - archgoon 4y ago
    [dead]

reisse 4y ago

> Blame UNIX for that, and the fork() system call. At least that design failure of UNIX has been fixed long ago. There are posix_spawn(3) and various clone(2) flavours which allow to spawn new process without copying the old one. And a lot of memory-intensive software actually use them, so modern Linux distros can be used without memory overprovisioning. I'd rather blame people who are still using fork(2) for anything that can consume more than 100MB of memory.

AnIdiotOnTheNet 4y ago
Can a modern distro really be used without over provisioning? Because the last time I tried it either the DE or display server hard locked immediately and I had to reboot the system. Having this ridiculous setting as the default has basically ensured that we can never turn it off because developers expect things to work this way. They have no idea what to do if malloc errors on them. They like being able to make 1TB allocs without worrying about the consequences and just letting the kernel shoot processes in the head randomly when it all goes south. Hell, the last time this came up many swore that there was literally nothing a programmer could do in the event of OOM. Learned helplessness. It's a goddamned mess and like many of Linux's goddamned messes not only are we still dealing with it in 2023, but every effort to do anything about it faces angry ranty backlash.
- lxgr 4y ago
  Almost everything in life is overprovisioned, if you think about it: Your ISP, the phone network, hospitals, bank reserves (and deposit insurance)... What makes the approach uniquely unsuitable for memory management? The entire idea of swapping goes out of the window without overprovisioning as well, for better or worse.
  - AnIdiotOnTheNet 4y ago
    Perhaps there is some confusion because I used "overprovision" when the appropriate term here is "overcommit", but Windows manages to work fine without unix-style overcommit. I suspect most OSs in history do not use unix's style of overcommit. > What makes the approach uniquely unsuitable for memory management? The fact that something like OOM killer even needs to exist. Killing random processes to free up memory you blindly promised but couldn't deliver is not a reasonable way to do things. Edit: https://lwn.net/Articles/627725/ https://lwn.net/Articles/627725/

eru 4y ago

Keep in mind that copy-on-write makes analysing the situation a bit more complicated.

nnntriplesec 4y ago
How so?
- pdpi 4y ago
  CoW is a strategy where you don't actually copy memory until you write to it. So, when the 10GB process spawns a child process, that child process also has 10GB of virtual memory, but both processes are backed by the same pages. It's only when one of them writes to a page that a copy happens. When you fork+exec you never actually touch most of those pages, so you never actually pay for them. (Obviously, that's the super-simplified version, and I don't fully understand the subtleties involved, but that's exactly what GP means: it's harder to analyse)
  - eru 4y ago
    Thanks for writing the explanation. To make it slightly more complicated: you don't pay for the 10 GB directly, but you still pay for setting up the metadata, and that scales with the amount of virtual memory used.

l33tman 4y ago

The Windows example is a non-sequitur as in both cases, you end up with the 36K sized process both in Windows and Linux if you want to spawn a sub-process that exec's. The fork() => exec() path is not the issue (if there is an issue at all here), and if you use threading the memory is not forked like this to start with (on either of the OSes). I guess the case you want to highlight is more if you for example mmap() 10 GB of RAM on that 16 GB machine that only has 5 GB unused swap space left and where all of the physical RAM is filled with dirty pages already. Should the mmap() succeed, and then the process is killed if it eventually tries to use more pages than will fit in RAM or the backing swap? This is the overcommit option which is selectable on Linux. I think the defaults seem pretty good and accept that a process can get killed long after the "explicit" memory mapping call is done.

josefx 4y ago

> blame UNIX for that, and the fork() system call. Given that most code I have seen would not be able to handle an allocation failure gracefully I wouldn't call it "blame", if the OS just silently failed memory allocations on whatever program tried to allocate next you would basically end up with a system where random applications crash, which is similar to what the OOM killer does, just with no attempt to be smart about it. Even better, it is outright impossible to gracefully handle allocation failures in some languages, see for example variable length arrays in C.

tjoff 4y ago
That is a very weak argument. Also, why would you bother to handle it gracefully when the OS won't allow you to do it? Also, outright impossible in some languages? Just don't use VLAs if then? "Problem" solved.
- josefx 4y ago
  > Also, why would you bother to handle it gracefully when the OS won't allow you to do it? There are many situations where you can get an allocation failure even with over provisioning enabled. > Just don't use VLAs if then? "Problem" solved. Yes, just don't use that language feature that is visually identical to a normal array. Then make sure that your standard library implementation doesn't have random malloc calls hidden in functions that cannot communicate an error and abort instead https://www.thingsquare.com/blog/articles/rand-may-call-malloc/ https://www.thingsquare.com/blog/articles/rand-may-call-mall.... Then ensure that your dependencies follow the same standards of handling allocation failures ... . I concede that it might be possible, but you are working against an ecosystem that is actively trying to sabotage you.
  - tjoff 4y ago
    VLAs are barely used and frowned upon by most. It is not relevant enough to discuss. Yes, mallocs in standard library is a problem. But this is rather the result of a mindset where over provision exist than anything else.
4y ago

0x0 4y ago

You don't HAVE to use fork() + exec() in Linux as far as I know. There is for example posix_swawn() which does the double whammy combo for you. Also, if you DO use fork() without immediately doing an exec(), and start writing all over your existing allocations... just don't?

the8472 4y ago
posix_spawn isn't powerful enough since it only supports a limited set of process setup operations. So if you need to do some specific syscalls before exec that aren't covered by it then the fork/exec dance is still necessary. In principle one can use vfork+exec instead but that very hard to get right. What's really needed is io_uring_spawn but that's still a WIP. https://lwn.net/Articles/908268/ https://lwn.net/Articles/908268/

KMag 4y ago

Option 3: vfork() has existed for a long time. The child process temporarily borrows all of the parent's address space. The calling process is frozen until the child exits or calls a flavor of exec. Granted, it's pretty brittle and any modification of non-stack address space other than changing a variable of type pid_t is undefined behavior before exec is called. However, it gets around the disadvantages of fork() while maintaining all of the flexibility of Unix's separation of process creation (fork/vfork) and process initialization (exec*). vfork followed immediately by exec gives you Windows-like process creation, and last I checked, despite having the overhead of a second syscall, was still faster than process creation on Windows.

whartung 4y ago

We actually ran into this a long time ago with Solaris and Java. Java has JSP, Java Server Pages. JSP processing translates a JSP file into Java source code, compiles it, then caches, loads, and executes the resulting class file. Back then, the server would invoke the javac compiler through a standard fork and exec. That’s all well and good, save when you have a large app server image sucking up the majority of the machine. As far as we could tell, it was a copy on write kind of process, it didn’t actually try to do the actual work when forking the app server. Rather it tried to do the allocation, found it didn’t have the space or swap, and just failed with a system OOM error (which differs from a Java out of memory/heap error). As I recall adding swap was the short term fix (once we convinced the ops guy that, yes it was possible to “run out of memory”). Long term we made sure all of our JSPs were pre-compiled. Later, this became a non issue for a variety of reasons, including being able to run the compiler natively within a running JVM.