4 ms·

That's not necessarily the case though. It may be worth it for, say, a document viewer to catch the OOM condition and show a user-friendly error instead of dyin

That's not necessarily the case though. It may be worth it for, say, a document viewer to catch the OOM condition and show a user-friendly error instead of dying. Of course, linux with overcommit memory can't do this. But on Windows, that's totally a thing that can happen.

godshatter 3y ago
I was curious so I did a Brave search to find out if that behavior can be changed. You can supposedly (I haven't tried it) echo 2 into /proc/sys/vm/overcommit_memory and the kernel will refuse to commit more memory for a process than the available swap space and a fraction of available memory (which is also configurable). See https://www.win.tue.nl/~aeb/linux/lk/lk-9.html#ss9.6 https://www.win.tue.nl/~aeb/linux/lk/lk-9.html#ss9.6 for more details. I usually write my programs to only grab a little more memory than is actually needed so I might play around with this at home. I wonder if this has lead to a culture of grabbing more memory than is actually needed since mallocs only fail at large values if everything is set the traditional way. Defaulting to overcommit seems risky. I'd much rather the system tell me no more memory is available than just having something segfault. I could always wait a bit and try again or something or at the very least shut down the program in a controlled manner.
- temac 3y ago
  That's a terrible idea to disable overcommit on a generalist Linux system because: * exec * some tools or even libs map gigantic area of anon mem but only touch few bits of it.
  - fweimer 3y ago
    You can add enough swap space that fork+execve always works in practice (although vfork or vfork-style clone is obviously better if the goal is to execve pretty much immediately anyway). Linux allows reserving address space with PROT_NONE, populating it later with mprotect or MAP_FIXED, and many programs do it like that. However, I stopped using vm.overcommit_memory=2 because the i915 driver has something called the GEM shrinker, and that never runs in that mode. That means all memory ends up going to the driver over time, and other allocations fail eventually. Other parts of the graphics stack do not handle malloc failures gracefully, either. In my experience, that meant I got many more desktop crashes in mode 2 than in the default mode with the usual kernel OOM handler and its forced process termination.

godshatter 3y ago

I was curious so I did a Brave search to find out if that behavior can be changed. You can supposedly (I haven't tried it) echo 2 into /proc/sys/vm/overcommit_memory and the kernel will refuse to commit more memory for a process than the available swap space and a fraction of available memory (which is also configurable). See https://www.win.tue.nl/~aeb/linux/lk/lk-9.html#ss9.6 https://www.win.tue.nl/~aeb/linux/lk/lk-9.html#ss9.6 for more details. I usually write my programs to only grab a little more memory than is actually needed so I might play around with this at home. I wonder if this has lead to a culture of grabbing more memory than is actually needed since mallocs only fail at large values if everything is set the traditional way. Defaulting to overcommit seems risky. I'd much rather the system tell me no more memory is available than just having something segfault. I could always wait a bit and try again or something or at the very least shut down the program in a controlled manner.

temac 3y ago
That's a terrible idea to disable overcommit on a generalist Linux system because: * exec * some tools or even libs map gigantic area of anon mem but only touch few bits of it.
- fweimer 3y ago
  You can add enough swap space that fork+execve always works in practice (although vfork or vfork-style clone is obviously better if the goal is to execve pretty much immediately anyway). Linux allows reserving address space with PROT_NONE, populating it later with mprotect or MAP_FIXED, and many programs do it like that. However, I stopped using vm.overcommit_memory=2 because the i915 driver has something called the GEM shrinker, and that never runs in that mode. That means all memory ends up going to the driver over time, and other allocations fail eventually. Other parts of the graphics stack do not handle malloc failures gracefully, either. In my experience, that meant I got many more desktop crashes in mode 2 than in the default mode with the usual kernel OOM handler and its forced process termination.