5 ms·

It should have been clarified, because it is not obvious: The kernel can evict memory-mappings of executable files which are currently running. When they jump

It should have been clarified, because it is not obvious: The kernel can evict memory-mappings of executable files which are currently running. When they jump to a part of the executable that is no longer in memory, it can page that part back in from the executable file on disk. This is pretty cool. But when memory is very low, the kernel will evict practically all user-space executable mappings from memory, and will be reading back in and evicting back out executable file contents on practically every single context switch. It's just trying so hard to squeeze out some space to make its tasks fit in memory and complete successfully. I think this was the desired behavior of big-iron batch-processing in the 90s? Not sure why it has persisted so long. I'm a big fan of linux and this is my biggest pet-peeve.

ajross 7y ago
What's the other option, though? It gets a fault for a page somewhere, it needs memory, and it has none. You have to evict something. What's your choice if not swap or filesystem-backed pages? Systems have never operated well under true VM pressure. Not in the 90's, not now. When the working set goes beyond available memory performance falls off a cliff. And the report in the article doesn't seem to have a good comparison anyway. I mean, do we seriously believe that Windows or OS X can handle this condition (4G, no swap) better? I mean, swap is default for a reason.
- throwaway2048 7y ago
  I'd much rather have OOMkiller trigger than have my system sit completely unresponsive for huge chunks of time, while I slowly struggle to kill something.
  - dahfizz 7y ago
    While this might add convenience when all you do is use a web browser, Linux is huge in the server space. Having the OS suddenly and somewhat unpredictability killing programs is absolutely not an option for most workloads.
    - quazeekotl 7y ago
      I don't think having a system sit completely unresponsive to anything for a huge amount of time is exactly optimal either.
    - _jal 7y ago
      Absolutely, which is why configurable knobs are things that exist. I already have to mess with the OOMkiller on my servers; I'm fine with fiddling with knobs on my desktop box too. This is Linux, after all. Wasn't one of the constant talking points in the war over the Init System That Shall Not Be Named that Linux needs to scale from embedded to big iron?
    - eikenberry 7y ago
      If you work in the cloud for any length of time you learn this is not true. Every system must be able to handle processes being killed unpredictably otherwise you will have problems. Better to deal with it and plan for it than expect a system to have 100% uptime. Because you never have 100% uptime.
    - TheDong 7y ago
      The oom killer already exists on servers and already can kill programs. If you want to turn off overcommit and have the system power off when it runs out of memory, the kernel allows that. Whatever knob they add will certainly be configurable, and ubuntu desktop can configure it one way while ubuntu server configures it the other, if it turns out people would prefer that. In practice, people running servers seem to want oom killers to kick in before the server barfs. One example of this is facebook's oomd [1]. I assure you, they're running that on their servers, not their web-browser-machines. [1]: https://github.com/facebookincubator/oomd https://github.com/facebookincubator/oomd
    - swiley 7y ago
      With no script it’s rare for Firefox to eat all my memory (with 4GB of ram) more often It’s GCC. I really just need to be involved when that happens, going and killing things isn’t the answer. One thing I do want to try is sand boxing the browser using UML (which I’ve been using a lot of lately.) This way since it gets its own kernel I can totally limit everything.
- ploxiln 7y ago
  It just has to kill something. The choice is between killing some things, or freezing everything, for way too long (many minutes). I'll agree that Windows and macOS aren't perfect here. But I expect better, that's why I prefer Linux. (Better networking, better filesystems, better configuration and debugging of core system services, easier installation of stable C libraries for development or scripting ...)
  - slacka 7y ago
    Can speak for OS X, but I dual booted Win 7 with swap disabled on my first machine with a SSD. Windows would just kill the Chrome tab that was eating all my Ram and I'd move right along. Linux locks up and is totally unresponsive even if you manually trigger the OOM killer. Windows behavior is much, much better for desktop users.
- inetknght 7y ago
  > * You have to evict something. What's your choice if not swap or filesystem-backed pages?* How about fail to allocate more memory than the system actually has?
rjmunro 7y ago
> ... on practically every single context switch ... How frequently do context switches happen? Can this be reduced, so that there is less disk thrashing? I.e. whatever process needs lots of memory swapped in, just don't switch to it for up to maybe a few seconds, rather than milliseconds or whatever? This seems less bad than having the OOM killer just kill the thing.

ajross 7y ago

What's the other option, though? It gets a fault for a page somewhere, it needs memory, and it has none. You have to evict something. What's your choice if not swap or filesystem-backed pages? Systems have never operated well under true VM pressure. Not in the 90's, not now. When the working set goes beyond available memory performance falls off a cliff. And the report in the article doesn't seem to have a good comparison anyway. I mean, do we seriously believe that Windows or OS X can handle this condition (4G, no swap) better? I mean, swap is default for a reason.

throwaway2048 7y ago
I'd much rather have OOMkiller trigger than have my system sit completely unresponsive for huge chunks of time, while I slowly struggle to kill something.
- dahfizz 7y ago
  While this might add convenience when all you do is use a web browser, Linux is huge in the server space. Having the OS suddenly and somewhat unpredictability killing programs is absolutely not an option for most workloads.
  - quazeekotl 7y ago
    I don't think having a system sit completely unresponsive to anything for a huge amount of time is exactly optimal either.
  - _jal 7y ago
    Absolutely, which is why configurable knobs are things that exist. I already have to mess with the OOMkiller on my servers; I'm fine with fiddling with knobs on my desktop box too. This is Linux, after all. Wasn't one of the constant talking points in the war over the Init System That Shall Not Be Named that Linux needs to scale from embedded to big iron?
  - eikenberry 7y ago
    If you work in the cloud for any length of time you learn this is not true. Every system must be able to handle processes being killed unpredictably otherwise you will have problems. Better to deal with it and plan for it than expect a system to have 100% uptime. Because you never have 100% uptime.
  - TheDong 7y ago
    The oom killer already exists on servers and already can kill programs. If you want to turn off overcommit and have the system power off when it runs out of memory, the kernel allows that. Whatever knob they add will certainly be configurable, and ubuntu desktop can configure it one way while ubuntu server configures it the other, if it turns out people would prefer that. In practice, people running servers seem to want oom killers to kick in before the server barfs. One example of this is facebook's oomd [1]. I assure you, they're running that on their servers, not their web-browser-machines. [1]: https://github.com/facebookincubator/oomd https://github.com/facebookincubator/oomd
  - swiley 7y ago
    With no script it’s rare for Firefox to eat all my memory (with 4GB of ram) more often It’s GCC. I really just need to be involved when that happens, going and killing things isn’t the answer. One thing I do want to try is sand boxing the browser using UML (which I’ve been using a lot of lately.) This way since it gets its own kernel I can totally limit everything.
ploxiln 7y ago
It just has to kill something. The choice is between killing some things, or freezing everything, for way too long (many minutes). I'll agree that Windows and macOS aren't perfect here. But I expect better, that's why I prefer Linux. (Better networking, better filesystems, better configuration and debugging of core system services, easier installation of stable C libraries for development or scripting ...)
- slacka 7y ago
  Can speak for OS X, but I dual booted Win 7 with swap disabled on my first machine with a SSD. Windows would just kill the Chrome tab that was eating all my Ram and I'd move right along. Linux locks up and is totally unresponsive even if you manually trigger the OOM killer. Windows behavior is much, much better for desktop users.
inetknght 7y ago
> * You have to evict something. What's your choice if not swap or filesystem-backed pages?* How about fail to allocate more memory than the system actually has?

rjmunro 7y ago

> ... on practically every single context switch ... How frequently do context switches happen? Can this be reduced, so that there is less disk thrashing? I.e. whatever process needs lots of memory swapped in, just don't switch to it for up to maybe a few seconds, rather than milliseconds or whatever? This seems less bad than having the OOM killer just kill the thing.