3 ms·

Because simplistic policies like this can never satisfy everyone. On a server the process using the most memory might be the mail server so killing it is a bad

Because simplistic policies like this can never satisfy everyone. On a server the process using the most memory might be the mail server so killing it is a bad idea... unless it normally never uses more than 2GB and for some reason it’s eating 14GB at the time. You have to define quality of service, set relative priorities within those categories, be able to correlate memory usage with load to detect abnormal spikes vs normal spikes, etc. For example a daemon’s memory usage is probably correlated with its open socket count. That in turn relies on a system much smarter than init scripts that understands the system is under memory load so re-starting a daemon that was just killed might not be appropriate (which gets into policy questions too). For interactive use things are even more complicated; you need a mechanism for the active application being used by the user to donate its ultra-high priority to any daemons or child processes it is talking to. If image_convert is running because I initiated it as the user then it should get more memory (and CPU/IO). If it is running because my desktop window manager is refreshing it’s icon cache then it should get its priority smashed hard - if the system is under pressure or burning battery it should even be killed and prevented from restarting temporarily. Who is going to do all the work in the kernel, userspace libraries, then get all the apps to properly tag their threads and requests with the correct priority? tl;dr: setting policy is hard and anyway it becomes the herding cats problem.

pmontra 7y ago
I think that all processes are needed on the typical server, monitoring included. If one often runs out of RAM, buy more physical or virtual RAM. Furthermore we probably have more than one server if we really care about the service they offer. One server crashes, another one spins up, the others copie with the load meanwhile. The option to kill processes is more interesting on personal machines, where sometimes adding RAM is not even an option. My anecdotal experience with a laptop without swap since 2014 (16 GB, then 32) is that I got near the limit a couple of times. I wouldn't mind if Linux killed any of the browsers, emacs (which is tiny nowadays), Thunderbird. They can recover. Maybe kill even databases. Leave me XOrg, the desktop environment and a terminal to investigate the problem.
- AstralStorm 7y ago
  You do understand that no amount of RAM will fix a buggy or leaky application? That buying more ram has weeks of latency or is altogether not possible by hardware or politics? If you know you want to keep bare Xorg alive with no window manager, taskbar or desktop, there's oom_adj magic knob, accessible by systemd. And memory cgroups. But on a shell server, you actually want to kill that Xorg but preserve background tasks.
  - pmontra 7y ago
    I understand that an out of control application can't be made to behave by any amount of RAM. But it's out of control, so if it dies it's (kind of) OK. I saw memory leaks in production and the usual workaround is rebooting the application or the OS once per day. Sometimes more than that. And then warming up the caches ect. On a shell server there is no Xorg running, not even on machines with a partial of complete X11 installation because (example) they need fonts to generate PDFs. I just checked a few of them. On a desktop/laptop Xorg is the last thing I want to hang or be killed because it makes it harder to check what went wrong. I'll look into oom_adj, thanks.
    - pmontra 7y ago
      I googled oom_adj and found https://dev.to/rrampage/surviving-the-linux-oom-killer-2ki9 https://dev.to/rrampage/surviving-the-linux-oom-killer-2ki9 I run the script to display the oom_score_adj of the processes running on my laptop. It's 0 for all of them except the blink based browers (including Slack). For those processes it's either 200 or 300. It means that they'll be the first ones to go. Apparently their developers played nice with the system.

pmontra 7y ago

I think that all processes are needed on the typical server, monitoring included. If one often runs out of RAM, buy more physical or virtual RAM. Furthermore we probably have more than one server if we really care about the service they offer. One server crashes, another one spins up, the others copie with the load meanwhile. The option to kill processes is more interesting on personal machines, where sometimes adding RAM is not even an option. My anecdotal experience with a laptop without swap since 2014 (16 GB, then 32) is that I got near the limit a couple of times. I wouldn't mind if Linux killed any of the browsers, emacs (which is tiny nowadays), Thunderbird. They can recover. Maybe kill even databases. Leave me XOrg, the desktop environment and a terminal to investigate the problem.

AstralStorm 7y ago
You do understand that no amount of RAM will fix a buggy or leaky application? That buying more ram has weeks of latency or is altogether not possible by hardware or politics? If you know you want to keep bare Xorg alive with no window manager, taskbar or desktop, there's oom_adj magic knob, accessible by systemd. And memory cgroups. But on a shell server, you actually want to kill that Xorg but preserve background tasks.
- pmontra 7y ago
  I understand that an out of control application can't be made to behave by any amount of RAM. But it's out of control, so if it dies it's (kind of) OK. I saw memory leaks in production and the usual workaround is rebooting the application or the OS once per day. Sometimes more than that. And then warming up the caches ect. On a shell server there is no Xorg running, not even on machines with a partial of complete X11 installation because (example) they need fonts to generate PDFs. I just checked a few of them. On a desktop/laptop Xorg is the last thing I want to hang or be killed because it makes it harder to check what went wrong. I'll look into oom_adj, thanks.
  - pmontra 7y ago
    I googled oom_adj and found https://dev.to/rrampage/surviving-the-linux-oom-killer-2ki9 https://dev.to/rrampage/surviving-the-linux-oom-killer-2ki9 I run the script to display the oom_score_adj of the processes running on my laptop. It's 0 for all of them except the blink based browers (including Slack). For those processes it's either 200 or 300. It means that they'll be the first ones to go. Apparently their developers played nice with the system.