5 ms·
It does though? $ uname -a Linux mimeomia 5.9.11 #1-NixOS SMP Tue Nov 24 12:39:15 UTC 2020 x86_64 GNU/Linux $ cat main.c #include <stdio.h>
by q3k 6y ago
It does though?
$ uname -a
Linux mimeomia 5.9.11 #1-NixOS SMP Tue Nov 24 12:39:15 UTC 2020 x86_64 GNU/Linux
$ cat main.c
#include <stdio.h>
#include <stdlib.h>
int main() {
printf("malloc: %lx\n", malloc(128LL << 30)); // 128GiB
perror("malloc err");
}
$ gcc main.c -o main && ./main
malloc: 0
malloc err: Cannot allocate memory
- dragontamer 6y agoThe original poster is almost certainly talking about Linux's overcommit policy: http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommi... You can't adopt a defensive / offensive stance with regards to out-of-memory issues, because on Linux... malloc doesn't fail when you run out of physical memory. Your "error" occurs on a pointer-dereference, which could be anywhere in your program. Once more memory is dereferenced than the system has capacity for, the OOM-killer is activated. There's no defensive stance possible, aside from setting overcommit to "Type2" (disabling the overcommit behavior of the Kernel). But that's somewhat non-standard, and possibly non-helpful, since some applications depend on overcommit.
- theamk 6y agoDisabling overcommit is totally feasible on some embedded boxes. And if you don’t want this for some reason, you can also mlock/madvice/manually touch all the memory to get errors right away as well. Yes, overcommit makes things tricker, but it is hardly “impossible”
- dragontamer 6y ago> you can also mlock/madvice/manually touch all the memory to get errors right away as well. All this does is run the OOM killer, which doesn't necessarily kill your process. Maybe its a process you didn't care about, maybe its a process you actually are communicating with and suddenly you're now in a weird deadlock state. Etc. etc. Even if YOU touch all your memory like you're supposed to, there's no guarantee that OTHER processes have touched their memory. So you touch memory unnecessarily: which means you're now officially reserving RAM (wasting it). Then, some OTHER process touches their memory and runs the OOM-killer. That wouldn't have happened if you just didn't touch your RAM. ----------- If Linux Overcommit is on default, you live with it and die with it. No point trying to fight OS-policy. Change the policy (aka: Overcommit to 2), and then carefully select applications to run with that follow the same overcommit policy as you.
- andi999 6y agoExactly. The misunderstanding a lot of ppl have about the OOM killer is, is that the process touching memory which cannot be fulfilled will be killed. This is not the case (for all practial reasonings one shd assume it kills random processes)
- theamk 6y agoIf you have an idea where do OOMs come from, you can (and should!) use "oom_score_adj"/"oom_adj" so that OOM kills processes are not random. There are many interesting strategies here: - Set your watchdog and logging to "never kill". Set watchdog to reboot on OOM, after uploading error report. This way, if the system OOMs, you can log the problems to your logging server, and then watchdog will reboot the system to recover. - If you have a "UI" process which can also eat all the memory, set it to "kill me first". Yes, the UI will crash, but the rest of system will restart it, and your updater would still work. - If you have separate resource-intensive process, you can set it to "kill me first" -- then the rest of the system can stay up and maybe even show a descriptive error message. - The latter approach works pretty nicely for general systems. Is your OOM killer activating because your build system started too many gcc instances? run it with "kill me first" setting -- it inherits to children processes, so you only build processes would be killed, the rest of the system (UI, browser, etc..) would all stay alive.
- andi999 6y agoThanks, this is interesting advice.