8 ms·

How about when you can't ssh to the server because ... oom-killer killed sshd? Thankfully that system wasn't mine so I just got a laugh of this situation. Sti

How about when you can't ssh to the server because ... oom-killer killed sshd? Thankfully that system wasn't mine so I just got a laugh of this situation. Still, 20 years later I still can't wrap my head around why Linux still needs oom-killer when Windows can run just fine without such functionality.

oshiar53-0 5y ago
Hint: disable or minimize overcommit and you're good to go.
- justsomehnguy 5y ago
  That works for the systems I control. Most of the time I encounter OOM shenanigans on the systems which aren't mine.
- rrss 5y ago
  I dearly wish overcommit could be disabled per-cgroup, but AFAIK this is not possible. malloc failing seems like a much simpler indication of memory pressure than the alternatives like PSI.
  - zozbot234 5y ago
    AIUI, you can always use plain old ulimit to restrict virtual memory usage per-process, which has the same outcome as disabling overcommit.
    - rrss 5y ago
      yeah, for a single process. but I’m mostly interested in doing this for a process tree (like a browser), and I don’t think this would work well in that case, since the limit would apply to each process individually. this would work if there was a cgroup virtual address space controller, but I think those have been proposed but never merged.
- kasabali 5y ago
  In my short experience you can't disable overcommit if you're using Linux on desktop because programs using gtk/glib freak out when they can't malloc.
cozzyd 5y ago
Here is where something like systemd is really nice because sshd will get restarted! As for Windows, the big difference is no fork()
- zinekeller 5y ago
  > As for Windows, the big difference is no fork() Missing the whole picture. fork() isn't the real reason, it's that Windows guarantees memory or no memory while Linux by default overcommits (both of which isn't necessarily a bad thing - it's just different ways of managing a limited resource). NT could implement (and does, it's just not exposed in Win32 API) a non-overcommiting fork() and its memory algorithms will be the same afterwards.
  - ayende 5y ago
    The problem with `fork()` is that its usage requires over commit. Consider the scenario of a process with 51% of memory usage, what happens when it forks? The system pretty much _has_ to allow overcommit at this point, otherwise fork() will fail. Now, you can change the accounting to say that when the process modify a memory page (thus creating a copy), we'll check that there is enough memory _then_. That would move the "danger" of memory being yanked from underneath you. But that means that a process that calls fork() may then run into a problem some indeterministic time later. That is kind of a killer.
    - zinekeller 5y ago
      ERRORS The fork() function will fail if: [EAGAIN] The system lacked the necessary resources to create another process, or the system-imposed limit on the total number of processes under execution system-wide or by a single user {CHILD_MAX} would be exceeded. The fork() function may fail if: [ENOMEM] Insufficient storage space is available. The POSIX specification is clear on this: you can hard fail. Also, as an aside malloc() can also fail and the program must be able to handle that. Unfortunately for some programmers (mainly coming from GC-aware languages), they just assume that it won't fail. (Also, most rely on fork-exec but there's posix_spawn() for de novo execution). Again, NT does indeed have a fork function, however the Windows API doesn't expose that (and this is the hack Cygwin used when forking), but it's designed to fail instead of overcommit when all memory is exhausted.
rcxdude 5y ago
Windows for me frequenctly needs a pagefile 8x the size of my memory in order to service the committed memory, even though it's never touched in practice.
kasabali 5y ago
> why Linux still needs oom-killer when Windows can run just fine without such functionality My Windows 10 installation just BSODs when Firefox fills up the RAM if the page file is disabled. I'd much prefer the OOM killer.
- archerx 5y ago
  Chrome just crashes, it seems like the fault may lie with Firefox.
- gruez 5y ago
  >My Windows 10 installation just BSODs when Firefox fills up the RAM if the page file is disabled. I never had that happen. What actually happens is firefox just crashes, or the screen goes blank for a few seconds while the graphics driver resets.
  - kasabali 5y ago
    I get all of those, too. BSOD occurs less often when the system was already pressured with several cycles of the above. But even the graphics driver crashing is already bad enough. If there's not enough memory, offending programs should be crashing, not drivers. (and yes, ram sticks are good and verifed with memtest86 and testmem5)
- ferongr 5y ago
  Running with the page file disabled is bad. The system runs out of address space, hence the crashes.
- justsomehnguy 5y ago
  > if the page file is disabled Ouch. Never run Windows without the swap. It can be as small as the system allows (AFAIR 16Mb for Win7), but it should be enabled. While this is the obvious footgun the problem is nobody ever tests with the swap disabled. I've seen very weird behaviour without the swap, the most funny one was when some 3D game (sadly don't remember the name) internal timer went completely nuts without the swap and game was working just fine with the swap enabled.
weare138 5y ago
Windows uses a dynamic page file for swapping so it will just keep increasing the page file size until it hits the limits of the OS or config settings. You can do the same thing in OSs like Linux and BSD but most distros still default to fixed swap partitions. For example macOS uses page/swap files instead of a swap partition by default. It's just a design choice. Both have advantages and disadvantages. Linux and BSD distros usually are more aggressive about kicking out applications in favor of keeping the system stable but the trade off with OSs like Windows is misbehaving applications can more easily take down the OS.

oshiar53-0 5y ago

Hint: disable or minimize overcommit and you're good to go.

justsomehnguy 5y ago
That works for the systems I control. Most of the time I encounter OOM shenanigans on the systems which aren't mine.
rrss 5y ago
I dearly wish overcommit could be disabled per-cgroup, but AFAIK this is not possible. malloc failing seems like a much simpler indication of memory pressure than the alternatives like PSI.
- zozbot234 5y ago
  AIUI, you can always use plain old ulimit to restrict virtual memory usage per-process, which has the same outcome as disabling overcommit.
  - rrss 5y ago
    yeah, for a single process. but I’m mostly interested in doing this for a process tree (like a browser), and I don’t think this would work well in that case, since the limit would apply to each process individually. this would work if there was a cgroup virtual address space controller, but I think those have been proposed but never merged.
kasabali 5y ago
In my short experience you can't disable overcommit if you're using Linux on desktop because programs using gtk/glib freak out when they can't malloc.

cozzyd 5y ago

Here is where something like systemd is really nice because sshd will get restarted! As for Windows, the big difference is no fork()

zinekeller 5y ago
> As for Windows, the big difference is no fork() Missing the whole picture. fork() isn't the real reason, it's that Windows guarantees memory or no memory while Linux by default overcommits (both of which isn't necessarily a bad thing - it's just different ways of managing a limited resource). NT could implement (and does, it's just not exposed in Win32 API) a non-overcommiting fork() and its memory algorithms will be the same afterwards.
- ayende 5y ago
  The problem with `fork()` is that its usage requires over commit. Consider the scenario of a process with 51% of memory usage, what happens when it forks? The system pretty much _has_ to allow overcommit at this point, otherwise fork() will fail. Now, you can change the accounting to say that when the process modify a memory page (thus creating a copy), we'll check that there is enough memory _then_. That would move the "danger" of memory being yanked from underneath you. But that means that a process that calls fork() may then run into a problem some indeterministic time later. That is kind of a killer.
  - zinekeller 5y ago
    ERRORS The fork() function will fail if: [EAGAIN] The system lacked the necessary resources to create another process, or the system-imposed limit on the total number of processes under execution system-wide or by a single user {CHILD_MAX} would be exceeded. The fork() function may fail if: [ENOMEM] Insufficient storage space is available. The POSIX specification is clear on this: you can hard fail. Also, as an aside malloc() can also fail and the program must be able to handle that. Unfortunately for some programmers (mainly coming from GC-aware languages), they just assume that it won't fail. (Also, most rely on fork-exec but there's posix_spawn() for de novo execution). Again, NT does indeed have a fork function, however the Windows API doesn't expose that (and this is the hack Cygwin used when forking), but it's designed to fail instead of overcommit when all memory is exhausted.

rcxdude 5y ago

Windows for me frequenctly needs a pagefile 8x the size of my memory in order to service the committed memory, even though it's never touched in practice.

kasabali 5y ago

> why Linux still needs oom-killer when Windows can run just fine without such functionality My Windows 10 installation just BSODs when Firefox fills up the RAM if the page file is disabled. I'd much prefer the OOM killer.

archerx 5y ago
Chrome just crashes, it seems like the fault may lie with Firefox.
gruez 5y ago
>My Windows 10 installation just BSODs when Firefox fills up the RAM if the page file is disabled. I never had that happen. What actually happens is firefox just crashes, or the screen goes blank for a few seconds while the graphics driver resets.
- kasabali 5y ago
  I get all of those, too. BSOD occurs less often when the system was already pressured with several cycles of the above. But even the graphics driver crashing is already bad enough. If there's not enough memory, offending programs should be crashing, not drivers. (and yes, ram sticks are good and verifed with memtest86 and testmem5)
ferongr 5y ago
Running with the page file disabled is bad. The system runs out of address space, hence the crashes.
justsomehnguy 5y ago
> if the page file is disabled Ouch. Never run Windows without the swap. It can be as small as the system allows (AFAIR 16Mb for Win7), but it should be enabled. While this is the obvious footgun the problem is nobody ever tests with the swap disabled. I've seen very weird behaviour without the swap, the most funny one was when some 3D game (sadly don't remember the name) internal timer went completely nuts without the swap and game was working just fine with the swap enabled.

weare138 5y ago

Windows uses a dynamic page file for swapping so it will just keep increasing the page file size until it hits the limits of the OS or config settings. You can do the same thing in OSs like Linux and BSD but most distros still default to fixed swap partitions. For example macOS uses page/swap files instead of a swap partition by default. It's just a design choice. Both have advantages and disadvantages. Linux and BSD distros usually are more aggressive about kicking out applications in favor of keeping the system stable but the trade off with OSs like Windows is misbehaving applications can more easily take down the OS.