8 ms·

I'm so happy someone has made a clear bug report here. Because damn, this is a thing.

throwaway2048 7y ago
Yep, even with no swap whatsoever, performance is completely trashed (talking even the mouse lags for 30+ seconds at a time) for like a solid 5+ minutes before the oomkiller triggers, with swap, you might as well just reboot because the system will take perhaps an hour to start responding. Linux is completely useless with ram that is almost full in a way that OSX and windows absolutely are not.
- mort96 7y ago
  Yeah, any time I end up in that situation I just hold the power button and prey to the journaling gods. It's a serious issue, I'm extremely glad it looks like there's actually some progress on fixing it.
  - minitech 7y ago
    Enable SysRq and use SysRq+F. https://news.ycombinator.com/item?id=16561746 https://news.ycombinator.com/item?id=16561746
- blattimwind 7y ago
  Linux absolutely, positively, requires a decent chunk of free memory because the kernel's algorithms simply do not work and in my 15+ years of Linux experience they NEVER worked. If Linux starts thrashing, and it does so easily, then the box takes what surely is an uncomputable amount of time to recover. One way to make relatively sure that this is always the case is to use a userspace daemon like earlyoom. Though to stay fair all desktop OS'es behave badly when put under memory pressure, it's just that Linux is an order of magnitude or so worse.
  - maximente 7y ago
    are any of the BSDs better in your experience?
    - myrandomcomment 7y ago
      Well my memory for BSD is not fully clear, but when I looked at this back in 2008 BSD handled it better then Linux. The king of sorting this was Solaris. It was rock solid. I would still argue that Solaris is better then Linux as a server, but it does not matter as Linux won. Also, I want my Amgia back :)
      - ComputerGuru 7y ago
        I only ditched the Solaris train for FreeBSD when all hope was gone after Schwartz’s reign came to an end. Solaris got so many things right architecturally that are damn hard to shoehorn in after the fact. Until today, I don’t know if any other *nix has sorted out ABI compatibility across architectures, which allowed running applications cross-compiled for another target to run against the memory-resident kernel without virtualization (only instruction emulation when needed). In practice, it meant that you could run “universal” x86 binaries against both the x86 and x86_64 kernels, even calling into drivers (!!), with guaranteed compatibility. The last I had checked, FreeBSD had made it a goal to unify all numeric values of IOCTLs across architectures, but I don’t believe they are there yet.
        floatboth 7y ago
        Well, a 32-bit Linux binary on 64-bit FreeBSD can call into the GPU driver and render 3D, at least :) But I don't think anything is guaranteed for sure. 32-bit crap is not exactly a priority, haha. On the other hand, syscalls across architectures of the same bitness are exactly the same, minus the newer architectures just not having some historical abominations like sbrk. (IIRC, syscalls are different between even aarch64 and amd64 on Linux, which is just… why?!?)
        7y ago
- floatingatoll 7y ago
  Does the time until oomkiller change dramatically on spinning metal versus SATA SSD versus NVMe, in a swap-off scenario?
- brendangregg 7y ago
  I don't think that's a fair comparison: Do you normally run OSX with the swap files completely disabled, or Windows with the pagefile completely disabled? That's what this bug is describing. I'd bet things get pretty nasty on OSX and Windows too, if you tried that. Perhaps the real bug is that Linux distros make it easy to run swapless.
  - throwaway2048 7y ago
    The behavior is even worse if you leave swap enabled, as I already detailed in my post...
  - Aaargh20318 7y ago
    Isn’t iOS basically a flavor of macOS that runs without swap ?
    - earenndil 7y ago
      Yes, but it's also a heavily integrated environment that aggressively quits background programs on memory pressure.
      - codedokode 7y ago
        That is the only working solution with HTML and JS based applications or apps using GC. Applications should save the state and quit (or be ready to quit) instead of using memory in backround mode.
      - Aaargh20318 7y ago
        But isn't that exactly what the linked article advocates Linux should also do ? Before quitting background applications it first sends them a request to free memory, in a well-behaved iOS program you use this to clean up your caches and ensure your don't use more RAM than you absolutely need. You should also suspend your state to disk when your app is backgrounded so you can just continue where you left off if your app is killed. Many macOS apps also do this, you can forcefully restart a Mac and after a reboot it'll restore your session to pretty much the exact state you left it in, including any open 'unsaved' files. Linux could implement a similar mechanism to signal apps to clean themselves up and maybe a 'save your state, you're about to get killed' signal.
jolmg 7y ago
Yeah. I didn't really think of it as a bug at first, but I'm glad someone called it that. I wish the system would just kill the browser or low priority processes instead of freezing everything in an instant.
- magicalhippo 7y ago
  Why does it have to kill the browser? Why can't it tell it "nope, no more memory for you" before it's all gone?
  - viraptor 7y ago
    It can. System-wide it's one extreme or another. With overcommit enabled, you'll pretty much never get refused. With overcommit disabled, you'll get refusals was soon as you reach the max memory, which means lots of mapped pages wasting space. The middle ground is your own config unfortunately - cgroups can limit available memory, but you'll have to set it up by hand.
    - Annatar 7y ago
      An operating system should never overallocate memory because one cannot build reliable applications and infrastructure on top of a kernel which is lying to the application.
      - angelsl 7y ago
        Then you disable overcommit. For the general case though, because of the way programs have been written, it is easier to have overcommit on.
      - rcxdude 7y ago
        Windows will never overcommit memory. As a result on my PC unless I dedicate a substantial (over 20%) fraction of my drive to swap (most of which will never, ever be touched) I will 'run out of RAM' far before even half of the physical RAM in my PC has been used. This seems extremely wasteful.
tbrock 7y ago
Agree, but this is only a decent bug report. A better one would just make a C program that mallocs more memory than is available. The "open enough tabs so that it crashes part" is like "banana for scale", it is incredible unspecific. You could probably open HackerNews 50x more than espn.com
- curryst 7y ago
  Why? The situation in which it occurs is unambiguous. Allocate more memory than is available and watch it fail less than gracefully. What you're suggesting is merely technical gatekeeping; there is nothing to gain from writing a special purpose OOM-causer, no less specifying that it must be written in C.
emmelaich 7y ago
There are _numerous_ similar bugs reported in the last 10+ years, in redhat's bugzilla, ubuntu, suse and others. Almost all of them get closed because they get old and no-one is willing to do the necessary work to refine the bug report to dependable reproducibility. And that's not surprising really; it's hard, time-consuming work which may be obsoleted on the next kernel version. That said, some have written memory stressors and have been able to crash or stall a machine but it's still a bit hit and miss.

throwaway2048 7y ago

Yep, even with no swap whatsoever, performance is completely trashed (talking even the mouse lags for 30+ seconds at a time) for like a solid 5+ minutes before the oomkiller triggers, with swap, you might as well just reboot because the system will take perhaps an hour to start responding. Linux is completely useless with ram that is almost full in a way that OSX and windows absolutely are not.

mort96 7y ago
Yeah, any time I end up in that situation I just hold the power button and prey to the journaling gods. It's a serious issue, I'm extremely glad it looks like there's actually some progress on fixing it.
- minitech 7y ago
  Enable SysRq and use SysRq+F. https://news.ycombinator.com/item?id=16561746 https://news.ycombinator.com/item?id=16561746
blattimwind 7y ago
Linux absolutely, positively, requires a decent chunk of free memory because the kernel's algorithms simply do not work and in my 15+ years of Linux experience they NEVER worked. If Linux starts thrashing, and it does so easily, then the box takes what surely is an uncomputable amount of time to recover. One way to make relatively sure that this is always the case is to use a userspace daemon like earlyoom. Though to stay fair all desktop OS'es behave badly when put under memory pressure, it's just that Linux is an order of magnitude or so worse.
- maximente 7y ago
  are any of the BSDs better in your experience?
  - myrandomcomment 7y ago
    Well my memory for BSD is not fully clear, but when I looked at this back in 2008 BSD handled it better then Linux. The king of sorting this was Solaris. It was rock solid. I would still argue that Solaris is better then Linux as a server, but it does not matter as Linux won. Also, I want my Amgia back :)
    - ComputerGuru 7y ago
      I only ditched the Solaris train for FreeBSD when all hope was gone after Schwartz’s reign came to an end. Solaris got so many things right architecturally that are damn hard to shoehorn in after the fact. Until today, I don’t know if any other *nix has sorted out ABI compatibility across architectures, which allowed running applications cross-compiled for another target to run against the memory-resident kernel without virtualization (only instruction emulation when needed). In practice, it meant that you could run “universal” x86 binaries against both the x86 and x86_64 kernels, even calling into drivers (!!), with guaranteed compatibility. The last I had checked, FreeBSD had made it a goal to unify all numeric values of IOCTLs across architectures, but I don’t believe they are there yet.
      - floatboth 7y ago
        Well, a 32-bit Linux binary on 64-bit FreeBSD can call into the GPU driver and render 3D, at least :) But I don't think anything is guaranteed for sure. 32-bit crap is not exactly a priority, haha. On the other hand, syscalls across architectures of the same bitness are exactly the same, minus the newer architectures just not having some historical abominations like sbrk. (IIRC, syscalls are different between even aarch64 and amd64 on Linux, which is just… why?!?)
        7y ago
floatingatoll 7y ago
Does the time until oomkiller change dramatically on spinning metal versus SATA SSD versus NVMe, in a swap-off scenario?
brendangregg 7y ago
I don't think that's a fair comparison: Do you normally run OSX with the swap files completely disabled, or Windows with the pagefile completely disabled? That's what this bug is describing. I'd bet things get pretty nasty on OSX and Windows too, if you tried that. Perhaps the real bug is that Linux distros make it easy to run swapless.
- throwaway2048 7y ago
  The behavior is even worse if you leave swap enabled, as I already detailed in my post...
- Aaargh20318 7y ago
  Isn’t iOS basically a flavor of macOS that runs without swap ?
  - earenndil 7y ago
    Yes, but it's also a heavily integrated environment that aggressively quits background programs on memory pressure.
    - codedokode 7y ago
      That is the only working solution with HTML and JS based applications or apps using GC. Applications should save the state and quit (or be ready to quit) instead of using memory in backround mode.
    - Aaargh20318 7y ago
      But isn't that exactly what the linked article advocates Linux should also do ? Before quitting background applications it first sends them a request to free memory, in a well-behaved iOS program you use this to clean up your caches and ensure your don't use more RAM than you absolutely need. You should also suspend your state to disk when your app is backgrounded so you can just continue where you left off if your app is killed. Many macOS apps also do this, you can forcefully restart a Mac and after a reboot it'll restore your session to pretty much the exact state you left it in, including any open 'unsaved' files. Linux could implement a similar mechanism to signal apps to clean themselves up and maybe a 'save your state, you're about to get killed' signal.

jolmg 7y ago

Yeah. I didn't really think of it as a bug at first, but I'm glad someone called it that. I wish the system would just kill the browser or low priority processes instead of freezing everything in an instant.

magicalhippo 7y ago
Why does it have to kill the browser? Why can't it tell it "nope, no more memory for you" before it's all gone?
- viraptor 7y ago
  It can. System-wide it's one extreme or another. With overcommit enabled, you'll pretty much never get refused. With overcommit disabled, you'll get refusals was soon as you reach the max memory, which means lots of mapped pages wasting space. The middle ground is your own config unfortunately - cgroups can limit available memory, but you'll have to set it up by hand.
  - Annatar 7y ago
    An operating system should never overallocate memory because one cannot build reliable applications and infrastructure on top of a kernel which is lying to the application.
    - angelsl 7y ago
      Then you disable overcommit. For the general case though, because of the way programs have been written, it is easier to have overcommit on.
    - rcxdude 7y ago
      Windows will never overcommit memory. As a result on my PC unless I dedicate a substantial (over 20%) fraction of my drive to swap (most of which will never, ever be touched) I will 'run out of RAM' far before even half of the physical RAM in my PC has been used. This seems extremely wasteful.

tbrock 7y ago

Agree, but this is only a decent bug report. A better one would just make a C program that mallocs more memory than is available. The "open enough tabs so that it crashes part" is like "banana for scale", it is incredible unspecific. You could probably open HackerNews 50x more than espn.com

curryst 7y ago
Why? The situation in which it occurs is unambiguous. Allocate more memory than is available and watch it fail less than gracefully. What you're suggesting is merely technical gatekeeping; there is nothing to gain from writing a special purpose OOM-causer, no less specifying that it must be written in C.

emmelaich 7y ago

There are _numerous_ similar bugs reported in the last 10+ years, in redhat's bugzilla, ubuntu, suse and others. Almost all of them get closed because they get old and no-one is willing to do the necessary work to refine the bug report to dependable reproducibility. And that's not surprising really; it's hard, time-consuming work which may be obsoleted on the next kernel version. That said, some have written memory stressors and have been able to crash or stall a machine but it's still a bit hit and miss.