4 ms·

I hit this bug yesterday on my laptop (16GB of RAM / 1GB of swap) with 2 instances of Firefox (about 60 tabs), Slack, Insomnia (Electron-based Postman clone) an

I hit this bug yesterday on my laptop (16GB of RAM / 1GB of swap) with 2 instances of Firefox (about 60 tabs), Slack, Insomnia (Electron-based Postman clone) and a couple of `node` processes watching and transpiling.. stuff. `kswapd0` was running at 100% CPU, I guess trying to free up some RAM by moving things to swap (the swap partition was full by this point). Luckily I managed to recover the system by switching to another tty and killing kswapd0 and the node instances. Sometimes instructing the kernel to clear its caches helps: `echo 1 | sudo tee /proc/sys/vm/drop_caches` [1] [1]: https://serverfault.com/questions/696156/kswapd-often-uses-100-cpu-when-swap-is-in-use/696185#696185 https://serverfault.com/questions/696156/kswapd-often-uses-1...

IshKebab 7y ago
I wouldn't enable swap on a desktop Linux system. When you run out of memory if you have swap the system grinds to a halt and you pretty much can't do anything to save it, or at least it is a battle. Without swap it just kills processed until there is enough memory, which is what you would have done anyway! I think the main annoyance with Linux here is that in Windows you get to choose what to kill, whereas in Linux it can't really communicate with you (because the kernel doesn't know about such modern things as GUIs) so it had to pick more or less randomly.
- caf 7y ago
  It's not really that random. To see what the OOM killer heuristic currently considers its top 5 targets on your machine: for P in /proc/[0-9]*; do echo $(cat $P/oom_score) $(cat $P/comm); done | sort -n | tail -5 For me right now that shows 4 "Web Content" processes (firefox tabs) and a firefox-esr. That seems to check out.
- michaelmrose 7y ago
  Having some swap with low swappiness allows the system to page out pages that are unlikely to be used or haven't been used in a long time but aren't backed by a file. In normal conditions this frees up memory for more useful data and helps you avoid getting to perverse conditions.
  - IshKebab 7y ago
    That's how it should work yes. Unfortunately it doesn't actually happen like that, hence this entire discussion.
    - michaelmrose 7y ago
      It actually DOES happen like this. When the entire working set for actively used apps fits in memory swap lets the system page out things that are little used. This works perfectly fine. This is to say that swapping out little used stuff delays the point where you are actually out of memory and performance goes straight to hell. This means the optimal arrangement for desktop use is some swap and low swappiness. One could imagine that perhaps something like https://github.com/rfjakob/earlyoom https://github.com/rfjakob/earlyoom Might be an easier route to better behavior especially as you can more easily tell it what it ought to kill. The behavior of the kernel could probably be improved but it is probably inherently lacking the data required to make a truly optimal choice along with a gui to communicate with users. Going forward possibly desktop oriented distros should probably come with some out of the box gui to handle this situation built into their graphical environments before it gets to the point of dysfunction.
kasabali 7y ago
> I guess trying to free up some RAM by moving things to swap (the swap partition was full by this point) In that case, it cannot be moving things the swap since it was alreay full. What did probably happen is that since the swap was already full and not enough it started to trash the system by paging out the executable files in the memory. Ironically enough you'd probably have much smaller risk of trashing had the system have more swap space.

IshKebab 7y ago

I wouldn't enable swap on a desktop Linux system. When you run out of memory if you have swap the system grinds to a halt and you pretty much can't do anything to save it, or at least it is a battle. Without swap it just kills processed until there is enough memory, which is what you would have done anyway! I think the main annoyance with Linux here is that in Windows you get to choose what to kill, whereas in Linux it can't really communicate with you (because the kernel doesn't know about such modern things as GUIs) so it had to pick more or less randomly.

caf 7y ago
It's not really that random. To see what the OOM killer heuristic currently considers its top 5 targets on your machine: for P in /proc/[0-9]*; do echo $(cat $P/oom_score) $(cat $P/comm); done | sort -n | tail -5 For me right now that shows 4 "Web Content" processes (firefox tabs) and a firefox-esr. That seems to check out.
michaelmrose 7y ago
Having some swap with low swappiness allows the system to page out pages that are unlikely to be used or haven't been used in a long time but aren't backed by a file. In normal conditions this frees up memory for more useful data and helps you avoid getting to perverse conditions.
- IshKebab 7y ago
  That's how it should work yes. Unfortunately it doesn't actually happen like that, hence this entire discussion.
  - michaelmrose 7y ago
    It actually DOES happen like this. When the entire working set for actively used apps fits in memory swap lets the system page out things that are little used. This works perfectly fine. This is to say that swapping out little used stuff delays the point where you are actually out of memory and performance goes straight to hell. This means the optimal arrangement for desktop use is some swap and low swappiness. One could imagine that perhaps something like https://github.com/rfjakob/earlyoom https://github.com/rfjakob/earlyoom Might be an easier route to better behavior especially as you can more easily tell it what it ought to kill. The behavior of the kernel could probably be improved but it is probably inherently lacking the data required to make a truly optimal choice along with a gui to communicate with users. Going forward possibly desktop oriented distros should probably come with some out of the box gui to handle this situation built into their graphical environments before it gets to the point of dysfunction.

kasabali 7y ago

> I guess trying to free up some RAM by moving things to swap (the swap partition was full by this point) In that case, it cannot be moving things the swap since it was alreay full. What did probably happen is that since the swap was already full and not enough it started to trash the system by paging out the executable files in the memory. Ironically enough you'd probably have much smaller risk of trashing had the system have more swap space.