5 ms·

You recall incorrectly; swap is not needed. It's not just me who runs without it; Google production machines did for many years. "The behavior is often worse w

You recall incorrectly; swap is not needed. It's not just me who runs without it; Google production machines did for many years. "The behavior is often worse without swap" is more vague / subjective. I prefer that a process die cleanly than everything slow to a crawl semi-permanently. I've previously written about swap causing the latter: https://news.ycombinator.com/item?id=13715917 https://news.ycombinator.com/item?id=13715917 To some extent the bad behavior can happen even without swap because non-locked file-backed pages get paged in slowly also, but it seems significantly worse with swap. zram is a decent idea though. I use it on my memory-limited Raspberry Pi machines.

tored 6y ago
I'm surprised that the swap implementation (or distros swap configuration) is so bad on Linux. I have a fresh Ubuntu 20.10 desktop installation and a few times it has come to slow crawl where you can't do anything, can't escape to tty or ssh, reboot is basically the only option. I recently learned searching the internets that it was the swapfile configuration. My 16GB RAM machine got a 2GB swapfile from the default installation so I doubled it to 4GB, but today it almost got full again, luckily I was just about to close a big memory hog and that saved me. I have now increased the swapfile to 6GB. If Ubuntu is giving default installations 2GB of swapfile on 16GB of RAM, shouldn't this happen to lots of users quite frequently? How many users are technical enough to understand this and fix it? How can this be a good solution? I can't recall that I ever got in trouble on Windows because of the swapfile.
- michael-ax 6y ago
  I had zero problems without swap-files for a long time, thinking that 16gb was a lot of ram, and that i could manage. it wasn't and i couldn't. Now I run two swap partitions the size of physical memory on all machines. Turns out that after some weeks a good third of allocated ram lives in swap proving its usefulness .. and that its quick to turn the aux-swap partition into one for lvm snapshots and i suppose to act as part of the set to emulate'baloon files' too. tl;dr; bad defaults. go with the size of physical ram and get suspend/hibernate/hybrid working and tested too. use a 2nd ram-sized partition for snapshots/lvm and emergency space/lvm|zfs.
  - tored 6y ago
    That explains it, I noticed that if my machine was on for a while a degradation in performance happened, it was probably the swapfile then too, especially when it only has 2GB to work with. Yes, I probably need to set my swapfile to 16GB, I'm also investigating the swappiness setting, default 60 for desktop.
- MayeulC 6y ago
  I often went into similar issues before upgrading my RAM. SysRq+F is your friend :) https://en.m.wikipedia.org/wiki/Magic_SysRq_key https://en.m.wikipedia.org/wiki/Magic_SysRq_key That manually triggers OOM kill, making your system responsive again at the risk of killing something it shouldn't. At some point, I was probably using it five times a day. It got better when I started using swap (easier to recover when both full). I really recommend zram.
neop1x 6y ago
>> It's not just me who runs without it; Google production machines did for many years. But then you need other mechanisms monitoring "out of memory" situations. For example Kubernetes also require nodes to have swap disabled. But running workloads should have resource request/limits defined and then kubelet config specifies minimums on free disk and RAM memory. If there is not emough RAM, workloads are evicted to another worker node with more free memory automatically.
- scottlamb 6y ago
  I agree with your comment if you remove the words "but then" and "other". It's a good idea for orchestration software to detect/avoid unhealthy nodes and enforce the resource limits it uses for bin-packing. It's also a good idea for monitoring systems (by which I mean ones which do alerting and visualization) to track memory usage and pressure stalls. [1] I wouldn't say swap is a substitute for these in any way. A swapping machine is an unhealthy machine. [1] https://www.kernel.org/doc/html/latest/accounting/psi.html https://www.kernel.org/doc/html/latest/accounting/psi.html
MayeulC 6y ago
Well, I'm not inventing it. This phoronix article [1] leads to a LKML post [2]: > Yeah that's a known problem, made worse SSD's in fact, as they are able > to keep refaulting the last remaining file pages fast enough, so there > is still apparent progress in reclaim and OOM doesn't kick in. And so on. That's part of the reason there is a working effort towards creating a userspace OOM daemon. I'm grateful for that effort, since I'd rather have some apps crash than having my system unusable. However, there is an issue with the Linux kernel trashing. I couldn't find mention of it in that thread, but it's mentioned multiple times in passing (as well as in the phoronix comments) that the system behaves worse without swap. It's also noted as one of the reproduction steps. I agree that it's vague and subjective, but I'm pretty sure I've seen some serious discussion of the topic. There are a few sources that point in that direction [3-4], but I haven't seen anything conclusive. [1]: https://www.phoronix.com/scan.php?page=news_item&px=Linux-Does-Bad-Low-RAM https://www.phoronix.com/scan.php?page=news_item&px=Linux-Do... [2]: https://lkml.org/lkml/2019/8/5/416 https://lkml.org/lkml/2019/8/5/416 [3]: https://unix.stackexchange.com/a/28176 https://unix.stackexchange.com/a/28176 [4]: https://serverfault.com/questions/255661/linux-oom-disk-i-o-also-swap-what-is-it-good-for https://serverfault.com/questions/255661/linux-oom-disk-i-o-...

tored 6y ago

I'm surprised that the swap implementation (or distros swap configuration) is so bad on Linux. I have a fresh Ubuntu 20.10 desktop installation and a few times it has come to slow crawl where you can't do anything, can't escape to tty or ssh, reboot is basically the only option. I recently learned searching the internets that it was the swapfile configuration. My 16GB RAM machine got a 2GB swapfile from the default installation so I doubled it to 4GB, but today it almost got full again, luckily I was just about to close a big memory hog and that saved me. I have now increased the swapfile to 6GB. If Ubuntu is giving default installations 2GB of swapfile on 16GB of RAM, shouldn't this happen to lots of users quite frequently? How many users are technical enough to understand this and fix it? How can this be a good solution? I can't recall that I ever got in trouble on Windows because of the swapfile.

michael-ax 6y ago
I had zero problems without swap-files for a long time, thinking that 16gb was a lot of ram, and that i could manage. it wasn't and i couldn't. Now I run two swap partitions the size of physical memory on all machines. Turns out that after some weeks a good third of allocated ram lives in swap proving its usefulness .. and that its quick to turn the aux-swap partition into one for lvm snapshots and i suppose to act as part of the set to emulate'baloon files' too. tl;dr; bad defaults. go with the size of physical ram and get suspend/hibernate/hybrid working and tested too. use a 2nd ram-sized partition for snapshots/lvm and emergency space/lvm|zfs.
- tored 6y ago
  That explains it, I noticed that if my machine was on for a while a degradation in performance happened, it was probably the swapfile then too, especially when it only has 2GB to work with. Yes, I probably need to set my swapfile to 16GB, I'm also investigating the swappiness setting, default 60 for desktop.
MayeulC 6y ago
I often went into similar issues before upgrading my RAM. SysRq+F is your friend :) https://en.m.wikipedia.org/wiki/Magic_SysRq_key https://en.m.wikipedia.org/wiki/Magic_SysRq_key That manually triggers OOM kill, making your system responsive again at the risk of killing something it shouldn't. At some point, I was probably using it five times a day. It got better when I started using swap (easier to recover when both full). I really recommend zram.

neop1x 6y ago

>> It's not just me who runs without it; Google production machines did for many years. But then you need other mechanisms monitoring "out of memory" situations. For example Kubernetes also require nodes to have swap disabled. But running workloads should have resource request/limits defined and then kubelet config specifies minimums on free disk and RAM memory. If there is not emough RAM, workloads are evicted to another worker node with more free memory automatically.

scottlamb 6y ago
I agree with your comment if you remove the words "but then" and "other". It's a good idea for orchestration software to detect/avoid unhealthy nodes and enforce the resource limits it uses for bin-packing. It's also a good idea for monitoring systems (by which I mean ones which do alerting and visualization) to track memory usage and pressure stalls. [1] I wouldn't say swap is a substitute for these in any way. A swapping machine is an unhealthy machine. [1] https://www.kernel.org/doc/html/latest/accounting/psi.html https://www.kernel.org/doc/html/latest/accounting/psi.html

MayeulC 6y ago

Well, I'm not inventing it. This phoronix article [1] leads to a LKML post [2]: > Yeah that's a known problem, made worse SSD's in fact, as they are able > to keep refaulting the last remaining file pages fast enough, so there > is still apparent progress in reclaim and OOM doesn't kick in. And so on. That's part of the reason there is a working effort towards creating a userspace OOM daemon. I'm grateful for that effort, since I'd rather have some apps crash than having my system unusable. However, there is an issue with the Linux kernel trashing. I couldn't find mention of it in that thread, but it's mentioned multiple times in passing (as well as in the phoronix comments) that the system behaves worse without swap. It's also noted as one of the reproduction steps. I agree that it's vague and subjective, but I'm pretty sure I've seen some serious discussion of the topic. There are a few sources that point in that direction [3-4], but I haven't seen anything conclusive. [1]: https://www.phoronix.com/scan.php?page=news_item&px=Linux-Does-Bad-Low-RAM https://www.phoronix.com/scan.php?page=news_item&px=Linux-Do... [2]: https://lkml.org/lkml/2019/8/5/416 https://lkml.org/lkml/2019/8/5/416 [3]: https://unix.stackexchange.com/a/28176 https://unix.stackexchange.com/a/28176 [4]: https://serverfault.com/questions/255661/linux-oom-disk-i-o-also-swap-what-is-it-good-for https://serverfault.com/questions/255661/linux-oom-disk-i-o-...