5 ms·
This is a wrong belief because a) SSDs make swap almost invisible, so you can have that escape ramp if something goes wrong b) SWAP space is not solely an escap
by bayindirh 1y ago
This is a wrong belief because a) SSDs make swap almost invisible, so you can have that escape ramp if something goes wrong b) SWAP space is not solely an escape ramp which RAM overflows into anymore.
In the age of microservices and cattle servers, reboot/reinstall might be cheap, but in the long run it is not. A long running server, albeit being cattle, is always a better solution because esp. with some excess RAM, the server "warms up" with all hot data cached and will be a low latency unit in your fleet, given you pay the required attention to your software development and service configuration.
Secondly, Kernel swaps out unused pages to SWAP, relieving pressure from RAM. So, SWAP is often used even if you fill 1% of your RAM. This allows for more hot data to be cached, allowing better resource utilization and performance in the long run.
So, eff it, we ball is never a good system administration strategy. Even if everything is ephemeral and can be rebooted in three seconds.
Sure, some things like Kubernetes forces "no SWAP, period" policies because it kills pods when pressure exceeds some value, but for more traditional setups, it's still valuable.
- gchamonlive 1y ago> SSDs make swap almost invisible It doesn't. SSDs came a long way but so did memory dies and buses, and with that the way programs work also changed as more and more they are able to fit their stacks and heaps on memory more often than not. I have had a problem with shellcheck that for some reason eats up all my ram when I open I believe .zshrc and trust me, it's not invisible. The system crawls to a halt.
- bayindirh 1y agoIt depends on the SSD, I may say. If we're talking about SATA SSDs which top at 600MBps, then yes, an aggressive application can make itself known. However, if you have a modern NVMe, esp. a 4x4 one like Samsung 9x0 series or if you're using a Mac, I bet you'll notice the problem much later, if ever. Remember the SSD trashing problem on M1 Macs? People never noticed that system used SWAP that heavily and trashed the SSD on board. Then, if you're using a server with a couple of SAS or NVMe SSDs, you'll not notice the problem again, esp. if these are backed by RAID (even md counts).
- gchamonlive 1y agoNow that you say, I have a new Lenovo yoga with those SoC ram with crazy parallel channel config (16gb spread across 8 dies of 2gb). It's noticeably faster than my Acer nitro with dual channel 16gb ddr5. I'll check that, but I'd say it's not what the average home user (and even server I'd risk saying) would have.
- justsomehnguy 1y agoWhat do you prefer: ( ) a 1% chance the system would crawl to a halt but would work ( ) a 1% change the kernel would die and nothing would work
- andai 1y agoCan someone explain this to me? Doesn't swap just delay the fundamental issue? Or is there a qualitative difference?
- eru 1y agoSwap delays the 'fundamental issue', if you have a leak that keeps growing. If your problem doesn't keep growing, and you just have more data that programs want to keep in memory than you have RAM, but the actual working set of what's accessed frequently still fits in RAM, then swap perfectly solves this. Think lots of programs open in the background, or lots of open tabs in your browser, but you only ever rapidly switch between at most a handful at a time. Or you are starting a memory hungry game and you don't want to be bothered with closing all the existing memory hungry programs that idle in the background while you play.
- justsomehnguy 1y agohttps://news.ycombinator.com/item?id=45007821 https://news.ycombinator.com/item?id=45007821 > Doesn't swap just delay the fundamental issue? The fundamental issue here is what the linux fanboys literally think what killing a working process and most of the time the process[0] is a good solution for not solving the fundamental problem of memory allocation in the Linux kernel. Availability of swap allows you to avoid malloc failure in a rare case your processes request more memory than physically (or 'physically', heh) present in the system. But in the mind of so called linux administrators even if a one byte of the swap would be used then the system would immediately crawl to a stop and never would recover itself. Why it always should be the worst and the most idiotic scenario instead of a sane 'needed 100MB more, got it - while some shit in the memory which wasn't accessed since the boot was swapped out - did the things it needed to do and freed that 100MB' is never explained by them. [0] imagine a dedicated machine for *SQL server - which process would have the most memory usage on that system?
- xienze 1y ago> it's not invisible. The system crawls to a halt. I’m gonna guess you’re not old enough to remember computers with memory measured in MB and IDE hard disks? Swapping was absolutely brutal back then. I agree with the other poster, swap hitting an SSD is a barely noticeable in comparison.
- pdimitar 1y agoI am not sure exactly what your point is. Is it "hey, it can be much worse"? If so, not a very interesting argument if your machine crawls to a halt.
- commandersaki 1y agoThis is a wrong belief This is not about belief, but lived experience. Setting up swap to me is a choice between a unresponsive system (with swap) or a responsive system with a few oom kills or downed system.
- bayindirh 1y ago> This is not about belief, but lived experience. I mean, I manage some servers, and this is my experience. > Setting up swap to me is a choice between a unresponsive system (with swap) or a responsive system with a few oom kills or downed system. Sorry, but are you sure that you budgeted your system requirements correctly? A Linux system shall neither fill SWAP nor trigger OOM regularly.
- eru 1y agoSwap also works really well for desktop workloads. (I guess that's why Apple uses it so heavily on their Macbooks etc.) With a good amount of swap, you don't have to worry about closing programs. As long as your 'working set' stays smaller than your RAM, your computer stays fast and responsive, regardless of what's open and idling in the background.
- bayindirh 1y agoYes, this is my experience, too. However, I still tend to observe my memory usage even if I have plenty of free RAM. Old habits die hard, but I'm not complaining about this one. :)
- commandersaki 1y agoIt doesn’t happen often, and I have a multi user system with unpredictable workloads. It’s also not about swap filling up, but giving the pretense the system is operable in a memory exhausted state which means oom killer doesn’t run, but the system is unresponsive and never recovers. Without swap oom killer runs and things become responsive.
- adastra22 1y agoWhat pressure? If your ram is underutilized, what pressure are you talking about? If the slowest drive on the machine is the SSD, how does caching to swap help?
- bayindirh 1y agoA long running Linux system uses 100% of its RAM. Every byte unused for applications will be used as a disk cache, given you read more data than your total RAM amount. This cache is evictable, but it'll be there eventually. Linux used to don't touch unused pages in the RAM in the older days if your RAM was not under pressure, but now it swaps out pages unused for a long time. This allows more cache space in RAM. > how does caching to swap help? I think I failed to convey what I tried to say. Let me retry: Kernel doesn't cache to SSD. It swaps out unused (not accessed) but unevictable pages to SWAP, assuming that these pages will stay stale for a very long time, allowing more RAM to be used as cache. When I look to my desktop system, in 12 days, Kernel moved 2592MB of my RAM to SWAP despite having ~20GB of free space. ~15GB of this free space is used as disk cache. So, to have 2.5GB more disk cache, Kernel moved 2592 MB of non-accessed pages to SWAP.
- wallstop 1y agoEdit: wallstop@fridge:~$ free -m total used free shared buff/cache available Mem: 15838 9627 3939 26 2637 6210 Swap: 4095 0 4095 wallstop@fridge:~$ uptime 00:43:54 up 37 days, 23:24, 1 user, load average: 0.00, 0.00, 0.00
- bayindirh 1y agoThe command you want to use is "free -m". This is from another system I have close: total used free shared buff/cache available Mem: 31881 1423 1042 10 29884 30457 Swap: 976 2 974 2MB of SWAP used, 1423 MB RAM used, 29GB cache, 1042 MB Free. Total RAM 32 GB.
- vasco 1y agoIn EC2 using any kind of swapping is just wrong, the comment you replied to already made all the points that can be made though.
- bayindirh 1y agoFrom my understanding, the comment I'm replying to uses EC2 example to portray that swapping is wrong in any and all circumstances, and I just replied with my experience with my system administrator hat. I'm not an AWS guy. I can see and touch the servers I manage, and in my experience, SWAP works, and works well.
- matt-p 1y agoJust for context EC2 typically uses network storage that, for obvious reasons, often has fairly rubbish latency and performance characteristics. Swap works fine if you have local storage, though obviously it burns through your SSD/NVME drive faster and can other side effects on it's performance (usually not particularly noticeable).
- bayindirh 1y agoThanks, I'll keep that in mind if I start to use EC2 for workloads. However, from my experience, normal (eviction based) usage of SWAP doesn't impact the life of an SSD in a measurable manner. My 256GB system SSD (of my desktop system) shows 78% life remaining after 4 years of power on hours, which also served as /home for at least half of its life.
- eru 1y agoHow long is long running? You should be getting the warm caches after at most a few hours. > Secondly, Kernel swaps out unused pages to SWAP, relieving pressure from RAM. So, SWAP is often used even if you fill 1% of your RAM. This allows for more hot data to be cached, allowing better resource utilization and performance in the long run. Yes, and you can observe that even in your desktop at home (if you are running something like Linux). > So, eff it, we ball is never a good system administration strategy. Even if everything is ephemeral and can be rebooted in three seconds. I wouldn't be so quick. Google ran their servers without swap for ages. (I don't know if they still do it.) They decided that taking the slight inefficiency in memory usage, because they have to keep the 'leaked' pages around in actual RAM, is worth it to get predictability in performance. For what it's worth, I add generous swap to all my personal machines, mostly so that the kernel can offload cold / leaked pages and keep more disk content cached in RAM. (As a secondary reason: I also like to have a generous amount of /tmp space that's backed by swap, if necessary.) With swap files, instead of swap partitions, it's fairly easy to shrink and grow your swap space, depending on what your needs for free space on your disk are.
- bayindirh 1y ago> Yes, and you can observe that even in your desktop... Yup, that part of my comment was culmination of using Linux desktops for the last two decades. :) > I wouldn't be so quick. Google ran their servers without swap for ages. If you're designing this from get go and planning accordingly, it doesn't fit into my definition of eff it, we ball, but let's try this and see whether we can make it work. > With swap files, instead of swap partitions,... I'm a graybeard. I eyeball a swap partition size while installing the OS, and just let it be. Being mindful and having good amount of RAM means that SWAP acts as a eviction area for OS first, and as an escape ramp second, in very rare cases. -- Sent from my desktop.
- kryptiskt 1y agoMy work Ubuntu laptop has 40GB of RAM and and a very fast Nvme SSD, if it gets under memory pressure it slows to a crawl and is for all practical purposes frozen while swapping wildly for 15-20 minutes. So no, my experience with swap isn't that it's invisible with SSD.
- interroboink 1y agoI don't know your exact situation, but be sure you're not mixing up "thrashing" with "using swap". Obviously, thrashing implies swap usage, but not the other way around.
- db48x 1y agoIf it’s frozen, or if the mouse suddenly takes seconds to respond to every movement, then it’s not just using some swap. It’s thrashing for sure.
- pdimitar 1y agoI get it that the distinction is real but nobody using the machine cares at this point. It must not happen and if disabling swap removes it then people will disable swap.
- webstrand 1y agoI've experimented with no-swap and find the same thing happens. I think the issue is that linux can also evict executable pages (since it can just reload them from disk). I've had good experience with linux's multi-generation LRU feature, specifically the /sys/kernel/mm/lru_gen/min_ttl_ms feature that triggers OOM-killer when the "working set of the last N ms doesn't fit in memory".
- ValdikSS 1y agoEnables Multi-Gen LRU (improved page reclaim and caching policy). Prevents thrashing, improves loading speeds under low ram conditions. Requires kernel 6.1+. Has dramatic effect especially on slower HDDs. For slower HDDs, consider 1000 instead of 300 for min_ttl_ms. sudo tee /etc/tmpfiles.d/mglru.conf <<EOF w- /sys/kernel/mm/lru_gen/enabled - - - - y w- /sys/kernel/mm/lru_gen/min_ttl_ms - - - - 300 EOF
- hhh 1y agoKubernetes supports swap now. I still don’t use it though.
- bayindirh 1y agoGood to know. Thanks!
- db48x 1y agoThis is not really true of most SSDs. When Linux is really thrashing the swap it’ll be essentially unusable unless the disk is _really_ fast. Fast enough SSDs are available though. Note that when it’s really thrashing the swap the workload is 100% random 4KB reads and writes in equal quantities. Many SSDs have high read speeds and high write speeds but have much worse performance under mixed workloads. I once used an Intel Optane drive as swap for a job that needed hundreds of gigabytes of ram (in a computer that maxed out at 64 gigs). The latency was so low that even while the task was running the machine was almost perfectly usable; in fact I could almost watch videos without dropping frames at the same time.
- fulafel 1y ago> Note that when it’s really thrashing the swap the workload is 100% random 4KB reads and writes in equal quantities. The free memory won't go below a configurable percentage and the contiguous io algorithms of the swap code and i/o stack can still do their work.
- db48x 1y agoThat may be the intention, but you shouldn’t rely on it. In practice the average IO size is, or at least was, almost always 4KB. Here’s a screenshot from atop while the task was running: <https://db48x.net/temp/Screenshot%20from%202019-11-19%2023-42-10.png https://db48x.net/temp/Screenshot%20from%202019-11-19%2023-4...>. Note the number of page faults, the swin and swout (swap in and swap out) numbers, and the disk activity on nvme0n1. Swap in is 150k, and the number of disk reads was 116k with an average size of 6KB. Swap out was 150k with 150k disk writes of 4KB. It’s also reading from sdh at a fair clip (though not as fast as I wanted!) <https://db48x.net/temp/Screenshot%20from%202019-12-09%2011-40-00.png https://db48x.net/temp/Screenshot%20from%202019-12-09%2011-4...> is interesting because it actually shows 24KB average write size. But notice that swout is 47k but there were actually 57k writes. That’s because the program I was testing had to write data out to disk to be useful, and I had it going to a different partition on the same nvme disk. Notice the high queue depth; this was a very large serial write. The swap activity was still all 4KB random IO.