7 ms·

Something seems to be seriously wrong with the swap implementation on modern systems. 20 years ago on Windows 98 it just started swapping, but it was no big de

Something seems to be seriously wrong with the swap implementation on modern systems. 20 years ago on Windows 98 it just started swapping, but it was no big deal. If something became too slow to be usable, you could just press ctrl+alt+del and kill that swapped program and everything worked fine afterwards. On the other hand, my modern linux laptop, it starts swapping, and it swaps and swaps and you can do nothing, not even move the mouse, till 30 minutes later something crashes.

rootbear 10y ago
Could this be a reflection of the increasing gulf between RAM speed and HD speed? Even with NVMe drives, which one probably shouldn't be swapping to anyway, RAM is orders of magnitude faster.
- Retric 10y ago
  With SSD's disk is not that slow.
  - SapphireSun 10y ago
    SSDs are only ~4x faster than magnetic last I checked. If RAM is 100ns per access, and hd access is down from say, 1ms to 0.25ms, that's still a huge huge gap. 4x isn't even an order of magnitude. EDIT: see comment below for more accurate numbers.
    - snowwindwaves 10y ago
      From the article: >A typical reference to RAM is in the area of 100ns, accessing data on a SSD 150μs (so 1500 times of the RAM) and accessing data on a rotating disk 10ms (so 100.000 times the RAM.
      - SapphireSun 10y ago
        Thank you for the correction. I should have read more carefully. Still, we're talking 3 orders of magnitude for SSD vs RAM.
      - xinyhn 10y ago
        reminded me of this... Latency Numbers Every Programmer Should Know https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832
- curlypaul924 10y ago
  I think, among other things, it has to do with the size of the swap space relative to the speed of the swap device. IME high disk i/o combined with large swap space means swap never fills up and the OOM killer doesn't kick in. On systems with less RAM and swap, OOM conditions were hit much sooner, even with slower disks. Default settings for dirty ratio and dirty background ratio exacerbate the issue: more data is held onto before it is written, and once the background ratio is hit, any application writing to disk will block.
thiagobbt 10y ago
Isn't this related to this change on kernel 4.10? https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4bcee913f2c71a859dcc184 https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4...
- ChuckMcM 10y ago
  Possibly, however since the writeback behavior is configurable I expect you could test that thesis by changing the aggressiveness of the writeback draining.
plorkyeran 10y ago
This is the exact opposite of my experience. Back in the Windows 9x days it was a fairly routine experience for the system to soft-lock with the HD grinding away and I'd sometimes end up just hard rebooting the computer after waiting a few minutes for the ctrl-alt-delete dialog to appear. On macOS with a SSD I don't even notice when my system is swapping heavily.
mwpmaybe 10y ago
If you have an SSD, try setting vm.swappiness to 1 (not 0).
tedunangst 10y ago
0. Possibly not true in all cases. 1. Modern systems are much more aggressive about enormous disk caches, which can ironically lead to io storms when it swaps out your application to buffer writes, then has to flush the cache to swap the app back in. 2. Difference in working set size and number of background programs waking up.
throwawayish 10y ago
I have been using various operating systems for a while. I feel like Linux has, in general, from a UX point of view, the worst behaviour when swapping and the worst behaviour in general under memory pressure. I feel like it has gotten worse over time, which might not be just the kernel but the general desktop ecosystem. If you require much more memory to move the mouse or show the task manager equivalent, then the system will be much less responsive when it thrashes itself. Honestly, I'ld much rather have Linux just crash and reboot, that'd be faster than it's thrashing-tantrums. Luckily, there's earlyoom, which just rampages the town quickly if memory pressure approaches. Like a reboot (ie. damage was done), just faster. In any case, it makes me sad (in a bad way) to see how bad the state of things is when it comes to the basics of computing, like managing memory.
- tluyben2 10y ago
  Not an excuse for bad implementations, but since I run i3wm, my feelings of happiness increased rapidly. To such an extend that I do not want to ever run anything else; stability, speed, memory use... Solves (for me) the issues you have.
  - sevensor 10y ago
    i3 is magnificent. The same display seems 10x bigger when using i3. As true for netbooks as for big desktops. My old x120 dual boots win7, which is unusably slow and unstable on it. Arch with i3 is still snappy. Unless I'm running a web browser. Web browsers have gone insane.
    - doubleplusgood 10y ago
      I couldn't figure out how to scale i3 to the high DPI on my Yoga 900, with Wayland on F25.
      - sevensor 10y ago
        If you're on Wayland, use Sway instead. It feels so much like i3 that I often forget it's not i3. Hidpi works pretty well: https://github.com/SirCmpwn/sway/issues/797 https://github.com/SirCmpwn/sway/issues/797. I use this on a Dell Precision with the 4k display.
alsadi 10y ago
Just type sudo swapoff -a sudo swapon -a
- benibela 10y ago
  Can't type while it is thrashing. Otherwise the offending program could just be killed
bsdetector 10y ago
> on Windows 98 it just started swapping, but it was no big deal. At that time, swapping out a 4k page was a significant part of memory: 4k of 16 MiB is 1/4096 of memory. Each swap gets back a lot of memory the program needs. Now the swap is still 4k pages, but memory has expanded by a thousand fold. Basically swap is a thousand times worse today than it was in the time of Windows 98. For harddrives swap isn't used now to expand memory, it's used to remove initialization code and other 'dead' memory. Swap should be set to only a tiny fraction of the memory size for this reason, to prevent it from being used to handle actual out-of-memory conditions. But realistically for most users it's not even worth enabling at all because of the occasional memory that needs to be swapped in from disk. For SSDs the seek speed has improved to match the extra memory so swap can still be used like in the old days to expand the effective memory size. But memory is so large a swap file that's a fraction of memory size to offload 'dead' memory is enough unless there's a specific reason to actually use swap for out-of-memory.
Asooka 10y ago
Because Windows 98 always kept enough resources available to show you the c-a-d dialog. On Linux, however, there is no "the shell must remain interactive at all times" requirement, so a daemon that gobbles memory and your rescue shell have the exact same priority. Modern Windows even has a graphics card watchdog and if any application issues a command to the GPU that takes too long, it's suspended and the user is asked if it should be killed. Probably not what you want on an HPC that does deep learning, but exactly what you want on an interactive desktop. I suppose it might be possible to whip something up with cgroups and policy that will keep the VT, bash, X and a few select programs always resident in memory and give them ultimate I/O priority, but I haven't tried.
leni536 10y ago
I feel you. X and some recovery critical software should have their reserved memory cgroup with some guaranteed, safe amount of physical memory and 0 swappiness. I speculate that on Windows it works so well because most of these stuff are in kernel space anyway.
digi_owl 10y ago
I think thats more related to Linux and its prioritization of IO than anything else. Note that the latest kernel release 4.10) contains an IO throttle that should improve this experience. https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4bcee913f2c71a859dcc184 https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4...

rootbear 10y ago

Could this be a reflection of the increasing gulf between RAM speed and HD speed? Even with NVMe drives, which one probably shouldn't be swapping to anyway, RAM is orders of magnitude faster.

Retric 10y ago
With SSD's disk is not that slow.
- SapphireSun 10y ago
  SSDs are only ~4x faster than magnetic last I checked. If RAM is 100ns per access, and hd access is down from say, 1ms to 0.25ms, that's still a huge huge gap. 4x isn't even an order of magnitude. EDIT: see comment below for more accurate numbers.
  - snowwindwaves 10y ago
    From the article: >A typical reference to RAM is in the area of 100ns, accessing data on a SSD 150μs (so 1500 times of the RAM) and accessing data on a rotating disk 10ms (so 100.000 times the RAM.
    - SapphireSun 10y ago
      Thank you for the correction. I should have read more carefully. Still, we're talking 3 orders of magnitude for SSD vs RAM.
    - xinyhn 10y ago
      reminded me of this... Latency Numbers Every Programmer Should Know https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832
curlypaul924 10y ago
I think, among other things, it has to do with the size of the swap space relative to the speed of the swap device. IME high disk i/o combined with large swap space means swap never fills up and the OOM killer doesn't kick in. On systems with less RAM and swap, OOM conditions were hit much sooner, even with slower disks. Default settings for dirty ratio and dirty background ratio exacerbate the issue: more data is held onto before it is written, and once the background ratio is hit, any application writing to disk will block.

thiagobbt 10y ago

Isn't this related to this change on kernel 4.10? https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4bcee913f2c71a859dcc184 https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4...

ChuckMcM 10y ago
Possibly, however since the writeback behavior is configurable I expect you could test that thesis by changing the aggressiveness of the writeback draining.

plorkyeran 10y ago

This is the exact opposite of my experience. Back in the Windows 9x days it was a fairly routine experience for the system to soft-lock with the HD grinding away and I'd sometimes end up just hard rebooting the computer after waiting a few minutes for the ctrl-alt-delete dialog to appear. On macOS with a SSD I don't even notice when my system is swapping heavily.

mwpmaybe 10y ago

If you have an SSD, try setting vm.swappiness to 1 (not 0).

tedunangst 10y ago

0. Possibly not true in all cases. 1. Modern systems are much more aggressive about enormous disk caches, which can ironically lead to io storms when it swaps out your application to buffer writes, then has to flush the cache to swap the app back in. 2. Difference in working set size and number of background programs waking up.

throwawayish 10y ago

I have been using various operating systems for a while. I feel like Linux has, in general, from a UX point of view, the worst behaviour when swapping and the worst behaviour in general under memory pressure. I feel like it has gotten worse over time, which might not be just the kernel but the general desktop ecosystem. If you require much more memory to move the mouse or show the task manager equivalent, then the system will be much less responsive when it thrashes itself. Honestly, I'ld much rather have Linux just crash and reboot, that'd be faster than it's thrashing-tantrums. Luckily, there's earlyoom, which just rampages the town quickly if memory pressure approaches. Like a reboot (ie. damage was done), just faster. In any case, it makes me sad (in a bad way) to see how bad the state of things is when it comes to the basics of computing, like managing memory.

tluyben2 10y ago
Not an excuse for bad implementations, but since I run i3wm, my feelings of happiness increased rapidly. To such an extend that I do not want to ever run anything else; stability, speed, memory use... Solves (for me) the issues you have.
- sevensor 10y ago
  i3 is magnificent. The same display seems 10x bigger when using i3. As true for netbooks as for big desktops. My old x120 dual boots win7, which is unusably slow and unstable on it. Arch with i3 is still snappy. Unless I'm running a web browser. Web browsers have gone insane.
  - doubleplusgood 10y ago
    I couldn't figure out how to scale i3 to the high DPI on my Yoga 900, with Wayland on F25.
    - sevensor 10y ago
      If you're on Wayland, use Sway instead. It feels so much like i3 that I often forget it's not i3. Hidpi works pretty well: https://github.com/SirCmpwn/sway/issues/797 https://github.com/SirCmpwn/sway/issues/797. I use this on a Dell Precision with the 4k display.

alsadi 10y ago

Just type sudo swapoff -a sudo swapon -a

benibela 10y ago
Can't type while it is thrashing. Otherwise the offending program could just be killed

bsdetector 10y ago

> on Windows 98 it just started swapping, but it was no big deal. At that time, swapping out a 4k page was a significant part of memory: 4k of 16 MiB is 1/4096 of memory. Each swap gets back a lot of memory the program needs. Now the swap is still 4k pages, but memory has expanded by a thousand fold. Basically swap is a thousand times worse today than it was in the time of Windows 98. For harddrives swap isn't used now to expand memory, it's used to remove initialization code and other 'dead' memory. Swap should be set to only a tiny fraction of the memory size for this reason, to prevent it from being used to handle actual out-of-memory conditions. But realistically for most users it's not even worth enabling at all because of the occasional memory that needs to be swapped in from disk. For SSDs the seek speed has improved to match the extra memory so swap can still be used like in the old days to expand the effective memory size. But memory is so large a swap file that's a fraction of memory size to offload 'dead' memory is enough unless there's a specific reason to actually use swap for out-of-memory.

Asooka 10y ago

Because Windows 98 always kept enough resources available to show you the c-a-d dialog. On Linux, however, there is no "the shell must remain interactive at all times" requirement, so a daemon that gobbles memory and your rescue shell have the exact same priority. Modern Windows even has a graphics card watchdog and if any application issues a command to the GPU that takes too long, it's suspended and the user is asked if it should be killed. Probably not what you want on an HPC that does deep learning, but exactly what you want on an interactive desktop. I suppose it might be possible to whip something up with cgroups and policy that will keep the VT, bash, X and a few select programs always resident in memory and give them ultimate I/O priority, but I haven't tried.

leni536 10y ago

I feel you. X and some recovery critical software should have their reserved memory cgroup with some guaranteed, safe amount of physical memory and 0 swappiness. I speculate that on Windows it works so well because most of these stuff are in kernel space anyway.

digi_owl 10y ago

I think thats more related to Linux and its prioritization of IO than anything else. Note that the latest kernel release 4.10) contains an IO throttle that should improve this experience. https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4bcee913f2c71a859dcc184 https://kernelnewbies.org/Linux_4.10#head-f6ecae920c0660b7f4...