4 ms·

I hate swap. My experience with it is that once a disk-backed machine (as opposed to SSD) has started swapping, it's essentially unusable until you manually for

I hate swap. My experience with it is that once a disk-backed machine (as opposed to SSD) has started swapping, it's essentially unusable until you manually force all anonymous pages to be paged in by turning off swap ("sudo swapoff -a" on Linux) or reboot. My hunch is that the OS is swapping stuff back in stupidly. Once memory is available, I'd like it to page everything back proactively, preferring stuff from swap and then from file-backed mmaps. But instead it seems to be purely reactive, each major page fault requiring a disk seek to page in what's needed with little if any readahead. Basically the whole VM space remains a minefield until you stumble over and detonate each mine in your normal operation. Much better to reboot and have a usable system again. On my Linux systems, I've turned off swap. On OS X...last I checked, I wasn't able to find a way to do this. I'd like to turn off swap entirely, or failing that, have some equivalent way to force all of swap to be paged in now so I don't have to reboot when I hit swap. Anyone know of a way?

calpaterson 10y ago
Contrarian anecdote: I've recovered from swapping a few times on a desktop machine with an HDD (typical scenario: an ON clause was omitted from a join and postgres is doing a cartesian join between two tables). I didn't find things unable instantly, I was able to recover by SIGTERMing the relevant process and then running `swapoff --all` while maybe going for a tea break. YMMV.
outworlder 10y ago
> My experience with it is that once a disk-backed machine (as opposed to SSD) has started swapping, it's essentially unusable until you manually force all anonymous pages to be paged in by turning off swap ("sudo swapoff -a" on Linux) or reboot. That depends. If your workload exceeds the amount of available memory, you will start "thrashing" the disk and that can make a system un-responsive. If you happen to launch a large application, or start working with a big file, unused pages will be evicted to disk to make room and, after some slowdown, the system should become perfectly usable again. YMMV On OSX, I don't know a way, but I can't recall the last time I had to reboot due to RAM/swap issues, even when I was developing apps on a 4GB Macbook Air. I guess memory compression, which is enabled by default, helps here. Most OSX systems have very fast SSDs as well.
- tluyben2 10y ago
  On my 2015 MBP with 8GB & SSD, I am often stuck for 10-15 minutes unable to do anything while thrashing. And I am someone who has Activity Monitor handy. I do not have this on my much older and weaker Ubuntu X220s doing the same type of development. Not sure why that is.
  - Yaggo 10y ago
    If it's 3rd party SSD, have you enabled TRIM? I had to do that for my old Mac Mini, made big difference. (2015 MBP of course has factory-installed SSD, but maybe this helps someone else.) http://osxdaily.com/2015/10/29/use-trimforce-trim-ssd-mac-os-x/ http://osxdaily.com/2015/10/29/use-trimforce-trim-ssd-mac-os...
    - tluyben2 10y ago
      It's all original Apple hardware.
  - raarts 10y ago
    It can be Spotlight reindexing, or Time Machine creating a backup into /Volumes/MobileBackups (done once per hour). Especially taxing if you have lots of files on your disk.
    - nitrogen 10y ago
      It's kind of annoying that these processes aren't niced and ioniced (or the macos equivalent). Especially when typing a password.
  - Spooky23 10y ago
    Look for hung mds (spotlight indexer) threads. They can get hung up and fail Un-gracefully with some files. I had a client whose app generate PDFs that would cause this to happen.
    - tluyben2 10y ago
zzzcpan 10y ago
"My experience with it is that once a disk-backed machine (as opposed to SSD) has started swapping, it's essentially unusable" The OS should start swapping very early to avoid bursts of disk I/O and rendering the system unusable. On linux this is somewhat configurable, even if not user friendly, but a combination of swappiness and vfs_cache_pressure could turn it into a usable machine, taking care of inefficient memory usage, memory leaks, unnecessary vfs cache, etc.
- scottlamb 10y ago
  > The OS should start swapping very early to avoid bursts of disk I/O and rendering the system unusable. I think you're talking about the I/O of paging dirty things out, but I'm talking about the fact that some memory location is no longer present in RAM, so accessing it will take 10 ms or more to page in. The system is not only useless while actively swapping. It's useless after it has ever swapped, and you can only recover by disabling swap ("sudo swapoff -a") or rebooting.
dormento 10y ago
On OSX it used to be that you had to disable the pager daemon, then remove the swap file. Probably need to disengage system protection for it to work though. sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist Then: rm /var/vm/swapfile* Disclaimer: haven't tried doing it this way on Sierra.

calpaterson 10y ago

Contrarian anecdote: I've recovered from swapping a few times on a desktop machine with an HDD (typical scenario: an ON clause was omitted from a join and postgres is doing a cartesian join between two tables). I didn't find things unable instantly, I was able to recover by SIGTERMing the relevant process and then running `swapoff --all` while maybe going for a tea break. YMMV.

outworlder 10y ago

> My experience with it is that once a disk-backed machine (as opposed to SSD) has started swapping, it's essentially unusable until you manually force all anonymous pages to be paged in by turning off swap ("sudo swapoff -a" on Linux) or reboot. That depends. If your workload exceeds the amount of available memory, you will start "thrashing" the disk and that can make a system un-responsive. If you happen to launch a large application, or start working with a big file, unused pages will be evicted to disk to make room and, after some slowdown, the system should become perfectly usable again. YMMV On OSX, I don't know a way, but I can't recall the last time I had to reboot due to RAM/swap issues, even when I was developing apps on a 4GB Macbook Air. I guess memory compression, which is enabled by default, helps here. Most OSX systems have very fast SSDs as well.

tluyben2 10y ago
On my 2015 MBP with 8GB & SSD, I am often stuck for 10-15 minutes unable to do anything while thrashing. And I am someone who has Activity Monitor handy. I do not have this on my much older and weaker Ubuntu X220s doing the same type of development. Not sure why that is.
- Yaggo 10y ago
  If it's 3rd party SSD, have you enabled TRIM? I had to do that for my old Mac Mini, made big difference. (2015 MBP of course has factory-installed SSD, but maybe this helps someone else.) http://osxdaily.com/2015/10/29/use-trimforce-trim-ssd-mac-os-x/ http://osxdaily.com/2015/10/29/use-trimforce-trim-ssd-mac-os...
  - tluyben2 10y ago
    It's all original Apple hardware.
- raarts 10y ago
  It can be Spotlight reindexing, or Time Machine creating a backup into /Volumes/MobileBackups (done once per hour). Especially taxing if you have lots of files on your disk.
  - nitrogen 10y ago
    It's kind of annoying that these processes aren't niced and ioniced (or the macos equivalent). Especially when typing a password.
- Spooky23 10y ago
  Look for hung mds (spotlight indexer) threads. They can get hung up and fail Un-gracefully with some files. I had a client whose app generate PDFs that would cause this to happen.
  - tluyben2 10y ago

zzzcpan 10y ago

"My experience with it is that once a disk-backed machine (as opposed to SSD) has started swapping, it's essentially unusable" The OS should start swapping very early to avoid bursts of disk I/O and rendering the system unusable. On linux this is somewhat configurable, even if not user friendly, but a combination of swappiness and vfs_cache_pressure could turn it into a usable machine, taking care of inefficient memory usage, memory leaks, unnecessary vfs cache, etc.

scottlamb 10y ago
> The OS should start swapping very early to avoid bursts of disk I/O and rendering the system unusable. I think you're talking about the I/O of paging dirty things out, but I'm talking about the fact that some memory location is no longer present in RAM, so accessing it will take 10 ms or more to page in. The system is not only useless while actively swapping. It's useless after it has ever swapped, and you can only recover by disabling swap ("sudo swapoff -a") or rebooting.

dormento 10y ago

On OSX it used to be that you had to disable the pager daemon, then remove the swap file. Probably need to disengage system protection for it to work though. sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist Then: rm /var/vm/swapfile* Disclaimer: haven't tried doing it this way on Sierra.