9 ms·

Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

155 points by yankcrime 16y ago

eliben 16y ago
The title of the link (unlike the linked page) suggests that Linux is somehow to blame here... Why? Using RAM for disk caching is a good thing. Besides, this technique is employed by Windows as well. It's just that the average Windows user probably doesn't know how much free RAM his PC has (well, the really average user doesn't know how much total RAM it has). My Windows 7 box takes at least 2GB of memory to itself for disk caching, AFAIU.
- yankcrime 16y ago
  My apologies, that's certainly not what I meant to imply (and hence the use of the word seemingly). The point I was coming from was with regards to the number of times I've had to explain these concepts to an exasperated friend or cow-orker, when they've exclaimed something along the lines of "I've just looked at `top` and there's no free memory!".
- roc 16y ago
  Windows started doing disk caching with Vista and users certainly noticed and complained in much the same way.
  - barrkel 16y ago
    Windows didn't start doing disk caching with Vista. Perhaps it used memory more aggressively, or changed the way it did the accounting, but it certainly didn't start doing disk caching just with Vista, as anyone who remembers the difference in performance of DOS before and after running SMARTDRV will attest.
    - ringm 16y ago
      2000/XP was always extremely aggressive about allocating memory for cache. It would very often swap stuff out to increase available cache, which made the apps you did not use for just a few seconds completely unresponsive.
      - barrkel 16y ago
        Specifically, minimized applications got their working set trimmed. You could see this in Task Manager: it's Memory Usage column was actually Working Set (I think they're more explicitly labelled these days) and it frequently went from many MB down to a few KB once you minimized the app. Working set is the set of pages, currently in memory, that the app has used "recently", and are thus less likely to be paged to disk. Shrinking the WS makes pages more likely to be paged out.
  - halfasleep 16y ago
    Windows started doing pre-emptive caching with Vista, based on the applications you tend to run.
- Luyt 16y ago
  I think the average user doesn't even know what RAM is. My neighbour, for example, doesn't know the distinction between the space available in RAM and the capacity of his harddisk. Which led to a strange misunderstanding when he told me his PC had '250 giga of internal memory'. I tried to explain than with 'internal memory' people usually mean something else, but I didn't think he understood. Never mind, he doesn't grasp hierarchical filesystems either, but all is fine as long as he can edit his photos (which in his perception are kept within Elements) and 'surf the internet'.
cullend 16y ago
So basically the same thing Vista was doing (prefetch, and admittedly it did a shitty job with >1GB ram), except since its Linux, Oh its all fine guys, we won't destroy the OS' reputation.
- bbatsell 16y ago
  Right, the only thing wrong with Windows Vista was its RAM caching strategy.
  - recoiledsnake 16y ago
    Maybe not, but there were a lot of people needlessly going bonkers over the disk light blinking (SuperFetch) and free memory numbers in the task manager(exactly the same problem that the referenced site is trying to explain). See http://www.codinghorror.com/blog/2006/09/why-does-vista-use-all-my-memory.html http://www.codinghorror.com/blog/2006/09/why-does-vista-use-...
- simias 16y ago
  The page is about explaining a correct behavior, not justifying some bogus memory usage. There's no reputation at stake. All modern operating systems are expected to behave that way, free RAM is wasted RAM.
- samdk 16y ago
  Linux has been doing this for a lot longer than Windows has. (Which I only actually know because there was so much FUD about Vista's pre-fetching when it came out.) edit: Sorry, getting prefetching and SuperFetch confused.
  - gaius 16y ago
    Unix has been doing it... You talk as if Linus personally invented the idea of caching.
  - recoiledsnake 16y ago
    Doing what? Prefetching? It has been there since XP.It is not just simple caching. Windows probably was doing caching even before Linux existed. What Vista does is SuperFetch and I don't think it's there in Linux even now, except that someone wrote a program that would do a similar thing but it's not really used that much.
pixelbeat 16y ago
Heh, nice site. Note I find this useful for showing how much RAM apps really are using: http://www.pixelbeat.org/scripts/ps_mem.py http://www.pixelbeat.org/scripts/ps_mem.py
StavrosK 16y ago
A good way I read somewhere was "Unused RAM is wasted RAM. Linux uses it to speed up your disk and gives it back to programs when they need it".
slavak 16y ago
Wouldn't it make more sense to just include the RAM used for disk-caching when calculating free memory? After all, the whole reason it's being used for disk-caching in the first place is that it's free! The Linuxoids that know what this even means can always explicitly look for the line that says how much memory is being used for caching...
- sp332 16y ago
  The Windows task manager has a separate field called "Available". Right now I have 8190 MB total, 3627 cached, 4151 Available, and 537 free (not sure if that adds up perfectly, it was changing while I typed it).
  - masklinn 16y ago
    OSX's activity monitor does something similar, it has 4 statuses: Wired, Active, Inactive and Free. Wired and Active are basically the RAM being used (Wired is a special activity status for stuff which can not be moved to disk), Inactive is the cache (generally programs which were used recently) and Free is… well, free. When checking used/free, Inactive and Free count towards free, Wired and Active towards used (though that's not always consistent). Also, I don't think you have 8TB RAM.
    - js2 16y ago
      If you really want the details on OS X, see http://developer.apple.com/mac/library/documentation/Performance/Conceptual/ManagingMemory/Articles/AboutMemory.html http://developer.apple.com/mac/library/documentation/Perform... and http://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/vm/vm.html http://developer.apple.com/library/mac/#documentation/Darwin... See http://www.usenix.org/event/usenix2000/freenix/full_papers/silvers/silvers_html/ http://www.usenix.org/event/usenix2000/freenix/full_papers/s... for information about the BSD unified buffer cache. And see http://www.westnet.com/~gsmith/content/linux-pdflush.htm http://www.westnet.com/~gsmith/content/linux-pdflush.htm for information about the Linux page cache.
    - barrkel 16y ago
ludwigvan 16y ago
free -m | awk 'NR==3 {print $4}'
colanderman 16y ago
This is misleading, as the kernel will often swap out processes rather than shrink the disk cache. Its tendency to do this is controlled by the vm.swappiness parameter, which by default heavily favors swapping: http://kerneltrap.org/node/3000 http://kerneltrap.org/node/3000 Decreasing this value will favor reducing the cache size and will tend to increase responsiveness. You can set it in the /etc/sysctl.conf file. Low values (≤ 5) are especially good for SSDs where you want to minimize swap usage.
- kindlyviking 16y ago
  The article mentions this on the second page at the bottom: http://www.linuxatemyram.com/play.html http://www.linuxatemyram.com/play.html
  - colanderman 16y ago
    "But look you found the notice didn't you?" "Yes," said Arthur, "yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying 'Beware of The Leopard'."
- jerf 16y ago
  I often just turn swap off entirely. A normal workload on my 4GB linux machine will frequently fail to fill even 2GB of buffers. At the moment my machine has still managed to fail to fill 800MB of my RAM with anything and I'm running two virtual machines and doing my usual work load. I do have a swap partition but only so I can hibernate.
  - requinot59 16y ago
    > I often just turn swap off entirely. And you're smart to do so. Swap is useless for 99% of end-user systems. ChromeOS doesn't use any swap partition, for instance. As for hibernation, it's not available if you use a crypted swap, which is wise to use (at least on a laptop). As a result, I, too, disabled swap completely.
wzdd 16y ago
This paragraph: 'But what do you call memory that is both used for something and available for applications? You would call that "free", but Linux calls it "used".' is IMO the best way I've ever seen of explaining the issue. It also reveals the solution: call it free! It is free, after all. Nobody cares about how much free ram is cache unless they are doing something really specific, in which case they should be passing specific flags, or using a specific tool, to report it. Unfortunately there is no chance Linux will do this, as the relevant reporting tools are too entrenched (and with good reason).
- hernan7 16y ago
  Maybe add a new (end user-friendly, preferably graphical) tool to report memory usage the user friendly way? Then vmstat can be left as is, and end users can be directed to use the new tool. By the way, this behavior is not Linux-specific. AIX and other Unices have been doing this forever. And end users (who should have known better, BTW) would complain too. The only machine that I remember as showing a sizable percentage of the memory as "free" was a just-imaged top of the line server. Half a gig or RAM, and almost all empty! -- this was the early '90s. Edit: Just noticed that graphical tools that report memory usage in user-friendly terms already exist (gnome system monitor). My bad, I always use vmstat and sar myself, and only recently found out about /proc/NNNNN/status
- wrs 16y ago
  Or just rename "free" to "wasted". I usually tell people the free number indicates how much RAM they paid for but aren't using.
  - sliverstorm 16y ago
    It is admittedly difficult to re-learn what you know about computers, and what Windows has taught people (including myself) is that RAM usage must be below ~75% for optimal performance. It has been my experience that paging becomes much more blocking above that.
ktf 16y ago
It looks like gnome-system-monitor reports the "correct" amount of free RAM. I'd imagine most newbies would use that rather than popping open a terminal and typing "free," right?
- NonMint 16y ago
  You'd be surprised. Some users who use Linux consider themselves to be pretty adept, and I can easily see this problem being an issue for people who perceive themselves to be clever.
  - deleted 16y ago
    [deleted]
  - uxp 16y ago
    I wouldn't say that only "clever" users would rather use `free`, since most Linux installations don't have GNOME or X11 installed to begin with.
JonnieCache 16y ago
How does this compare to darwin's memory management?
Florin_Andrei 16y ago
It's merely a matter of definition - whether to include the disk cache in the "used" memory or not.
radu_floricica 16y ago
I have had the same problem, plus some, when trying to guesstimate memory usage in vps servers. The caching is transparent, so free always reports zero swap. ps gives sensible numbers, but the total used memory is neither RSS nor VSZ summed. I've heard about library memory being double-counted, but I've found no way to accurately get the numbers. In the end I pretty much gave up and settled for the settings that worked. tl;dr: on vps things can be even uglier.
stcredzero 16y ago
Hmm, this situation is like owning an electric vehicle. The manufacturers are passionate about technology, and they invented a flywheel that can store massive amounts of energy, giving your vehicle incredible transient acceleration response. They ship many vehicles with this new feature, but unfortunately, they forgot to account for what this would do to the battery charge display after starting. As a result, their reps keep on having to explain that the vehicles are not rapidly leaking charge shortly after starting -- instead that energy is being placed in the flywheel where it can greatly increase the car's responsiveness. Whoops! The difference between the scenario above and Linux's disk cache? Techies blame the former on the car company but the latter on the end-users.
- Andys 16y ago
  If only electric cars were free like Linux.
  - stcredzero 16y ago
    My analogy has more to do with approaches to UX than it does with economic model.
jawee 16y ago
All this does is make me regret paying more for extra memory... With a dozen programs open, I am sitting at 2.3GB free, according to this. I guess it was wrong to splurge for that extra memory.
jefe78 16y ago
I love explaining it! The response is priceless, "Wow, that's pretty cool!" "No, that's Linux."
spudlyo 16y ago
You can't disable disk caching. The only reason anyone ever wants to disable disk caching is because they think it takes memory away from their applications, which it doesn't! Disk cache makes applications load faster and run smoother, but it NEVER EVER takes memory away from them! Therefore, there's absolutely no reason to disable it! Never say never. You can and should disable/purge linux buffer caching in certain situations. See O_DIRECT and/or POSIX_FADV_DONTNEED. If I'm writing a database, I take care of my own caching, I do not need the OS doing it as well. Also, if I'm doing a huge rsync of a directory tree, just because I just read every byte of every file in that tree, I most certainly do not want to load all of that crap into the linux buffer cache. It will evict my already warm set, as well as potentially cause a ton of swapping.
- js2 16y ago
  Funny, I just noticed this in the Mac OS X rsync man page for the very first time: --cache Apple specific option to enable filesystem caching of rsync file i/o Otherwise fcntl(F_NOCACHE) is used to limit memory growth.
  - Nick_C 16y ago
    There was a patch for all architectures, but it got dropped after 3.07 (current is 3.09). Andrew wrote "I don't feel this patch is general enough nor straight-forward enough to make it into rsync." See these for more: http://insights.oetiker.ch/linux/fadvise/ http://insights.oetiker.ch/linux/fadvise/ https://bugzilla.samba.org/show_bug.cgi?id=7004 https://bugzilla.samba.org/show_bug.cgi?id=7004

155 points by yankcrime 16y ago

The title of the link (unlike the linked page) suggests that Linux is somehow to blame here... Why? Using RAM for disk caching is a good thing. Besides, this technique is employed by Windows as well. It's just that the average Windows user probably doesn't know how much free RAM his PC has (well, the really average user doesn't know how much total RAM it has). My Windows 7 box takes at least 2GB of memory to itself for disk caching, AFAIU.

yankcrime 16y ago
My apologies, that's certainly not what I meant to imply (and hence the use of the word seemingly). The point I was coming from was with regards to the number of times I've had to explain these concepts to an exasperated friend or cow-orker, when they've exclaimed something along the lines of "I've just looked at `top` and there's no free memory!".
roc 16y ago
Windows started doing disk caching with Vista and users certainly noticed and complained in much the same way.
- barrkel 16y ago
  Windows didn't start doing disk caching with Vista. Perhaps it used memory more aggressively, or changed the way it did the accounting, but it certainly didn't start doing disk caching just with Vista, as anyone who remembers the difference in performance of DOS before and after running SMARTDRV will attest.
  - ringm 16y ago
    2000/XP was always extremely aggressive about allocating memory for cache. It would very often swap stuff out to increase available cache, which made the apps you did not use for just a few seconds completely unresponsive.
    - barrkel 16y ago
      Specifically, minimized applications got their working set trimmed. You could see this in Task Manager: it's Memory Usage column was actually Working Set (I think they're more explicitly labelled these days) and it frequently went from many MB down to a few KB once you minimized the app. Working set is the set of pages, currently in memory, that the app has used "recently", and are thus less likely to be paged to disk. Shrinking the WS makes pages more likely to be paged out.
- halfasleep 16y ago
  Windows started doing pre-emptive caching with Vista, based on the applications you tend to run.
Luyt 16y ago
I think the average user doesn't even know what RAM is. My neighbour, for example, doesn't know the distinction between the space available in RAM and the capacity of his harddisk. Which led to a strange misunderstanding when he told me his PC had '250 giga of internal memory'. I tried to explain than with 'internal memory' people usually mean something else, but I didn't think he understood. Never mind, he doesn't grasp hierarchical filesystems either, but all is fine as long as he can edit his photos (which in his perception are kept within Elements) and 'surf the internet'.

cullend 16y ago

So basically the same thing Vista was doing (prefetch, and admittedly it did a shitty job with >1GB ram), except since its Linux, Oh its all fine guys, we won't destroy the OS' reputation.

bbatsell 16y ago
Right, the only thing wrong with Windows Vista was its RAM caching strategy.
- recoiledsnake 16y ago
  Maybe not, but there were a lot of people needlessly going bonkers over the disk light blinking (SuperFetch) and free memory numbers in the task manager(exactly the same problem that the referenced site is trying to explain). See http://www.codinghorror.com/blog/2006/09/why-does-vista-use-all-my-memory.html http://www.codinghorror.com/blog/2006/09/why-does-vista-use-...
simias 16y ago
The page is about explaining a correct behavior, not justifying some bogus memory usage. There's no reputation at stake. All modern operating systems are expected to behave that way, free RAM is wasted RAM.
samdk 16y ago
Linux has been doing this for a lot longer than Windows has. (Which I only actually know because there was so much FUD about Vista's pre-fetching when it came out.) edit: Sorry, getting prefetching and SuperFetch confused.
- gaius 16y ago
  Unix has been doing it... You talk as if Linus personally invented the idea of caching.
- recoiledsnake 16y ago
  Doing what? Prefetching? It has been there since XP.It is not just simple caching. Windows probably was doing caching even before Linux existed. What Vista does is SuperFetch and I don't think it's there in Linux even now, except that someone wrote a program that would do a similar thing but it's not really used that much.

pixelbeat 16y ago

Heh, nice site. Note I find this useful for showing how much RAM apps really are using: http://www.pixelbeat.org/scripts/ps_mem.py http://www.pixelbeat.org/scripts/ps_mem.py

StavrosK 16y ago

A good way I read somewhere was "Unused RAM is wasted RAM. Linux uses it to speed up your disk and gives it back to programs when they need it".

slavak 16y ago

Wouldn't it make more sense to just include the RAM used for disk-caching when calculating free memory? After all, the whole reason it's being used for disk-caching in the first place is that it's free! The Linuxoids that know what this even means can always explicitly look for the line that says how much memory is being used for caching...

sp332 16y ago
The Windows task manager has a separate field called "Available". Right now I have 8190 MB total, 3627 cached, 4151 Available, and 537 free (not sure if that adds up perfectly, it was changing while I typed it).
- masklinn 16y ago
  OSX's activity monitor does something similar, it has 4 statuses: Wired, Active, Inactive and Free. Wired and Active are basically the RAM being used (Wired is a special activity status for stuff which can not be moved to disk), Inactive is the cache (generally programs which were used recently) and Free is… well, free. When checking used/free, Inactive and Free count towards free, Wired and Active towards used (though that's not always consistent). Also, I don't think you have 8TB RAM.
  - js2 16y ago
    If you really want the details on OS X, see http://developer.apple.com/mac/library/documentation/Performance/Conceptual/ManagingMemory/Articles/AboutMemory.html http://developer.apple.com/mac/library/documentation/Perform... and http://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/vm/vm.html http://developer.apple.com/library/mac/#documentation/Darwin... See http://www.usenix.org/event/usenix2000/freenix/full_papers/silvers/silvers_html/ http://www.usenix.org/event/usenix2000/freenix/full_papers/s... for information about the BSD unified buffer cache. And see http://www.westnet.com/~gsmith/content/linux-pdflush.htm http://www.westnet.com/~gsmith/content/linux-pdflush.htm for information about the Linux page cache.
  - barrkel 16y ago

ludwigvan 16y ago

free -m | awk 'NR==3 {print $4}'

colanderman 16y ago

This is misleading, as the kernel will often swap out processes rather than shrink the disk cache. Its tendency to do this is controlled by the vm.swappiness parameter, which by default heavily favors swapping: http://kerneltrap.org/node/3000 http://kerneltrap.org/node/3000 Decreasing this value will favor reducing the cache size and will tend to increase responsiveness. You can set it in the /etc/sysctl.conf file. Low values (≤ 5) are especially good for SSDs where you want to minimize swap usage.

kindlyviking 16y ago
The article mentions this on the second page at the bottom: http://www.linuxatemyram.com/play.html http://www.linuxatemyram.com/play.html
- colanderman 16y ago
  "But look you found the notice didn't you?" "Yes," said Arthur, "yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying 'Beware of The Leopard'."
jerf 16y ago
I often just turn swap off entirely. A normal workload on my 4GB linux machine will frequently fail to fill even 2GB of buffers. At the moment my machine has still managed to fail to fill 800MB of my RAM with anything and I'm running two virtual machines and doing my usual work load. I do have a swap partition but only so I can hibernate.
- requinot59 16y ago
  > I often just turn swap off entirely. And you're smart to do so. Swap is useless for 99% of end-user systems. ChromeOS doesn't use any swap partition, for instance. As for hibernation, it's not available if you use a crypted swap, which is wise to use (at least on a laptop). As a result, I, too, disabled swap completely.

wzdd 16y ago

This paragraph: 'But what do you call memory that is both used for something and available for applications? You would call that "free", but Linux calls it "used".' is IMO the best way I've ever seen of explaining the issue. It also reveals the solution: call it free! It is free, after all. Nobody cares about how much free ram is cache unless they are doing something really specific, in which case they should be passing specific flags, or using a specific tool, to report it. Unfortunately there is no chance Linux will do this, as the relevant reporting tools are too entrenched (and with good reason).

hernan7 16y ago
Maybe add a new (end user-friendly, preferably graphical) tool to report memory usage the user friendly way? Then vmstat can be left as is, and end users can be directed to use the new tool. By the way, this behavior is not Linux-specific. AIX and other Unices have been doing this forever. And end users (who should have known better, BTW) would complain too. The only machine that I remember as showing a sizable percentage of the memory as "free" was a just-imaged top of the line server. Half a gig or RAM, and almost all empty! -- this was the early '90s. Edit: Just noticed that graphical tools that report memory usage in user-friendly terms already exist (gnome system monitor). My bad, I always use vmstat and sar myself, and only recently found out about /proc/NNNNN/status
wrs 16y ago
Or just rename "free" to "wasted". I usually tell people the free number indicates how much RAM they paid for but aren't using.
- sliverstorm 16y ago
  It is admittedly difficult to re-learn what you know about computers, and what Windows has taught people (including myself) is that RAM usage must be below ~75% for optimal performance. It has been my experience that paging becomes much more blocking above that.

ktf 16y ago

It looks like gnome-system-monitor reports the "correct" amount of free RAM. I'd imagine most newbies would use that rather than popping open a terminal and typing "free," right?

NonMint 16y ago
You'd be surprised. Some users who use Linux consider themselves to be pretty adept, and I can easily see this problem being an issue for people who perceive themselves to be clever.
- deleted 16y ago
  [deleted]
- uxp 16y ago
  I wouldn't say that only "clever" users would rather use `free`, since most Linux installations don't have GNOME or X11 installed to begin with.

JonnieCache 16y ago

How does this compare to darwin's memory management?

Florin_Andrei 16y ago

It's merely a matter of definition - whether to include the disk cache in the "used" memory or not.

radu_floricica 16y ago

I have had the same problem, plus some, when trying to guesstimate memory usage in vps servers. The caching is transparent, so free always reports zero swap. ps gives sensible numbers, but the total used memory is neither RSS nor VSZ summed. I've heard about library memory being double-counted, but I've found no way to accurately get the numbers. In the end I pretty much gave up and settled for the settings that worked. tl;dr: on vps things can be even uglier.

stcredzero 16y ago

Hmm, this situation is like owning an electric vehicle. The manufacturers are passionate about technology, and they invented a flywheel that can store massive amounts of energy, giving your vehicle incredible transient acceleration response. They ship many vehicles with this new feature, but unfortunately, they forgot to account for what this would do to the battery charge display after starting. As a result, their reps keep on having to explain that the vehicles are not rapidly leaking charge shortly after starting -- instead that energy is being placed in the flywheel where it can greatly increase the car's responsiveness. Whoops! The difference between the scenario above and Linux's disk cache? Techies blame the former on the car company but the latter on the end-users.

Andys 16y ago
If only electric cars were free like Linux.
- stcredzero 16y ago
  My analogy has more to do with approaches to UX than it does with economic model.

jawee 16y ago

All this does is make me regret paying more for extra memory... With a dozen programs open, I am sitting at 2.3GB free, according to this. I guess it was wrong to splurge for that extra memory.

jefe78 16y ago

I love explaining it! The response is priceless, "Wow, that's pretty cool!" "No, that's Linux."

spudlyo 16y ago

You can't disable disk caching. The only reason anyone ever wants to disable disk caching is because they think it takes memory away from their applications, which it doesn't! Disk cache makes applications load faster and run smoother, but it NEVER EVER takes memory away from them! Therefore, there's absolutely no reason to disable it! Never say never. You can and should disable/purge linux buffer caching in certain situations. See O_DIRECT and/or POSIX_FADV_DONTNEED. If I'm writing a database, I take care of my own caching, I do not need the OS doing it as well. Also, if I'm doing a huge rsync of a directory tree, just because I just read every byte of every file in that tree, I most certainly do not want to load all of that crap into the linux buffer cache. It will evict my already warm set, as well as potentially cause a ton of swapping.

js2 16y ago
Funny, I just noticed this in the Mac OS X rsync man page for the very first time: --cache Apple specific option to enable filesystem caching of rsync file i/o Otherwise fcntl(F_NOCACHE) is used to limit memory growth.
- Nick_C 16y ago
  There was a patch for all architectures, but it got dropped after 3.07 (current is 3.09). Andrew wrote "I don't feel this patch is general enough nor straight-forward enough to make it into rsync." See these for more: http://insights.oetiker.ch/linux/fadvise/ http://insights.oetiker.ch/linux/fadvise/ https://bugzilla.samba.org/show_bug.cgi?id=7004 https://bugzilla.samba.org/show_bug.cgi?id=7004