3 ms·

Isn't summing RSS going to end up with the wrong numbers? My understanding is that RSS is memory allocated (heap + stack) + executable pages inclusive of share

Isn't summing RSS going to end up with the wrong numbers? My understanding is that RSS is memory allocated (heap + stack) + executable pages inclusive of shared libraries. i.e, if 10 processes load libfoobar which is 10MB of code, each processes RSS value will be 10MB higher. So if you sum RSS you see 10 x 10MB or 100MB extra usage. But the pages for that library will be shared across processes, with the result that "real ram usage" only goes up by 10MB. Please correct me if I'm wrong. This is based on my understanding of Linux too. FreeBSD might be different.

adamzegelin 4y ago
Just a quick experiment: I started 4 copies of VLC. `top` shows this: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1806287 adam 20 0 1091068 74664 56228 S 0.0 0.2 0:00.22 vlc 1806426 adam 20 0 1091064 74548 56128 S 0.0 0.2 0:00.21 vlc 1806537 adam 20 0 1091064 74572 56144 S 0.0 0.2 0:00.22 vlc 1806600 adam 20 0 1091064 74960 56536 S 0.0 0.2 0:00.23 vlc Summing RES (RSS) gives 298744. VLC loads many shared libraries: $ lsof -p 1806426 | grep .so | wc -l 195 I'd hope some/most of those are shared between the instances of VLC and are included in SHR (if I'm reading the top man page correctly). Doing RES-SHR isn't totally correct either. Those shared pages might be "marked as shared" (i.e, a .so that's only loaded by one process). But lets assume that most of what VLC uses as SHR is shared libraries, and they're used by each instance. RES-SHR = 73708 I also ran `free` before and after launching the 4 instances of VLC. Used increased by 66068. Hard to rely on the accuracy of this number since this is a desktop system and I'd imagine used goes up and down all the time. But it's strikingly close to the RES-SHR figure. IMHO just summing RES isn't correct. Linux memory usage is complex.
- vbernat 4y ago
  On Linux, you also have PSS which is like RSS, but the shared part is divided among the processes sharing the same memory.
- codedokode 4y ago
  SHR is not "shared" size, it is "potentially shareable size". Process might have 100 Mb of SHR and none of them actually shared. You should use PSS which accounts for shared memory pages (by dividing their size across all processes).
- grahamjperrin 4y ago
  > … RES (RSS) … If RES is resident memory <https://www.freebsd.org/cgi/man.cgi?query=top&sektion=1&manpath=FreeBSD#THE_DISPLAY https://www.freebsd.org/cgi/man.cgi?query=top&sektion=1&manp...>, what's RSS?
codedokode 4y ago
You are right. RSS values cannot be added. Imagine if there are 3 processes, each use 100 Mb of shared memory and 100 Mb of private. Summing the RSS we will get 200 + 200 + 200 = 600 Mb, however the true memory consumption is 100 (shared) + 100 + 100 + 100 = 400 Mb. To count the memory properly, you should divide size of shared segments by number of processes using it. This is how PSS in Linux is implemented. You can safely add PSS values of different processes. If you have swap, you should also sum swap usage by processes. As I understand, you can reliably count memory usage only in Linux because other OSes like Windows or Mac do not provide PSS in their Task Managers. Windows Task Manager at least has a documentation that describes what "memory" column means, and I failed to find such documentation for Mac, so we can safely assume that it displays random numbers remotely related to memory usage.

adamzegelin 4y ago

Just a quick experiment: I started 4 copies of VLC. `top` shows this: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1806287 adam 20 0 1091068 74664 56228 S 0.0 0.2 0:00.22 vlc 1806426 adam 20 0 1091064 74548 56128 S 0.0 0.2 0:00.21 vlc 1806537 adam 20 0 1091064 74572 56144 S 0.0 0.2 0:00.22 vlc 1806600 adam 20 0 1091064 74960 56536 S 0.0 0.2 0:00.23 vlc Summing RES (RSS) gives 298744. VLC loads many shared libraries: $ lsof -p 1806426 | grep .so | wc -l 195 I'd hope some/most of those are shared between the instances of VLC and are included in SHR (if I'm reading the top man page correctly). Doing RES-SHR isn't totally correct either. Those shared pages might be "marked as shared" (i.e, a .so that's only loaded by one process). But lets assume that most of what VLC uses as SHR is shared libraries, and they're used by each instance. RES-SHR = 73708 I also ran `free` before and after launching the 4 instances of VLC. Used increased by 66068. Hard to rely on the accuracy of this number since this is a desktop system and I'd imagine used goes up and down all the time. But it's strikingly close to the RES-SHR figure. IMHO just summing RES isn't correct. Linux memory usage is complex.

vbernat 4y ago
On Linux, you also have PSS which is like RSS, but the shared part is divided among the processes sharing the same memory.
codedokode 4y ago
SHR is not "shared" size, it is "potentially shareable size". Process might have 100 Mb of SHR and none of them actually shared. You should use PSS which accounts for shared memory pages (by dividing their size across all processes).
grahamjperrin 4y ago
> … RES (RSS) … If RES is resident memory <https://www.freebsd.org/cgi/man.cgi?query=top&sektion=1&manpath=FreeBSD#THE_DISPLAY https://www.freebsd.org/cgi/man.cgi?query=top&sektion=1&manp...>, what's RSS?

codedokode 4y ago

You are right. RSS values cannot be added. Imagine if there are 3 processes, each use 100 Mb of shared memory and 100 Mb of private. Summing the RSS we will get 200 + 200 + 200 = 600 Mb, however the true memory consumption is 100 (shared) + 100 + 100 + 100 = 400 Mb. To count the memory properly, you should divide size of shared segments by number of processes using it. This is how PSS in Linux is implemented. You can safely add PSS values of different processes. If you have swap, you should also sum swap usage by processes. As I understand, you can reliably count memory usage only in Linux because other OSes like Windows or Mac do not provide PSS in their Task Managers. Windows Task Manager at least has a documentation that describes what "memory" column means, and I failed to find such documentation for Mac, so we can safely assume that it displays random numbers remotely related to memory usage.