4 ms·

I think this is a pretty common misconception overall. Working at startups, often find devs with multiple hats sometimes doing ops tasks. Seen many who hop on

I think this is a pretty common misconception overall. Working at startups, often find devs with multiple hats sometimes doing ops tasks. Seen many who hop on a system trying to diagnose some issue, fire up top and proclaim "OMG, the problem is we're running out of memory!" 2nd is explaining virtual/resident set size.

francoisdevlin 15y ago
Okay, I'm one of those devs. What is this virtual/resident set size thing you're talking about? EDIT: Thank you for all the helpful responses!
- ivan78 15y ago
  Well, if you don't understand it, you must read some basic book about modern computer architecture. Short answer: It is possible to reserve memory address space without assigning actual physical memory. As your program runs it can dynamically assign physical memory pages to its address space when it needs it. Again - you must read a book if you want to understand it.
  - ivan78 15y ago
    Why downvoting? It may be not the best explanation, but IMO the closest to the way a "developer" should understand it. On Linux a virtual memory is committed with mmap call. Check this for details: http://stackoverflow.com/questions/2782628/any-way-to-reserve-but-not-commit-memory-in-linux http://stackoverflow.com/questions/2782628/any-way-to-reserv... This is basically how shared libraries and other shared objects are attached to your process.
    - henrikschroder 15y ago
      > Why downvoting? > you must read a book if you want to understand it. "Go read a book" is not a very constructive or helpful answer to a question that - as it turns out - could be answered with a brief comment.
      - onemoreact 15y ago
        It can be when a short description completely and accurately describes what's going on, but the person creating the comment needed to read a few books on the subject to actually understand both what's happening and why due to other issues. EX: How does virtual memory get mapped to L1 cache?
    - ivan78 15y ago
      to henrikschroder: I expected that this reaction was caused by mentioning a book. It is my deepest belief that questions like memory management can not be answered with brief comment. You must read at least a basic book if you want to understand it in depth.
- Steve_Baker 15y ago
  The virtual size of a program includes all shared objects (shared libraries, shared copy on write memory pages, the read only executable pages and other shared memory) that a process uses in addition to the memory that only that particular process is using (its resident set size.) Shared objects like shared libraries can be mapped into memory by multiple processes and thus don't use any additional physical memory for each additional process that uses it. The RSS thus usually indicates the amount of heap and stack a process is using that is unique to it.
  - mgedmin 15y ago
    RSS also includes shared memory pages (those that are currently resident).
- illumin8 15y ago
  Short and overly simple answer: If a program allocates 2GB of RAM, it might not need to use it all right away. The kernel pretends like it just gave the program 2GB of RAM, but it doesn't really need to flush all those unused pages of cache yet. It can still use them for other things like disk cache, and when the program tries to access those pages then the kernel will dynamically page them in and out as needed. This explains why a program might have 2GB virtual but only 128MB is resident in actual physical memory pages. This is overly simple and there are a lot of nuances such as shared memory segments, etc.
  - adavies42 15y ago
    then there's over-commit, e.g. "you want to malloc 200% of the ram? here you go! enjoy! oh wait, you wrote to the second half? have fun finding the power button, sucker!" can someone explain to me why so many distros, including "enterprise" server stuff, ship with /proc/sys/vm/overcommit_memory set to 1?
    - aliguori 15y ago
      Most software cannot cope with malloc failing. A little swapping is at least recoverable. A failed malloc almost guarantees that your application will crash--sometimes in a very unpleasant way.
      - adavies42 15y ago
        ime the main alternative is the box crashing, frequently without leaving behind enough information to know what went wrong. at least if the app crashes you have a pretty good idea who was incompetent.
- maushu 15y ago
  I'm a developer too and this is what I understand (might be wrong): - Virtual memory is basically "abstract memory" that is linked (mapped) to RAM or HDD, this means that by accessing this "memory" you might actually be accessing the HDD and, because of this, larger than physical ram. - Virtual set size is the allocated virtual memory (the above) to the process. - Resident set size is the allocated physical (RAM) memory to the process. - Shared memory is memory that is shared on multiple processes, meaning that if you have 10 processes using 10mb of resident memory each and have 2mb of shared memory each, the total of resident memory used is not 100mb but 82mb.
  - scott_s 15y ago
    Virtual memory is actually more subtle than that. At a mechanical level, virtual memory is permission from the operating system to use addresses in your address space. It is so called because, as you point out, it allows us to separate the concept of "memory for a process" from "physical memory on a chip." The reason I further refine the concept is that allocating virtual memory does not allocate actual memory. Let's look at an example: void* addr = mmap(NULL, 10 * 1024 * 1024, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); That call allocates 10 MB of virtual memory. But there is no physical memory backing any of it. All that has happened is that the operating system has now said "Okay, starting at the address I return to you, you can now access 10 MB of memory. I will do all of the work of making sure physical memory backs the virtual memory when you access it." That is, once I try to access the memory, it will trigger a page fault, and the OS will find a page in physical memory to back my virtual memory. But until that happens, no memory - not in RAM, not on disk - backs the virtual memory.
  - qwe123_troll 15y ago
    No, not at all. Virtual memory isn't really memory at all, it's just an entry in a table which marks addresses as accessible to your process. It's up to the OS to decide whether or not these addresses correspond to anything physical or not. In Linux (and every other modern OS, AFAIK) the OS allocates physical memory on-the-fly, only when your program actually starts reading or writing addresses in the virtual memory table. Thus, on a 64-bit machine you can allocate thousands of gigabytes of "virtual memory" without using any real memory at all.
- sciurus 15y ago
  There's a good overview of this in the first four pages of http://www.atoptool.nl/download/case_leakage.pdf http://www.atoptool.nl/download/case_leakage.pdf The rest of that document shows how to use atop to monitor per-process memory usage and identify a memory leak.
- yread 15y ago
  On Windows it's called committed memory http://blogs.technet.com/b/markrussinovich/archive/2008/11/17/3155406.aspx#Committed_Memory http://blogs.technet.com/b/markrussinovich/archive/2008/11/1... (the article doesn't have an anchor there... but you know what I mean)
ajross 15y ago
That's depressing. Developers are the ones who have to understand this. In IT it's common to find people (even "DBAs") who have made a career of following procedures someone else wrote.
- covati 15y ago
  I don't think that is necessarily true. Most modern programming languages, as well as the relative cheap nature of RAM, have rendered a true understanding of the inner workings of RAM less important. Would they be better at their job if they did? Probably. But do they have to? I'm not so sure anymore.
  - ajross 15y ago
    Sure, but we're not talking about the "narrow, career day-job programmers" here. We're talking about the "wear multiple hats and diagnose problems wherever they lie" set. Those people shouldn't be diagnosing problems with their own ignorance. At the very least they should understand what they don't know.
    - krobertson 15y ago
      Agreed. It can sometimes be a side effect of small companies, or young developers living the startup lifestyle. Not saying its bad or the root of the problem, just an observation. More so, I think it comes more with modern languages that don't force you to manage memory anymore. When you don't need to alloc/free everything you're using, it is easy to get lazy. Factor in never generation who've never worked outside a managed memory language. There are minute details about retaining references, having multiple copies of the same data, or other leaks (file descriptors?).
Legion 15y ago
2 minute fix: get people to use htop, not top.
- JadeRobbins 15y ago
  Thanks! htop is MUCH better than plain ol top.
  - spudlyo 15y ago
    You'd be surprised how good and feature rich plain ole top actually is. My guess is you rarely deviate from the default options. I made a video which shows how you can get top to show you some pretty interesting stuff. http://www.youtube.com/watch?v=yFKRsLj_Jhg http://www.youtube.com/watch?v=yFKRsLj_Jhg
    - laurentoget 15y ago
      thanks for sharing!
- sciurus 15y ago
  Having tried both, I'd recommend atop over htop. There's a great overview of its capabilities at https://lwn.net/Articles/387202/ https://lwn.net/Articles/387202/
  - mgedmin 15y ago
    While more powerful, atop is rather confusing. htop is friendly.
    - sciurus 15y ago
      I'm not sure what about using atop is confusing, but I'll admit to printing out and referring to the manpage when I discovered it. That was as much because of the great explanations atop's manual provided for the metrics it could display as it was because I wanted to learn the interactive commands. Some things about htop that I could see as friendlier: * Htop scrolls with the arrow keys, while atop uses ^F and ^B. * Htop displays a reminder for some commonly-used commands at the bottom of the screen (e.g. that you need to press 'F1' for help), whereas in atop you have to remember the commands or look them up by pressing 'h' or '?'. * Htop displays gauges for system-level activity. I think this is a bad tradeoff, though: CPU | sys 0% | user 1% | irq 0% | idle 799% | wait 0% | MEM | tot 7.8G | free 5.7G | cache 735.6M | buff 351.2M | slab 248.1M is much more useful than Avg[| 0.2%] Mem[||||||||||||||||| 1008/7967MB] There are features of htop that I wish atop had. For example, the toggleable display of threads, the tree view, and integration with strace and lsof. Even without those I find atop more useful, but YMMV. The killer feature for atop is logging per-process performance data and reviewing it after the fact.

francoisdevlin 15y ago

Okay, I'm one of those devs. What is this virtual/resident set size thing you're talking about? EDIT: Thank you for all the helpful responses!

ivan78 15y ago
Well, if you don't understand it, you must read some basic book about modern computer architecture. Short answer: It is possible to reserve memory address space without assigning actual physical memory. As your program runs it can dynamically assign physical memory pages to its address space when it needs it. Again - you must read a book if you want to understand it.
- ivan78 15y ago
  Why downvoting? It may be not the best explanation, but IMO the closest to the way a "developer" should understand it. On Linux a virtual memory is committed with mmap call. Check this for details: http://stackoverflow.com/questions/2782628/any-way-to-reserve-but-not-commit-memory-in-linux http://stackoverflow.com/questions/2782628/any-way-to-reserv... This is basically how shared libraries and other shared objects are attached to your process.
  - henrikschroder 15y ago
    > Why downvoting? > you must read a book if you want to understand it. "Go read a book" is not a very constructive or helpful answer to a question that - as it turns out - could be answered with a brief comment.
    - onemoreact 15y ago
      It can be when a short description completely and accurately describes what's going on, but the person creating the comment needed to read a few books on the subject to actually understand both what's happening and why due to other issues. EX: How does virtual memory get mapped to L1 cache?
  - ivan78 15y ago
    to henrikschroder: I expected that this reaction was caused by mentioning a book. It is my deepest belief that questions like memory management can not be answered with brief comment. You must read at least a basic book if you want to understand it in depth.
Steve_Baker 15y ago
The virtual size of a program includes all shared objects (shared libraries, shared copy on write memory pages, the read only executable pages and other shared memory) that a process uses in addition to the memory that only that particular process is using (its resident set size.) Shared objects like shared libraries can be mapped into memory by multiple processes and thus don't use any additional physical memory for each additional process that uses it. The RSS thus usually indicates the amount of heap and stack a process is using that is unique to it.
- mgedmin 15y ago
  RSS also includes shared memory pages (those that are currently resident).
illumin8 15y ago
Short and overly simple answer: If a program allocates 2GB of RAM, it might not need to use it all right away. The kernel pretends like it just gave the program 2GB of RAM, but it doesn't really need to flush all those unused pages of cache yet. It can still use them for other things like disk cache, and when the program tries to access those pages then the kernel will dynamically page them in and out as needed. This explains why a program might have 2GB virtual but only 128MB is resident in actual physical memory pages. This is overly simple and there are a lot of nuances such as shared memory segments, etc.
- adavies42 15y ago
  then there's over-commit, e.g. "you want to malloc 200% of the ram? here you go! enjoy! oh wait, you wrote to the second half? have fun finding the power button, sucker!" can someone explain to me why so many distros, including "enterprise" server stuff, ship with /proc/sys/vm/overcommit_memory set to 1?
  - aliguori 15y ago
    Most software cannot cope with malloc failing. A little swapping is at least recoverable. A failed malloc almost guarantees that your application will crash--sometimes in a very unpleasant way.
    - adavies42 15y ago
      ime the main alternative is the box crashing, frequently without leaving behind enough information to know what went wrong. at least if the app crashes you have a pretty good idea who was incompetent.
maushu 15y ago
I'm a developer too and this is what I understand (might be wrong): - Virtual memory is basically "abstract memory" that is linked (mapped) to RAM or HDD, this means that by accessing this "memory" you might actually be accessing the HDD and, because of this, larger than physical ram. - Virtual set size is the allocated virtual memory (the above) to the process. - Resident set size is the allocated physical (RAM) memory to the process. - Shared memory is memory that is shared on multiple processes, meaning that if you have 10 processes using 10mb of resident memory each and have 2mb of shared memory each, the total of resident memory used is not 100mb but 82mb.
- scott_s 15y ago
  Virtual memory is actually more subtle than that. At a mechanical level, virtual memory is permission from the operating system to use addresses in your address space. It is so called because, as you point out, it allows us to separate the concept of "memory for a process" from "physical memory on a chip." The reason I further refine the concept is that allocating virtual memory does not allocate actual memory. Let's look at an example: void* addr = mmap(NULL, 10 * 1024 * 1024, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); That call allocates 10 MB of virtual memory. But there is no physical memory backing any of it. All that has happened is that the operating system has now said "Okay, starting at the address I return to you, you can now access 10 MB of memory. I will do all of the work of making sure physical memory backs the virtual memory when you access it." That is, once I try to access the memory, it will trigger a page fault, and the OS will find a page in physical memory to back my virtual memory. But until that happens, no memory - not in RAM, not on disk - backs the virtual memory.
- qwe123_troll 15y ago
  No, not at all. Virtual memory isn't really memory at all, it's just an entry in a table which marks addresses as accessible to your process. It's up to the OS to decide whether or not these addresses correspond to anything physical or not. In Linux (and every other modern OS, AFAIK) the OS allocates physical memory on-the-fly, only when your program actually starts reading or writing addresses in the virtual memory table. Thus, on a 64-bit machine you can allocate thousands of gigabytes of "virtual memory" without using any real memory at all.
sciurus 15y ago
There's a good overview of this in the first four pages of http://www.atoptool.nl/download/case_leakage.pdf http://www.atoptool.nl/download/case_leakage.pdf The rest of that document shows how to use atop to monitor per-process memory usage and identify a memory leak.
yread 15y ago
On Windows it's called committed memory http://blogs.technet.com/b/markrussinovich/archive/2008/11/17/3155406.aspx#Committed_Memory http://blogs.technet.com/b/markrussinovich/archive/2008/11/1... (the article doesn't have an anchor there... but you know what I mean)

ajross 15y ago

That's depressing. Developers are the ones who have to understand this. In IT it's common to find people (even "DBAs") who have made a career of following procedures someone else wrote.

covati 15y ago
I don't think that is necessarily true. Most modern programming languages, as well as the relative cheap nature of RAM, have rendered a true understanding of the inner workings of RAM less important. Would they be better at their job if they did? Probably. But do they have to? I'm not so sure anymore.
- ajross 15y ago
  Sure, but we're not talking about the "narrow, career day-job programmers" here. We're talking about the "wear multiple hats and diagnose problems wherever they lie" set. Those people shouldn't be diagnosing problems with their own ignorance. At the very least they should understand what they don't know.
  - krobertson 15y ago
    Agreed. It can sometimes be a side effect of small companies, or young developers living the startup lifestyle. Not saying its bad or the root of the problem, just an observation. More so, I think it comes more with modern languages that don't force you to manage memory anymore. When you don't need to alloc/free everything you're using, it is easy to get lazy. Factor in never generation who've never worked outside a managed memory language. There are minute details about retaining references, having multiple copies of the same data, or other leaks (file descriptors?).

Legion 15y ago

2 minute fix: get people to use htop, not top.

JadeRobbins 15y ago
Thanks! htop is MUCH better than plain ol top.
- spudlyo 15y ago
  You'd be surprised how good and feature rich plain ole top actually is. My guess is you rarely deviate from the default options. I made a video which shows how you can get top to show you some pretty interesting stuff. http://www.youtube.com/watch?v=yFKRsLj_Jhg http://www.youtube.com/watch?v=yFKRsLj_Jhg
  - laurentoget 15y ago
    thanks for sharing!
sciurus 15y ago
Having tried both, I'd recommend atop over htop. There's a great overview of its capabilities at https://lwn.net/Articles/387202/ https://lwn.net/Articles/387202/
- mgedmin 15y ago
  While more powerful, atop is rather confusing. htop is friendly.
  - sciurus 15y ago
    I'm not sure what about using atop is confusing, but I'll admit to printing out and referring to the manpage when I discovered it. That was as much because of the great explanations atop's manual provided for the metrics it could display as it was because I wanted to learn the interactive commands. Some things about htop that I could see as friendlier: * Htop scrolls with the arrow keys, while atop uses ^F and ^B. * Htop displays a reminder for some commonly-used commands at the bottom of the screen (e.g. that you need to press 'F1' for help), whereas in atop you have to remember the commands or look them up by pressing 'h' or '?'. * Htop displays gauges for system-level activity. I think this is a bad tradeoff, though: CPU | sys 0% | user 1% | irq 0% | idle 799% | wait 0% | MEM | tot 7.8G | free 5.7G | cache 735.6M | buff 351.2M | slab 248.1M is much more useful than Avg[| 0.2%] Mem[||||||||||||||||| 1008/7967MB] There are features of htop that I wish atop had. For example, the toggleable display of threads, the tree view, and integration with strace and lsof. Even without those I find atop more useful, but YMMV. The killer feature for atop is logging per-process performance data and reviewing it after the fact.