4 ms·

One of my interview questions starts with "Can a program allocate more memory than is physically available on the server?" Everybody gets this wrong (which is

One of my interview questions starts with "Can a program allocate more memory than is physically available on the server?" Everybody gets this wrong (which is funny for a binary question) but it starts an interesting discussion through which I hope to learn how much they know about OS and virtual memory.

floxy 5y ago
Don't leave us hanging. The "obvious" answer would seem to be "yes" because of swap. But if everyone gets that wrong...
- foobarian 5y ago
  It's a vague question though. What kind of OS? What kind of runtime? Does "allocate" mean call malloc() and get non-null return value?
  - lmilcin 5y ago
    If you can point all that stuff out I will already be a little happier as an interviewer. Most people just say "no".
    - foobarian 5y ago
      Fair enough, I wouldn’t be happy with a single word either, no matter if “yes” or “no.”
- yjftsjthsd-h 5y ago
  No, it's worse than that - the answer is "yes", because virtual memory + overcommit means that most of the time the OS will happily allow you to allocate more memory than physical+swap, and essentially gamble that you won't actually need all of it (and this is implemented because apparently that's almost always a winning bet).
  - lmilcin 5y ago
    Yeah. And the issue is that the actual problem happens sometime later when the application actually tries to use that memory. So you replaced an error that is relatively simple to handle with something that is impossible to handle reliably. So the operating system very much doesn't like to admit it doesn't have physical memory to back the area you are trying to use. Now it does not have a simple way to signal this to the application (there is no longer an option to return an error code) and so either everything slows down (as OS hopes that another process will return a little bit of memory to get things going for a little while) or one of the processes gets killed.
    - kzrdude 5y ago
      And we additionally end up in a feedback loop where coders don't check the return value from malloc(!) Since why bother when it never errors. Then we don't have enough resilience capital there either.
    - tsimionescu 5y ago
      Interestingly, Linux's OOMKiller actually gives you (the sysadmin or system designer) more control over what happens when the system is low on memory than disabling overcommit. In a system without overcommit, every process is taking memory out of the shared pool, until some random process is the unlucky one that can't allocate more. In a happy case, that unlucky process also has some data that it can let go of. But this is entirely random - you could have a bunch of gigabyte-sized application caches in half of your processes, but the NTP daemon might be the one who cends up failing because it can't allocate a few more bytes. Even worse, it could be the SSH server or bash failing to spawn a new shell, preventing any kind of intervention on the system. With OOMKiller, you can at least define some priorities, and ensure some critical processes are never going to be killed or stalled.
      - kaba0 5y ago
        There is userspace integration available as well for OOM killers, afaik Facebook using/having developed them. They can take into account much more fine-grained selection criteria before killing a process.
prabhu-yu 5y ago
Should the question be like this? "Can a program ~~allocate~~ use more memory than is physically available on the server (assuming no swap)?" Because, malloc allocates from virtual memory, not from the physical memory. Only when we access the the memory explicitly by read/write for the first time, then the page fault happens and page allocation begins. If page can't be allocated, then program gets terminated with a SIGNAL. Here, malloc was successful since allocation from VM was successful. But that is no guarantee that we have all memory we got. We can still run out of memory even though we allocated everything at start of program... It is so un predictable ... So, I assume, your question is asked with above intention. Please correct me if I am wrong. Your question and whole thread made me to think! -p

floxy 5y ago

Don't leave us hanging. The "obvious" answer would seem to be "yes" because of swap. But if everyone gets that wrong...

foobarian 5y ago
It's a vague question though. What kind of OS? What kind of runtime? Does "allocate" mean call malloc() and get non-null return value?
- lmilcin 5y ago
  If you can point all that stuff out I will already be a little happier as an interviewer. Most people just say "no".
  - foobarian 5y ago
    Fair enough, I wouldn’t be happy with a single word either, no matter if “yes” or “no.”
yjftsjthsd-h 5y ago
No, it's worse than that - the answer is "yes", because virtual memory + overcommit means that most of the time the OS will happily allow you to allocate more memory than physical+swap, and essentially gamble that you won't actually need all of it (and this is implemented because apparently that's almost always a winning bet).
- lmilcin 5y ago
  Yeah. And the issue is that the actual problem happens sometime later when the application actually tries to use that memory. So you replaced an error that is relatively simple to handle with something that is impossible to handle reliably. So the operating system very much doesn't like to admit it doesn't have physical memory to back the area you are trying to use. Now it does not have a simple way to signal this to the application (there is no longer an option to return an error code) and so either everything slows down (as OS hopes that another process will return a little bit of memory to get things going for a little while) or one of the processes gets killed.
  - kzrdude 5y ago
    And we additionally end up in a feedback loop where coders don't check the return value from malloc(!) Since why bother when it never errors. Then we don't have enough resilience capital there either.
  - tsimionescu 5y ago
    Interestingly, Linux's OOMKiller actually gives you (the sysadmin or system designer) more control over what happens when the system is low on memory than disabling overcommit. In a system without overcommit, every process is taking memory out of the shared pool, until some random process is the unlucky one that can't allocate more. In a happy case, that unlucky process also has some data that it can let go of. But this is entirely random - you could have a bunch of gigabyte-sized application caches in half of your processes, but the NTP daemon might be the one who cends up failing because it can't allocate a few more bytes. Even worse, it could be the SSH server or bash failing to spawn a new shell, preventing any kind of intervention on the system. With OOMKiller, you can at least define some priorities, and ensure some critical processes are never going to be killed or stalled.
    - kaba0 5y ago
      There is userspace integration available as well for OOM killers, afaik Facebook using/having developed them. They can take into account much more fine-grained selection criteria before killing a process.

prabhu-yu 5y ago

Should the question be like this? "Can a program ~~allocate~~ use more memory than is physically available on the server (assuming no swap)?" Because, malloc allocates from virtual memory, not from the physical memory. Only when we access the the memory explicitly by read/write for the first time, then the page fault happens and page allocation begins. If page can't be allocated, then program gets terminated with a SIGNAL. Here, malloc was successful since allocation from VM was successful. But that is no guarantee that we have all memory we got. We can still run out of memory even though we allocated everything at start of program... It is so un predictable ... So, I assume, your question is asked with above intention. Please correct me if I am wrong. Your question and whole thread made me to think! -p