5 ms·

Memory FAQ

291 points by mbel 10y ago

jdi 10y ago
https://github.com/0xAX/linux-insides https://github.com/0xAX/linux-insides A different source for documentation on the Linux insides, for those who don't already know about it.
- bogomipz 10y ago
  Yeah this is a really great resource, I'm looking forward to seeing the final book.
kensai 10y ago
Interesting information. But this FAQ collection should have a date. Is it always relevant?
- thomaslee 10y ago
  +1 on the date, but generally relevant to most modern x86/x86-64 hardware IMO. (The rest of this comment is largely my own inflated opinion.) From a developer's perspective, whether it's _useful_ when operating at a certain level of abstraction is another question. Really do think it's good to know this stuff, but if you're writing, say, Ruby or Python web apps you probably don't _need_ to know it. Low latency/high throughput/systems-level stuff it starts to become more relevant. That said, there's certainly been operational scenarios where knowing some of this stuff can help narrow down misconfigured systems and/or squeeze more life out of existing hardware (e.g. configuring a JVM to use huge pages).
- schmichael 10y ago
  ~$ curl -v http://nommu.org/memory-faq.txt > /dev/null > GET /memory-faq.txt HTTP/1.1 > Host: nommu.org > < HTTP/1.1 200 OK < Server: GitHub.com < Last-Modified: Thu, 05 May 2016 01:37:27 GMT Hm, github you say? https://github.com/nommu/nommu.org https://github.com/nommu/nommu.org Jun 8, 2015 is the last commit to this particular file.
twoodfin 10y ago
Another benefit of huge pages: Some database systems use a model where client processes operate on a massive pool of shared memory (typically a buffer cache). This allows the system to leverage the OS process model for isolation, privileges, etc. Huge pages greatly reduce the overhead of new client processes lazily mapping that pool into their address space.
- eloff 10y ago
  This helps e.g. Postgres which has a process per connection model. MySQL uses threads, which is lighter weight and doesn't have this problem.
zokier 10y ago
Some paragraphs are weirdly duplicated, such as "The OOM killer's process-killing capability is a reasonable way to deal with runaway processes [...]" and "People who don't understand how virtual memory works often insist on tracking the relationship between virtual and physical memory [...]"
zymhan 10y ago
That was interesting to read, but I don't understand what the context is at all.
- mbel 10y ago
  The FAQ is hosted my NOMMULinux project (nomen omen a Linux distribution that is not using virtual memory, it was discussed on HN earlier today). I found it to be a very nice summary of how OS manages memory, that is quite interesting even without context. That's why I posted it. It is a little bit light on the details, but in my opinion it is able to show memory management as a complete process, which many other materials fail to do (at least the ones that I've seen).
  - wahern 10y ago
    Some of us disagree violently with that FAQ's discussion of the OOM killer and strict overcommit. The arguments aren't even coherent. For example, I disable overcommit and run my machines without any swap whatsoever. (Though this doesn't resolve the problem entirely in Linux.) The way to deal with the reality of finite memory isn't to pretend like it's infinite and then randomly kill processes when you hit a wall; or to use magic constants in user processes that try to guess the maximum number of logical tasks that can be handled for some amount of system RAM (which you could never possibly do correctly or consistently as a general matter for non-trivial purposes). Rather, I would argue that overcommit encourages people to allocate huge amounts of swap in an attempt to avoid the OOM killer. And I would argue that overcommit and swap make it much easier to DoS a site, all things being equal. But certainly overcommit makes it more difficult to implement resilient software. The solution is to fail allocation requests as soon as possible, and to fail them inline (not as signals). That way user processes can unwind and isolate the logical task attempting the allocation. Or if that's too difficult and inconvenient, a process can simply choose to panic. Without overcommit a process can choose the policy most suitable for the manner in which it was implemented. Among other things, that creates actionable back-pressure bound to the logic request. The point about ELF shared libraries is just plain wrong. A shared library is a file, which already has a backing store in the filesystem. You need neither overcommit nor swap space to be able to map a shared library file into virtual address space and share it across multiple processes. Lots of operating system do this without overcommitting physical RAM. And because all modern systems have a unified buffer cache, you actually get better performance by keeping those pages backed by the regular file system. If you treat them like anonymous pages, you have to copy them back out to swap space under memory pressure, whereas if they remained file-backed you don't have to write them out at all. Likewise, if a process wants to speculatively allocate a huge buffer that might possibly go unused, rather than using malloc or anonymous mmap, create and map a temporary file. For a dozen different reasons this is preferable. The only possible negative is that it doesn't kow-tow to broken software that didn't think about memory issues. But I don't want to install that software, anyhow, because if the author couldn't put in the minimal amount of effort to be able to work on systems without overcommit, they're unlikely to have put much thought into buffer overflows, integer overflows, and many other correctness and security issues. Finally, because Linux has to obey traditional Unix resources limit, as well as cgroup limits, it has to maintain an accounting of allocated pages whether or not overcommit is enabled. That fact should also be a strong hint that proper memory accounting is also a security concern, not simply an annoyance that we can wave away.
valarauca1 10y ago
Really good read. Saved in my book mark. Solid information dense document.
AceJohnny2 10y ago
> Virtual address ranges for which there is current memory mapping are said to be "unmapped" Probably supposed to read "for which there is no current memory mapping" ?
- davidsong 10y ago
  Yeah I spotted that too. Another posted pointed out that it's on github, so you could always submit a pull request: https://github.com/nommu/nommu.org https://github.com/nommu/nommu.org
waynecochran 10y ago
I always wondered if forking a giant processes (i.e., large memory footprint) to create a small processes caused a lot of needless memory to be allocated, but now I understand why its no big deal: For example, the fork/exec combo creates transient virtual memory usage spikes, which go away again almost immediately without ever breaking the copy on write status of most of the pages in the forked page tables. Thus if a large process forks off a smaller process, enormous physical memory demands threaten to happen (as far as overcommit is concerned), but never materialize.
- paulfurtado 10y ago
  Note that this also occasionally results in "failure to fork" with the default linux overcommit setting. At fork-time if there isn't enough memory to duplicate the entire process, the fork call fails, even though you never intended to use that memory. You can use vfork to get around this.
- swiley 10y ago
  I believe the fork() manpage (or at least the clone() manpage) actually mentions this.
sigcode 10y ago
I now have GB's of RAM and most times I have no shortage of physical memory; I have more than enough to run all my small, statically compiled programs in their own memory space. While I know the kernel is still designed for it, I have not used a dedicated "swap partition" in a decade. Hardware has come so far from the limitations of the old days; it is my dream to have the option to compile a UNIX-like kernel with virtual memory "disabled". ./build.sh -nommu
quotemstr 10y ago
The FAQ's tone of dismissive superiority with respect to overcommit is very offputting. The author suggests that only people who don't know how virtual memory works can oppose overcommit; it's curious then how the author reveals some virtual memory knowledge gaps. Specifically, the difference between address space reservation (which does not require backing storage) and commit charge (which does) seems lost on him. Yes, you need backing storage to handle the possibility that a process might actually make copies of all COW pages (that's what we mean by "commit charge" --- the number of pages the kernel has voluntarily promised to deliver at some point. The right way to reduce the pressure on commit that these mappings cause is to not have so many of them in the first place. There's no reason PIC shared libraries need to be PROT_WRITE; there's no reason big programs like web browsers need to use fork(2) instead of the VM-efficient vfork(2). (Small programs can use fork and its COW commit charge without causing real problems, even on non-sloppy (err, "strict overcommit") systems. Linux's default overcommit policy is terrible in part because it discourages people from learning about the difference between reserving address space and actually reserving memory, and the "FAQ" just confuses the issue more.
- pps 10y ago
  What is the best book/site to learn this all in good way?
- Jasper_ 10y ago
  vfork is specified by POSIX that you cannot even do so much as close an fd or call setenv between fork and exec, both of which are very common things to want to do to set up an environment before something is run. I disagree with the author too about overcommit, but vfork isn't a realistic solution.
  - quotemstr 10y ago
    In your vfork, you can exec another process that does all these things in a fresh environment. Realistically, you'll want a posix_spawn interface that uses vfork internally.
derricgilling 10y ago
Under your TLB section, you mention: "The TLB is a cache for the MMU. All memory in the CPU's L1 cache must have an associated TLB entry, and invalidating a TLB entry flushes the associated cache line(s)." Many CPUs such as x86 ones don't actually flush the L1 on a TLB eviction, there would be no reason to. Eventually the line will be evicted through a snoop, capacity eviction, C6 transition etc. Of course, if a later memory access comes to re-read the line, the TLB will have to be repopulated in order to have a mapping. Also, on the "appeal of huge pages", I would put what are the negatives. Huge pages have appeal of reducing TLB pressure for stuff like big data/HPC workloads with large contiguous data sets, but it can also increase page fault latency and memory usage for more fragmented workloads that touch many pages. For example, on persist, Redis forks a process that may touch many pages. Each COW would have to find room for and allocate 2MB rather than just 4K

291 points by mbel 10y ago

jdi 10y ago

https://github.com/0xAX/linux-insides https://github.com/0xAX/linux-insides A different source for documentation on the Linux insides, for those who don't already know about it.

bogomipz 10y ago
Yeah this is a really great resource, I'm looking forward to seeing the final book.

kensai 10y ago

Interesting information. But this FAQ collection should have a date. Is it always relevant?

thomaslee 10y ago
+1 on the date, but generally relevant to most modern x86/x86-64 hardware IMO. (The rest of this comment is largely my own inflated opinion.) From a developer's perspective, whether it's _useful_ when operating at a certain level of abstraction is another question. Really do think it's good to know this stuff, but if you're writing, say, Ruby or Python web apps you probably don't _need_ to know it. Low latency/high throughput/systems-level stuff it starts to become more relevant. That said, there's certainly been operational scenarios where knowing some of this stuff can help narrow down misconfigured systems and/or squeeze more life out of existing hardware (e.g. configuring a JVM to use huge pages).
schmichael 10y ago
~$ curl -v http://nommu.org/memory-faq.txt > /dev/null > GET /memory-faq.txt HTTP/1.1 > Host: nommu.org > < HTTP/1.1 200 OK < Server: GitHub.com < Last-Modified: Thu, 05 May 2016 01:37:27 GMT Hm, github you say? https://github.com/nommu/nommu.org https://github.com/nommu/nommu.org Jun 8, 2015 is the last commit to this particular file.

twoodfin 10y ago

Another benefit of huge pages: Some database systems use a model where client processes operate on a massive pool of shared memory (typically a buffer cache). This allows the system to leverage the OS process model for isolation, privileges, etc. Huge pages greatly reduce the overhead of new client processes lazily mapping that pool into their address space.

eloff 10y ago
This helps e.g. Postgres which has a process per connection model. MySQL uses threads, which is lighter weight and doesn't have this problem.

zokier 10y ago

Some paragraphs are weirdly duplicated, such as "The OOM killer's process-killing capability is a reasonable way to deal with runaway processes [...]" and "People who don't understand how virtual memory works often insist on tracking the relationship between virtual and physical memory [...]"

zymhan 10y ago

That was interesting to read, but I don't understand what the context is at all.

mbel 10y ago
The FAQ is hosted my NOMMULinux project (nomen omen a Linux distribution that is not using virtual memory, it was discussed on HN earlier today). I found it to be a very nice summary of how OS manages memory, that is quite interesting even without context. That's why I posted it. It is a little bit light on the details, but in my opinion it is able to show memory management as a complete process, which many other materials fail to do (at least the ones that I've seen).
- wahern 10y ago
  Some of us disagree violently with that FAQ's discussion of the OOM killer and strict overcommit. The arguments aren't even coherent. For example, I disable overcommit and run my machines without any swap whatsoever. (Though this doesn't resolve the problem entirely in Linux.) The way to deal with the reality of finite memory isn't to pretend like it's infinite and then randomly kill processes when you hit a wall; or to use magic constants in user processes that try to guess the maximum number of logical tasks that can be handled for some amount of system RAM (which you could never possibly do correctly or consistently as a general matter for non-trivial purposes). Rather, I would argue that overcommit encourages people to allocate huge amounts of swap in an attempt to avoid the OOM killer. And I would argue that overcommit and swap make it much easier to DoS a site, all things being equal. But certainly overcommit makes it more difficult to implement resilient software. The solution is to fail allocation requests as soon as possible, and to fail them inline (not as signals). That way user processes can unwind and isolate the logical task attempting the allocation. Or if that's too difficult and inconvenient, a process can simply choose to panic. Without overcommit a process can choose the policy most suitable for the manner in which it was implemented. Among other things, that creates actionable back-pressure bound to the logic request. The point about ELF shared libraries is just plain wrong. A shared library is a file, which already has a backing store in the filesystem. You need neither overcommit nor swap space to be able to map a shared library file into virtual address space and share it across multiple processes. Lots of operating system do this without overcommitting physical RAM. And because all modern systems have a unified buffer cache, you actually get better performance by keeping those pages backed by the regular file system. If you treat them like anonymous pages, you have to copy them back out to swap space under memory pressure, whereas if they remained file-backed you don't have to write them out at all. Likewise, if a process wants to speculatively allocate a huge buffer that might possibly go unused, rather than using malloc or anonymous mmap, create and map a temporary file. For a dozen different reasons this is preferable. The only possible negative is that it doesn't kow-tow to broken software that didn't think about memory issues. But I don't want to install that software, anyhow, because if the author couldn't put in the minimal amount of effort to be able to work on systems without overcommit, they're unlikely to have put much thought into buffer overflows, integer overflows, and many other correctness and security issues. Finally, because Linux has to obey traditional Unix resources limit, as well as cgroup limits, it has to maintain an accounting of allocated pages whether or not overcommit is enabled. That fact should also be a strong hint that proper memory accounting is also a security concern, not simply an annoyance that we can wave away.

valarauca1 10y ago

Really good read. Saved in my book mark. Solid information dense document.

AceJohnny2 10y ago

> Virtual address ranges for which there is current memory mapping are said to be "unmapped" Probably supposed to read "for which there is no current memory mapping" ?

davidsong 10y ago
Yeah I spotted that too. Another posted pointed out that it's on github, so you could always submit a pull request: https://github.com/nommu/nommu.org https://github.com/nommu/nommu.org

waynecochran 10y ago

I always wondered if forking a giant processes (i.e., large memory footprint) to create a small processes caused a lot of needless memory to be allocated, but now I understand why its no big deal: For example, the fork/exec combo creates transient virtual memory usage spikes, which go away again almost immediately without ever breaking the copy on write status of most of the pages in the forked page tables. Thus if a large process forks off a smaller process, enormous physical memory demands threaten to happen (as far as overcommit is concerned), but never materialize.

paulfurtado 10y ago
Note that this also occasionally results in "failure to fork" with the default linux overcommit setting. At fork-time if there isn't enough memory to duplicate the entire process, the fork call fails, even though you never intended to use that memory. You can use vfork to get around this.
swiley 10y ago
I believe the fork() manpage (or at least the clone() manpage) actually mentions this.

sigcode 10y ago

I now have GB's of RAM and most times I have no shortage of physical memory; I have more than enough to run all my small, statically compiled programs in their own memory space. While I know the kernel is still designed for it, I have not used a dedicated "swap partition" in a decade. Hardware has come so far from the limitations of the old days; it is my dream to have the option to compile a UNIX-like kernel with virtual memory "disabled". ./build.sh -nommu

quotemstr 10y ago

The FAQ's tone of dismissive superiority with respect to overcommit is very offputting. The author suggests that only people who don't know how virtual memory works can oppose overcommit; it's curious then how the author reveals some virtual memory knowledge gaps. Specifically, the difference between address space reservation (which does not require backing storage) and commit charge (which does) seems lost on him. Yes, you need backing storage to handle the possibility that a process might actually make copies of all COW pages (that's what we mean by "commit charge" --- the number of pages the kernel has voluntarily promised to deliver at some point. The right way to reduce the pressure on commit that these mappings cause is to not have so many of them in the first place. There's no reason PIC shared libraries need to be PROT_WRITE; there's no reason big programs like web browsers need to use fork(2) instead of the VM-efficient vfork(2). (Small programs can use fork and its COW commit charge without causing real problems, even on non-sloppy (err, "strict overcommit") systems. Linux's default overcommit policy is terrible in part because it discourages people from learning about the difference between reserving address space and actually reserving memory, and the "FAQ" just confuses the issue more.

pps 10y ago
What is the best book/site to learn this all in good way?
Jasper_ 10y ago
vfork is specified by POSIX that you cannot even do so much as close an fd or call setenv between fork and exec, both of which are very common things to want to do to set up an environment before something is run. I disagree with the author too about overcommit, but vfork isn't a realistic solution.
- quotemstr 10y ago
  In your vfork, you can exec another process that does all these things in a fresh environment. Realistically, you'll want a posix_spawn interface that uses vfork internally.

derricgilling 10y ago

Under your TLB section, you mention: "The TLB is a cache for the MMU. All memory in the CPU's L1 cache must have an associated TLB entry, and invalidating a TLB entry flushes the associated cache line(s)." Many CPUs such as x86 ones don't actually flush the L1 on a TLB eviction, there would be no reason to. Eventually the line will be evicted through a snoop, capacity eviction, C6 transition etc. Of course, if a later memory access comes to re-read the line, the TLB will have to be repopulated in order to have a mapping. Also, on the "appeal of huge pages", I would put what are the negatives. Huge pages have appeal of reducing TLB pressure for stuff like big data/HPC workloads with large contiguous data sets, but it can also increase page fault latency and memory usage for more fragmented workloads that touch many pages. For example, on persist, Redis forks a process that may touch many pages. Each COW would have to find room for and allocate 2MB rather than just 4K