5 ms·

The FAQ is hosted my NOMMULinux project (nomen omen a Linux distribution that is not using virtual memory, it was discussed on HN earlier today). I found it to

by mbel 10y ago

The FAQ is hosted my NOMMULinux project (nomen omen a Linux distribution that is not using virtual memory, it was discussed on HN earlier today). I found it to be a very nice summary of how OS manages memory, that is quite interesting even without context. That's why I posted it. It is a little bit light on the details, but in my opinion it is able to show memory management as a complete process, which many other materials fail to do (at least the ones that I've seen).

wahern 10y ago
Some of us disagree violently with that FAQ's discussion of the OOM killer and strict overcommit. The arguments aren't even coherent. For example, I disable overcommit and run my machines without any swap whatsoever. (Though this doesn't resolve the problem entirely in Linux.) The way to deal with the reality of finite memory isn't to pretend like it's infinite and then randomly kill processes when you hit a wall; or to use magic constants in user processes that try to guess the maximum number of logical tasks that can be handled for some amount of system RAM (which you could never possibly do correctly or consistently as a general matter for non-trivial purposes). Rather, I would argue that overcommit encourages people to allocate huge amounts of swap in an attempt to avoid the OOM killer. And I would argue that overcommit and swap make it much easier to DoS a site, all things being equal. But certainly overcommit makes it more difficult to implement resilient software. The solution is to fail allocation requests as soon as possible, and to fail them inline (not as signals). That way user processes can unwind and isolate the logical task attempting the allocation. Or if that's too difficult and inconvenient, a process can simply choose to panic. Without overcommit a process can choose the policy most suitable for the manner in which it was implemented. Among other things, that creates actionable back-pressure bound to the logic request. The point about ELF shared libraries is just plain wrong. A shared library is a file, which already has a backing store in the filesystem. You need neither overcommit nor swap space to be able to map a shared library file into virtual address space and share it across multiple processes. Lots of operating system do this without overcommitting physical RAM. And because all modern systems have a unified buffer cache, you actually get better performance by keeping those pages backed by the regular file system. If you treat them like anonymous pages, you have to copy them back out to swap space under memory pressure, whereas if they remained file-backed you don't have to write them out at all. Likewise, if a process wants to speculatively allocate a huge buffer that might possibly go unused, rather than using malloc or anonymous mmap, create and map a temporary file. For a dozen different reasons this is preferable. The only possible negative is that it doesn't kow-tow to broken software that didn't think about memory issues. But I don't want to install that software, anyhow, because if the author couldn't put in the minimal amount of effort to be able to work on systems without overcommit, they're unlikely to have put much thought into buffer overflows, integer overflows, and many other correctness and security issues. Finally, because Linux has to obey traditional Unix resources limit, as well as cgroup limits, it has to maintain an accounting of allocated pages whether or not overcommit is enabled. That fact should also be a strong hint that proper memory accounting is also a security concern, not simply an annoyance that we can wave away.
- ndesaulniers 10y ago
  > Some of us disagree violently with that FAQ's discussion of the OOM killer and strict overcommit. Agreed. Android for example has made major architectural changes due to not having swap. I wasn't around during the early days, and by no means an expert, but it looks to me like the following decisions were made: 1. No swapping, flash at the time was very slow, limited in capacity, and still has limited lifetime. Introduce OOM killer. [0] 2. SysV semaphores and shared memory can leak when killing a process. [1][2] It looks like these have since been added to bionic (Android's libc). [3] 3. Introduce The Binder kernel driver for IPC instead. The Binder handles reference counting, and can do neat things like death notifications. [4] 4. Introduce ashmem for shared memory. [5] [0] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/drivers/staging/android/lowmemorykiller.c?id=refs/tags/v4.8.1 https://git.kernel.org/cgit/linux/kernel/git/stable/linux-st... (extreamly readable at 212 LOC) [1] See `man 7 sem_overview`'s and `man 7 shm_overview`'s Persistence sections. [2] https://android.googlesource.com/platform/ndk/+/jb-release/docs/system/libc/SYSV-IPC.html https://android.googlesource.com/platform/ndk/+/jb-release/d... [3] https://android.googlesource.com/platform/bionic/+/master/libc/include/semaphore.h https://android.googlesource.com/platform/bionic/+/master/li..., https://android.googlesource.com/platform/bionic/+/master/libc/include/sys/sem.h https://android.googlesource.com/platform/bionic/+/master/li... [3] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/drivers/android/binder.c?id=refs/tags/v4.8.1 https://git.kernel.org/cgit/linux/kernel/git/stable/linux-st... [4] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/drivers/staging/android/ashmem.c?id=refs/tags/v4.8.1 https://git.kernel.org/cgit/linux/kernel/git/stable/linux-st...
- gpderetta 10y ago
  I dislike overcommit as well (although I have made use of it by relying on lazy allocation), but I think it is very hard to implement unix fork semantics without it: on a fork the kernel would have to reserve memory for a copy of the whole process, even though the next thing the child is most likely to do is throw it away by calling exec. Unless all software were to be changed to use vfork or posix_spawn, I think we are stuck with it. I would love to be able to disable overcommit on a per-process basis though.

by mbel 10y ago

wahern 10y ago

Some of us disagree violently with that FAQ's discussion of the OOM killer and strict overcommit. The arguments aren't even coherent. For example, I disable overcommit and run my machines without any swap whatsoever. (Though this doesn't resolve the problem entirely in Linux.) The way to deal with the reality of finite memory isn't to pretend like it's infinite and then randomly kill processes when you hit a wall; or to use magic constants in user processes that try to guess the maximum number of logical tasks that can be handled for some amount of system RAM (which you could never possibly do correctly or consistently as a general matter for non-trivial purposes). Rather, I would argue that overcommit encourages people to allocate huge amounts of swap in an attempt to avoid the OOM killer. And I would argue that overcommit and swap make it much easier to DoS a site, all things being equal. But certainly overcommit makes it more difficult to implement resilient software. The solution is to fail allocation requests as soon as possible, and to fail them inline (not as signals). That way user processes can unwind and isolate the logical task attempting the allocation. Or if that's too difficult and inconvenient, a process can simply choose to panic. Without overcommit a process can choose the policy most suitable for the manner in which it was implemented. Among other things, that creates actionable back-pressure bound to the logic request. The point about ELF shared libraries is just plain wrong. A shared library is a file, which already has a backing store in the filesystem. You need neither overcommit nor swap space to be able to map a shared library file into virtual address space and share it across multiple processes. Lots of operating system do this without overcommitting physical RAM. And because all modern systems have a unified buffer cache, you actually get better performance by keeping those pages backed by the regular file system. If you treat them like anonymous pages, you have to copy them back out to swap space under memory pressure, whereas if they remained file-backed you don't have to write them out at all. Likewise, if a process wants to speculatively allocate a huge buffer that might possibly go unused, rather than using malloc or anonymous mmap, create and map a temporary file. For a dozen different reasons this is preferable. The only possible negative is that it doesn't kow-tow to broken software that didn't think about memory issues. But I don't want to install that software, anyhow, because if the author couldn't put in the minimal amount of effort to be able to work on systems without overcommit, they're unlikely to have put much thought into buffer overflows, integer overflows, and many other correctness and security issues. Finally, because Linux has to obey traditional Unix resources limit, as well as cgroup limits, it has to maintain an accounting of allocated pages whether or not overcommit is enabled. That fact should also be a strong hint that proper memory accounting is also a security concern, not simply an annoyance that we can wave away.

ndesaulniers 10y ago
> Some of us disagree violently with that FAQ's discussion of the OOM killer and strict overcommit. Agreed. Android for example has made major architectural changes due to not having swap. I wasn't around during the early days, and by no means an expert, but it looks to me like the following decisions were made: 1. No swapping, flash at the time was very slow, limited in capacity, and still has limited lifetime. Introduce OOM killer. [0] 2. SysV semaphores and shared memory can leak when killing a process. [1][2] It looks like these have since been added to bionic (Android's libc). [3] 3. Introduce The Binder kernel driver for IPC instead. The Binder handles reference counting, and can do neat things like death notifications. [4] 4. Introduce ashmem for shared memory. [5] [0] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/drivers/staging/android/lowmemorykiller.c?id=refs/tags/v4.8.1 https://git.kernel.org/cgit/linux/kernel/git/stable/linux-st... (extreamly readable at 212 LOC) [1] See `man 7 sem_overview`'s and `man 7 shm_overview`'s Persistence sections. [2] https://android.googlesource.com/platform/ndk/+/jb-release/docs/system/libc/SYSV-IPC.html https://android.googlesource.com/platform/ndk/+/jb-release/d... [3] https://android.googlesource.com/platform/bionic/+/master/libc/include/semaphore.h https://android.googlesource.com/platform/bionic/+/master/li..., https://android.googlesource.com/platform/bionic/+/master/libc/include/sys/sem.h https://android.googlesource.com/platform/bionic/+/master/li... [3] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/drivers/android/binder.c?id=refs/tags/v4.8.1 https://git.kernel.org/cgit/linux/kernel/git/stable/linux-st... [4] https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/drivers/staging/android/ashmem.c?id=refs/tags/v4.8.1 https://git.kernel.org/cgit/linux/kernel/git/stable/linux-st...
gpderetta 10y ago
I dislike overcommit as well (although I have made use of it by relying on lazy allocation), but I think it is very hard to implement unix fork semantics without it: on a fork the kernel would have to reserve memory for a copy of the whole process, even though the next thing the child is most likely to do is throw it away by calling exec. Unless all software were to be changed to use vfork or posix_spawn, I think we are stuck with it. I would love to be able to disable overcommit on a per-process basis though.