3 ms·

The paging system will only page in what's being used right now though and paging out has zero cost. Old data will naturally be paged out. To put it directly th

The paging system will only page in what's being used right now though and paging out has zero cost. Old data will naturally be paged out. To put it directly the answer is each mmap file will need 1 page of physical memory (the area currently being read/written). There may be old pages left around since there's no reason for the OS to page anything out unless some other application asked for the memory. But if they do mmap will go to 1 page just fine and there's zero cost to paging out. I feel mmap gets a bad reputation when people look at memory usage tools that look at total virtual memory allocated. I can mmap a 100GB of files, use 0 physical memory and a lot of memory usage tools will report 100GB of memory usage of a certain type (virtual memory allocated). You then get articles about application X using GB of memory. Anyone trying to correct this is ignored. Google Chrome is somewhat unfairly hit by this. All those articles along the lines of "Why is Google using 4GB with no tabs after i viewed some large PDFs". The answer is that it reserved 4GB of 'addresses' that it has mapped to files. If another application wants to use that memory there's zero cost to paging out those files from memory. The OS is designed to do this and it's what mmap is for.

labawi 6y ago
> paging out has zero cost Paging out, as in removing a mapping, can be surprisingly costly, because you need to invalidate any cached TLB entries, possibly even in other CPUs. > each mmap file will need 1 page of physical memory Technically, a lower limit would be about 2 or so usable pages, because you can't use more than that simultaneously. However unmaps are expensive, so the system won't be too eager to page out. Also, for pages to be accessible, they need to be specified in the page table (actually tree, of virtual -> physical mappings). A random address may require about 1-3 pages for page table aside from the 1 page of actual data (but won't need more page tables for the next MB). > application X using GB of memory I think there is a difference between reserved, allocated and file-backed mmapped memory. Address space, file-backed mmapped memory is easily paged-out, not sure what different types of reserved addresses/memory are, but chrome probably doesn't have lots of mmapped memory that can be paged out. If it's modified, then it must be swapped, otherwise it's just reserved and possibly mapped, but never used memory.
- AnotherGoodName 6y ago
  I'd argue the costs with paging out are already accounted for by the other process paging in though. The other process that paged in and led to the need to page out had already led to the need to change the page table and flush cache.
  - labawi 6y ago
    Paging in free memory (adding a mapping) is cheap (no need to flush). Removing a mapping is expensive (need to flush). Also, processes have their own (mostly) independent page tables. I don't think it would be reasonable accounting, when paging-in is cheap, but only if there is no need to page out (available free memory). Especially when trying to argue that paging out is zero-cost.
owl57 6y ago
Are page tables garbage-collected in Linux? That seems like a potential non-trivial leak source: up to 200MB for your hypothetical 100GB file.

labawi 6y ago

> paging out has zero cost Paging out, as in removing a mapping, can be surprisingly costly, because you need to invalidate any cached TLB entries, possibly even in other CPUs. > each mmap file will need 1 page of physical memory Technically, a lower limit would be about 2 or so usable pages, because you can't use more than that simultaneously. However unmaps are expensive, so the system won't be too eager to page out. Also, for pages to be accessible, they need to be specified in the page table (actually tree, of virtual -> physical mappings). A random address may require about 1-3 pages for page table aside from the 1 page of actual data (but won't need more page tables for the next MB). > application X using GB of memory I think there is a difference between reserved, allocated and file-backed mmapped memory. Address space, file-backed mmapped memory is easily paged-out, not sure what different types of reserved addresses/memory are, but chrome probably doesn't have lots of mmapped memory that can be paged out. If it's modified, then it must be swapped, otherwise it's just reserved and possibly mapped, but never used memory.

AnotherGoodName 6y ago
I'd argue the costs with paging out are already accounted for by the other process paging in though. The other process that paged in and led to the need to page out had already led to the need to change the page table and flush cache.
- labawi 6y ago
  Paging in free memory (adding a mapping) is cheap (no need to flush). Removing a mapping is expensive (need to flush). Also, processes have their own (mostly) independent page tables. I don't think it would be reasonable accounting, when paging-in is cheap, but only if there is no need to page out (available free memory). Especially when trying to argue that paging out is zero-cost.

owl57 6y ago

Are page tables garbage-collected in Linux? That seems like a potential non-trivial leak source: up to 200MB for your hypothetical 100GB file.