5 ms·
it'll call mmap (probably), which will then find 1GB(+ overhead) of contiguous address space mmap will go and create the page table mappings (entries) for that
by blibble 7y ago
it'll call mmap (probably), which will then find 1GB(+ overhead) of contiguous address space
mmap will go and create the page table mappings (entries) for that address space (~262144 of them with 4k pages)
unless someone's been messing with overcommit: the physical memory allocation is normally lazy
this is done by setting the "faulted" bit on the allocated page table entries
when the application first reads/writes from a page not yet allocated: a fault will occur, the kernel will step in, (hopefully) find an unused physical 4kb of memory, update the page table entry to point to it, unmark the faulted bit, then return to the application (which will be blissfully unaware that anything happened)
- cesarb 7y ago> mmap will go and create the page table mappings (entries) for that address space (~262144 of them with 4k pages) I haven't looked at the mmap implementation in detail, but I believe it doesn't even do that: it just creates a new VMA (struct vm_area_struct in the kernel) for that address space range, and the page table is updated lazily.
- ajross 7y ago> mmap will go and create the page table mappings (entries) for that address space (~262144 of them with 4k pages) It does not, actually[1]. Page tables are a hardware datastructure with significant overhead. All the kernel does at mmap() time is create the a virtualized record of the mapping. It's not until the process takes a page fault to a mapped address that the hardware page tables are allocated and filled out. In practice this is a win because (especially with use cases like malloc) most processes tend to map large chunks of which they'll only use a subset in the short term. [1] By default. There is a MAP_POPULATE flag you can pass that will allocate the page tables and allocate/prefault all the mapped pages before the system call returns.
- dividuum 7y ago> it'll call mmap (probably), which will then find 1GB(+ overhead) of contiguous address space It does. Last time I dug into that I think there was a 128KB threshold. Smaller allocations would use sbrk while larger once would use mmap.
- saagarjha 7y agoThe limit is now dynamic on glibc: https://sourceware.org/glibc/wiki/MallocInternals#Malloc_Algorithm https://sourceware.org/glibc/wiki/MallocInternals#Malloc_Alg..., http://man7.org/linux/man-pages/man3/mallopt.3.html http://man7.org/linux/man-pages/man3/mallopt.3.html
- blattimwind 7y agosbrk also just modifies a mapping. All userspace memory management happens through mappings (to files, anonymous or perhaps I/O space).
- rrss 7y agoDoes the kernel allocate the physical page when it is read, or only when it is written to? My mental model was that the new VA pages are first mapped to the zero page as copy on write.
- deleted 7y ago[deleted]
- tedunangst 7y agoI don't know specifically either, but cow mapping a zero page on read seems like an optimization that can backfire. If you write to it later, you need to fix up the page table a second time.