4 ms·
Yes, Linux (and Windows for that matter) does everything DragonflyBSD just removed.
by jdub 10y ago
Yes, Linux (and Windows for that matter) does everything DragonflyBSD just removed.
- bch 10y agoWhat does this mean? They do the old-dragonfly way, or the new-dragonfly way ?
- jdub 10y agoThe old way.
- cpeterso 10y agoI wonder how long until someone submits a Linux patch to remove idle page-zeroing. :)
- koenigdavidmj 10y agoJust be careful not to submit one patch per file, times 1500 files.
- bboreham 10y agoWindows does more of it, in the sense the heap manager likes to give freed blocks back to the OS, whereas UNIX programs traditionally never gave memory back until program exit. I had a massive production outage a few years back involving this behaviour on Windows; symptom was 100% CPU usage in-kernel on exactly two threads and no other processes doing anything much. Still recall that as one of the most interesting bugs to track down.
- viraptor 10y agoHow did you track it down? MS's performance debugging toolkit? (Can't remember the actual name) Or something else?
- bboreham 10y agoHours and hours of narrowing down the symptom from a program that would recreate it, plus disassembling the Windows heap manager and tracing through it in the debugger.
- _wmd 10y agoThis hasn't really been true on Linux for a long time, glibc happily munmaps unused arenas when they empty and some dynamic threshold is reached
- nikanj 10y agoHave you written about that outage anywhere? Sounds really interesting!
- bboreham 10y agoYeah, I should really do that.
- caf 10y agoNo, Linux does not pre-zero pages in the background, at least on x86 and x86-64. Not-present faults on anonymous VMAs are handled by do_anonymous_page() in mm/memory.c: https://lxr.missinglinkelectronics.com/#linux+v4.6/mm/memory.c#L2729 https://lxr.missinglinkelectronics.com/#linux+v4.6/mm/memory... As you can see, a write fault on a not-present page in an anonymous VMA calls alloc_zeroed_user_highpage_movable(), which calls __alloc_zeroed_user_highpage(). The x86 architecture provides an architecture-override for this as a macro which calls alloc_page_vma() with the GFP_ZERO flag added. This calls down through alloc_pages_vma() and __alloc_pages_nodemask() which eventually gets a page through get_page_from_freelist() (potentially via __alloc_pages_slowpath()). get_page_from_freelist() uses prep_new_page() to act on the GFP_ZERO flag ( https://lxr.missinglinkelectronics.com/#linux+v4.6/mm/page_alloc.c#L1519 https://lxr.missinglinkelectronics.com/#linux+v4.6/mm/page_a... ). Unless the (non-default) PAGE_POISONING_ZERO configuration option is set, this clears the page directly with clear_highpage(). The PAGE_POISONING_ZERO option zeroes the page at the time it is freed (so not in the background) - it's intended as a sanitization/hardening option, not a performance one. A read fault on a not-present page in an anonymous VMA maps in the singleton "Zero PTE" which is a read-only PTE pointing at a page of zeroes. A subsequent write fault on this page will end up in wp_page_copy(), which recognises the Zero PTE and calls into alloc_zeroed_user_highpage_movable() instead of copying the page, as before. It would be possible for another architecture to provide a version of __alloc_zeroed_user_highpage() which did use background-zeroing, if it was more efficient on that architecture - I haven't checked to see if any do, though.
- jdub 10y agoNow that is a properly researched and presented well-actually! Thanks, this is cool. :-)