3 ms·
Please be aware that the article describes a problem with a specific implementation of THP. Other operating systems implement it differently and don't suffer fr
by markjdb 9y ago
Please be aware that the article describes a problem with a specific implementation of THP. Other operating systems implement it differently and don't suffer from the same caveats (though any implementation will of course have its own disadvantages, since THP support requires making various tradeoffs and policy decisions). FreeBSD's implementation (based on [1]) is more conservative and works by opportunistically reserving physically contiguous ranges of memory in a way that allows THP promotion if the application (or kernel) actually makes use of all the pages backed by the large mapping. It's tied in to the page allocator in a way that avoids the "leaks" described in the article, and doesn't make use of expensive scans. Moreover, the reservation system enables other optimizations in the memory management subsystem.
[1] https://www.cs.rice.edu/~druschel/publications/superpages.pdf https://www.cs.rice.edu/~druschel/publications/superpages.pd...
- drostie 9y agoHey, thanks for being a FreeBSD dev. Every time I've been on a FreeBSD system my reaction has been "this is kinda weird, but really nice." (Especially, compiling from ports.) The fact that y'all are connected with academic communities who can solve these problems in more principled ways is really wonderful, as are the BSD/Solaris attitudes of "hey let's wall off things that don't need to interact" and "the kernel is the most important part, but it's not the only thing."
- loeg 9y agoIt's worth pointing out that the FreeBSD implementation (on AMD64) only promotes 4kB pages to 2MB pages and doesn't transparently promote to 1GB pages. Given alc@ was an author on the paper (and the paper's FreeBSD 4.x implementation supported multiple superpage sizes), I'm not really sure why FreeBSD's pmap doesn't have support for 1GB page promotions.
- kev009 9y agoF5/LineRate did it but it got NACKed in a fairly underhanded and unfortunate way on the mailing lists :/ https://github.com/Seb-LineRate/freebsd/commits/seb/stable-10/1-gig-pages https://github.com/Seb-LineRate/freebsd/commits/seb/stable-1...
- markjdb 9y agoThat patch set does not implement transparent creation of 1GB mappings. It also contains dubious things like this, which make me think the branch was a WIP: https://github.com/Seb-LineRate/freebsd/commit/66a8d3474d41030d4da5bfa2042aa573ff1b281f https://github.com/Seb-LineRate/freebsd/commit/66a8d3474d410... The only mailing list thread I see regarding this is here, and it doesn't seem particularly underhanded to me: https://lists.freebsd.org/pipermail/freebsd-hackers/2014-November/046541.html https://lists.freebsd.org/pipermail/freebsd-hackers/2014-Nov...
- kev009 9y agoAll the technical critique seems fair but it seemed like they (both as an individual and as a company) were a first time contributor and no outreach was really done to pull them in further. I guess LineRate imploded within F5 so there could have been structural problems inside there prevented them from doing a fully baked contribution anyway.
- loeg 9y agoCan you clarify what you mean by "underhanded and unfortunate way?" The thread I can see on freebsd-hackers@ had a little feedback, including some very valid critiques (the code was developed against 9.x, then ported to 10.x without testing, at a time when 11 was CURRENT) and the original author just didn't follow up at all: https://lists.freebsd.org/pipermail/freebsd-hackers/2014-November/046449.html https://lists.freebsd.org/pipermail/freebsd-hackers/2014-Nov... But maybe I'm missing something? See also: https://lists.freebsd.org/pipermail/freebsd-hackers/2013-September/043385.html https://lists.freebsd.org/pipermail/freebsd-hackers/2013-Sep...