3 ms·
MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (whic
by mastax 1y ago
MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).
- inetknght 1y ago> MAP_HUGETLB can't be used for mmaping files on disk False. I've successfully used it to memory-map networked files.
- inetknght 1y agoMy bad, don't use `MAP_HUGETLB`, just use `MAP_HUGE_1GB`. See a quick example I whipped up here: https://github.com/inetknght/mmap-hugetlb https://github.com/inetknght/mmap-hugetlb
- jared_hulbert 1y agoAdding MAP_HUGE_1GB and not MAP_HUGETLB does compile and run for me. Not convinced that its' actually doing anything. Performance is the same.
- inetknght 1y agoWell now that it works, feel free to start poking around at it for a follow-up blog post :)
- bawolff 1y agoThe mmap man page kind of implies that would be a no-op, but i haven't tested myself.
- loloquwowndueo 1y agoShare your code?
- inetknght 1y agoMy bad, don't use `MAP_HUGETLB`, just use `MAP_HUGE_1GB`. See a quick example I whipped up here: https://github.com/inetknght/mmap-hugetlb https://github.com/inetknght/mmap-hugetlb
- inetknght 1y agoI don't work there any more (it was a decade ago) and I'm pretty busy right now with a new job coming up (offered today). Do you have kernel documentation that says that hugetlb doesn't work for files? I don't see that stated anywhere.
- minitech 1y agoThat doesn’t sound like the intended meaning of “on disk”.
- inetknght 1y agoKernel doesn't really care about "on disk", it cares about "on filesystem". The "on disk" distinction is a simplification.
- squirrellous 1y agoThis is quite interesting since I, too, was under the impression that mmap cannot be used on disk-backed files with huge pages. I tried and failed to find any official kernel documentation around this, but I clearly remember trying to do this at work (on a regular ECS machine with Ubuntu) and getting errors. Based on this SO discussion [1], it is possibly a limitation with popular filesystems like ext4? If anyone knows more about this, I'd love to know what exactly are the requirements for using hugepages this way. [1] https://stackoverflow.com/questions/44060678/huge-pages-for-memory-mapped-files-on-linux https://stackoverflow.com/questions/44060678/huge-pages-for-...
- inetknght 1y agoMy bad, don't use `MAP_HUGETLB`, just use `MAP_HUGE_1GB`. See a quick example I whipped up here: https://github.com/inetknght/mmap-hugetlb https://github.com/inetknght/mmap-hugetlb
- squirrellous 1y agoCool! Thanks for the example. The aforementioned work thing requires MAP_SHARED as well which IIRC is the reason it would fail when used together with files and huge pages, but private mappings work as you show.
- bawolff 1y agoTrying to google this i found https://lwn.net/Articles/718102/ https://lwn.net/Articles/718102/ which suggests that there was discussion about it back in 2017. But i can't find anything else about it except a patchset that i guess wasnt merged (?). So maybe it was just a proposal that never made it in. Honestly i never knew any of this i thought huge pages just worked for all of mmap.
- mananaysiempre 1y agoIt looks like there is in theory support for that[1]? But the patches for ext4[2] did not go through. [1] https://lwn.net/Articles/686690/ https://lwn.net/Articles/686690/ [2] https://lwn.net/Articles/718102/ https://lwn.net/Articles/718102/