4 ms·
Huge pages are an absolutely great idea. I will continue to complain about our eternal commitment to 4k pages, with all their pitfalls, which it appears we may
by aseipp 4y ago
Huge pages are an absolutely great idea. I will continue to complain about our eternal commitment to 4k pages, with all their pitfalls, which it appears we may be stuck with until the heat death of the universe. At the very minimum we could go to 16k pages, which are good for many reasons including, in particular, being able to have bigger VIPT cache sizes without needing an increase in associativity (and thus latency). Not the end of the world but a very solid win on top of the TLB wins.
But transparent hugepages continue to be a massive source of bugs, weird behaviors, and total system failures in my experience. I just got a bug report this week where a simple THP enabled system simply spun out of control with a kernel task locking the system at 100% CPU for minutes, with a 10 line reproducer via mmap(2). This is in combination with qemu/libvirt in a virtual machine, and it's possible the virtualization stack is just exposing bugs, but, like. This is very well tested stuff! I'm not sure "Google enabled it fleet wide, so it can be done" is very re-assuring to me when most of us don't have fleetops/infra/kernel teams capable of dealing with this stuff. The person who reported this bug said they started seeing odd behavior a month ago, before boiling it down; it wasn't readily apparent at all. Is this just a massive footgun for our distro users? I dunno. Something that works in the p95 and then collapses horrifically in the p99 cases like this doesn't feel great. I try not to be superstitious about things like this but, yeah. It's weird.
Anyway. This reminds me I have to submit some patches to disable jemalloc in a few aarch64 packages so I can use them on Asahi Linux. 4k pages will haunt us until the end of time.
- zajio1am 4y ago> Huge pages are an absolutely great idea. I will continue to complain about our eternal commitment to 4k pages, with all their pitfalls, which it appears we may be stuck with until the heat death of the universe. I think it is useful to distinguish just larger regular pages (i.e. 16k or 64k pages on arm64) and extraordinary huge pages (2M on systems where regular pages are 4k). In the first case, the system uses these larger pages uniformly, as there is no smaller page.
- aseipp 4y agoYou're completely right. But I need to a place to launder^W air my related complaints, too. :)
- Sesse__ 4y agoIf you think 2M pages are extraordinarily huge, what would you call 1G pages? :-)
- zajio1am 4y agoI say they are extraordinary, because they are larger than regular page size as reported by sysconf(_SC_PAGESIZE). Also, i meant "extraordinary (huge pages)", not "(extraordinary huge) pages".
- NelsonMinar 4y agoThe part in the article about ARM64 Linux giving up on 64kb pages was particularly damning. Although as it notes, Apple is using something other than 4gb pages at least.
- juped 4y agoHuge pages are not a great idea, increasing the page size from 4k is.