5 ms·

I only looked curiously at the relevant Linux code. But my understanding is that compound pages don't show up in the page table of a process. Yes, there's an as

I only looked curiously at the relevant Linux code. But my understanding is that compound pages don't show up in the page table of a process. Yes, there's an associated data structure with each physical page, but that's separate from per process tracking. The problem with the page table and postgres is that they're separately maintained for each process, even if all processes share the same shared memory area. With 1GB or 2MB pages that's not too bad. But with 4k pages you can easily end up with dozens to hundreds of MB for each connection. Wasted memory + wasted cycles (tlb misses).

gavinray 4y ago
There's a new Linux feature called Shared Page Table Entries that allows processes to share these It was requested by the Oracle DB team https://lwn.net/Articles/919143/ https://lwn.net/Articles/919143/ I recognize your username so you'd know better than most whether Postgres would be able to use this or not though
- anarazel 4y ago
  I did vaguely follow the effort - last time I checked nothing seems to have been merged yet. Did it get merged since? I'd expect it to benefit postgres substantially.
  - gavinray 4y ago
    It doesn't appear to be merged yet in any of the 6.2 RC's from what I can see =/ I am eagerly awaiting its release as a DB enthusiast and lifelong Postgres user
gnfargbl 4y ago
Oh, so what I had assumed was some kind of page table fragmentation was in fact due to the number of postgres clients increasing, meaning that the number of page table entries increased due to duplication between processes? That would make sense, in retrospect: the OOMs almost always occurred at the point of highest client load.
menaerus 4y ago
Compound page is just an implementation detail how page-table data structure represents the pages larger than 4K. And this is reasonable to me because in order to create a mapping for 2MB page, kernel needs to group 512 consecutive 4K pages, and per my understanding this is exactly what the kernel will do. For a 1GB page it will have to group 512 page-tables, each containing 512 4K entries. There is no separate page-table data structure that the kernel will maintain for 2MB or 1GB pages. There's only one. However, you're right that multiple processes maintaining separate page-tables where each is containing mappings to the same physical memory regions can be wasteful. Still, due to how huge pages are represented in the page-table data structure, I don't see what difference would it make to switch to 2MB or 1GB page, unless you're running a recentish kernel (~3 years) that has been specifically optimized for exact this case, e.g. to reduce the memory footprint of huge-page representation in page-tables. Specifically, a single 2MB huge page is now represented by 8 page entries (512 bytes) instead of 512 page entries (2MB). And there was another patch proposing to cut 8 page entries to only 2 page entries. In any case, that's a dramatic cut of memory footprint when compared to how it used be and which is why I believe that this is the actual reason why OP sees the difference.

gavinray 4y ago

There's a new Linux feature called Shared Page Table Entries that allows processes to share these It was requested by the Oracle DB team https://lwn.net/Articles/919143/ https://lwn.net/Articles/919143/ I recognize your username so you'd know better than most whether Postgres would be able to use this or not though

anarazel 4y ago
I did vaguely follow the effort - last time I checked nothing seems to have been merged yet. Did it get merged since? I'd expect it to benefit postgres substantially.
- gavinray 4y ago
  It doesn't appear to be merged yet in any of the 6.2 RC's from what I can see =/ I am eagerly awaiting its release as a DB enthusiast and lifelong Postgres user

gnfargbl 4y ago

Oh, so what I had assumed was some kind of page table fragmentation was in fact due to the number of postgres clients increasing, meaning that the number of page table entries increased due to duplication between processes? That would make sense, in retrospect: the OOMs almost always occurred at the point of highest client load.

menaerus 4y ago

Compound page is just an implementation detail how page-table data structure represents the pages larger than 4K. And this is reasonable to me because in order to create a mapping for 2MB page, kernel needs to group 512 consecutive 4K pages, and per my understanding this is exactly what the kernel will do. For a 1GB page it will have to group 512 page-tables, each containing 512 4K entries. There is no separate page-table data structure that the kernel will maintain for 2MB or 1GB pages. There's only one. However, you're right that multiple processes maintaining separate page-tables where each is containing mappings to the same physical memory regions can be wasteful. Still, due to how huge pages are represented in the page-table data structure, I don't see what difference would it make to switch to 2MB or 1GB page, unless you're running a recentish kernel (~3 years) that has been specifically optimized for exact this case, e.g. to reduce the memory footprint of huge-page representation in page-tables. Specifically, a single 2MB huge page is now represented by 8 page entries (512 bytes) instead of 512 page entries (2MB). And there was another patch proposing to cut 8 page entries to only 2 page entries. In any case, that's a dramatic cut of memory footprint when compared to how it used be and which is why I believe that this is the actual reason why OP sees the difference.