4 ms·
It sounds like the specific concerns here are actually around buffer pool management performance in and around the TLB: "Once you have a significant number of c
by cmrdporcupine 3y ago
It sounds like the specific concerns here are actually around buffer pool management performance in and around the TLB: "Once you have a significant number of connections we end up spending a *lot* of time in TLB misses, and that's inherent to the process model, because you can't share the TLB across processes. "
Many of the comments here seem to be missing this and talking about CPU-boundedness generally and thread-per-request vs process etc models, but this seems orthogonal to that, and is actually quite specific about the VM subsystem and seems like a legitimate bottleneck with the approach Postgres has to take for buffer/page mgmt with the process model it has now.
I'm no Postgres hacker (or a Linux kernel hacker), and I only did a 6 month stint doing DB internals, but it feels to me like perhaps the right answer here is that instead of Postgres getting deep down in the weeds refactoring and rewriting to a thread based model -- with all the risks in that that people have pointed out -- some assistance could be reached for by working on specific targeted patches in the Linux kernel?
The addition of e.g. userfaultfd shows that there is room for innovation and acceptance of changes in and around kernel re: page management. Some new flags for mmap, shm_open, etc. to handle some specific targeted use cases to help Postgres out?
Also wouldn't be the first time that people have done custom kernel patches or tuning parameters to crank performance out of a database.
- thesnide 3y agoExactly my thinking. If the problem is TLB evictions, why not improve these? PG isn't the only software that is hit by those. And if you carefully craft your CPU scheduling and address spaces mapping, you can reduce them by a lot. Yet rewrites are always easier and more sexy. At first.
- anarazel 3y agoThe TLB issue are more a hardware issue than a software/OS one. To my knowledge neither x86 nor arm provide a way to partially share TLB contents between processes/entities. Tlb entries can be tagged with a process context identifier, but that's an all or nothing thing. Either the entire address space is shared, or it's not. > Yet rewrites are always easier and more sexy. At first. Moving to threads would not at all be a rewrite.