3 ms·
Wonder what the tradeoff on connection overhead would be? I've never been around a connection pooler that would be anywhere near this load.
by leetrout 4y ago
Wonder what the tradeoff on connection overhead would be? I've never been around a connection pooler that would be anywhere near this load.
- aeyes 4y agoNot too much if connections are persistent. A problem is that pgbouncer is 1 CPU only so you need a couple of them which makes the pooling less efficient.
- Mavvie 4y agoDoes pgbouncer tend to run out of CPU? Or just at the large scale this article is about?
- aeyes 4y agoI have a 100k qps workload and we run 10 pgBouncer instances. I ran into the CPU limit with 6. Connections to pgBouncer are persistent so it's not caused by connection churn. The amount of incoming connections is in the low hundreds per instance. I tried without SSL as well and it didn't make a measurable difference. I'm not quite sure if this is a common problem or if it's our workload. It always made me wonder if moving pooling into Postgres is really a good idea.
- geenat 4y agoSpeaking only for postgres 15+, but in my testing, app drivers have been great about facilitating "connection pooling" without a separate server. Eliminates connection overhead at the app level for many projects. Golang and ASGI Python (FastAPI, Starlette, Sanic) can do a small connection pool per worker, each worker can handle thousands of users at once. PHP in 2023 has database connection persistence using FPM + PDO (effectively, one big connection pool shared between all workers on a server). Again, eliminates connection overhead- works well. Just remember to raise your max_connections in postgres, like in this: https://gist.github.com/gnat/cfe3754c3dc817c7fb8b2225ef4db628 https://gist.github.com/gnat/cfe3754c3dc817c7fb8b2225ef4db62...