4 ms·
Postgres is great, but the level of work required to scale it on large workloads is really quite overwhelming. You really need expert-level knowledge to scale
by kelseydh 2y ago
Postgres is great, but the level of work required to scale it on large workloads is really quite overwhelming. You really need expert-level knowledge to scale it. Just ask anybody who has had to shard their database.
- genewitch 2y agoI ran a 4000qps postgres database set to use no more than 640KB of RAM (its in the configs). The DBA was having fun with my silly idea until Slow-Query takes 10 seconds and we flipped back over to the production systems with much more allowed memory for postgres. My purpose was to show that Linux itself is pretty fast at caching and paging, even if Postgres was hamstrung. The db actually ran fine except for the slow queries, and we probably could have cleaned them up, too. But I proved my point. DB was ~300GB, metal had 512GB RAM. This was in 2012.
- dano 2y agoYep. I converted a business from to Postgres in 2004 and never looked back. Stable, reliable, no surprises. Postgres is the answer until proven otherwise.
- sgarland 2y agoNo offense, but 4000 QPS is not large. I ran a 100K+ QPS MySQL instance with 384 GiB of RAM. This is all subjective, of course, but I wouldn’t call 100K QPS large, either. It’s getting on the high end of what you might reasonably want to handle without horizontal scaling, but it’s still not “large.”
- zeeg 2y agoDefine large workload. Sentry runs large workloads and Postgres isn't a bottleneck. We have also never employed a DBA. Most users never need to shard their database, and at most can just partition datasets by tables for _extremely_ high volume workloads. You just have to consider architecture and optimize things that are slow, just like any other software. Nothing is free at the end of the day, and nothing else gives you the flexibility that Postgres does, particularly these days with its growing high value extensions.