3 ms·

That's one part where having multiple completely independent processes is a disadvantage to just having threads. I never benchmarked this, but I always heard th

That's one part where having multiple completely independent processes is a disadvantage to just having threads. I never benchmarked this, but I always heard that Postgres processes are expensive and you should always use a connection pool. But this is probably more noticeable on the DB server side, where each process uses memory. I never used it myself, but PG bouncer might be useful to you in this case. The number of requests still feels slow, and it isn't clear to me from the blog post whether that is DB limited or CPU limited on the application servers. Even with writes on each access Postgres should still be bored at that kind of load.

ericholscher 5y ago
Yea, our IO and CPU of our Postgres instance is around 20% currently. We haven't hit too many issues with Postgres, so haven't delved too deeply into the performance. I think people seeing performance posts are used to people pushing huge numbers. Our intent here was to show that using standard off the shelf tools with a tiny bit of architecting, you can hit huge performance numbers at a reasonable cost. We have lots of various ways to shave performance, but we're pretty sure we could scale 2-3x with the same stack without a ton of work.
davidfischer 5y ago
Postgres is mostly bored (CPU sub 20% utilized) with 60 write requests per second which is normal peak traffic. Even traffic spikes to 100-120 don't usually change that much. As our ad network grows though, everything has to grow pretty linearly (writes, reads, requests, etc.) and I hope our approach will continue to work well even with double or triple these numbers.
- philipkimmey 5y ago
  Fairly common practice for Postgres setups is to sit pgbouncer (https://www.pgbouncer.org/ https://www.pgbouncer.org/) between your app process and your Postgres instance. Connection setup/tear-down is cheap in MySQL but expensive in Postgres.

ericholscher 5y ago

Yea, our IO and CPU of our Postgres instance is around 20% currently. We haven't hit too many issues with Postgres, so haven't delved too deeply into the performance. I think people seeing performance posts are used to people pushing huge numbers. Our intent here was to show that using standard off the shelf tools with a tiny bit of architecting, you can hit huge performance numbers at a reasonable cost. We have lots of various ways to shave performance, but we're pretty sure we could scale 2-3x with the same stack without a ton of work.

davidfischer 5y ago

Postgres is mostly bored (CPU sub 20% utilized) with 60 write requests per second which is normal peak traffic. Even traffic spikes to 100-120 don't usually change that much. As our ad network grows though, everything has to grow pretty linearly (writes, reads, requests, etc.) and I hope our approach will continue to work well even with double or triple these numbers.

philipkimmey 5y ago
Fairly common practice for Postgres setups is to sit pgbouncer (https://www.pgbouncer.org/ https://www.pgbouncer.org/) between your app process and your Postgres instance. Connection setup/tear-down is cheap in MySQL but expensive in Postgres.