5 ms·

Hi - as mentioned in the article all connections went through pgbouncer (limited to 20) and I was careful to ensure that all configurations saturated the CPU so

Hi - as mentioned in the article all connections went through pgbouncer (limited to 20) and I was careful to ensure that all configurations saturated the CPU so I'm pretty confident they were not waiting on connections to open. Opening a connection from pgbouncer over a unix socket is very fast indeed - my guess is perhaps a couple of orders of magnitude faster than without it. 20 connections divided by 4 CPUs is a lot, and pretty much all CPU time was still spent in Python. Sidenote here: one thing I found but didn't mention (the reason I put in the pooling, both in Python and pgbouncer) is that otherwise, under load, the async implementions would flood postgres with open connections and everything would just break down. I think making a database query and responding with JSON is a very realistic workload. I've coded that up many times. Changing it to make requests to other things (mimicking a microservice architecture) is also interesting and if you did that I'd be interested to read your write up.

supermatt 6y ago
Aren't you still capping the throughput by the query rate of your connection pool though? By limiting that, you are limiting the application as a whole - i.e. your benchmark is bound by the speed of your database, and has (almost) nothing to do with the performance of a specific python implementation.
- arghwhat 6y ago
  Only if there are spare resources left to saturate the connection pools, which didn't seem to be the case. If the system as a whole is well saturated, and the python processes dominate the system load with a DB load proportional to the requests served, then I don't think we would hit any external bottlenecks. The benchmarks performed are not that great (e.g., virtualized, same machine for all components, etc.), but I don't think the errors are enough to throw off the result. Note, of course, that such results are not universal, and some loads might perform better async.
- a1a1a1a1a1a1 6y ago
  If the benchmark is bound by the database speed, wouldn't the expected result be that all implementations returned roughly the same number of requests per second?
fraggle222 6y ago
>Sidenote here: one thing I found but didn't mention (the reason I put in the pooling, both in Python and pgbouncer) is that otherwise, under load, the async implementions would flood postgres with open connections and everything would just break down. Doesn't this prove that async is waiting for connections when you put a limit on it? The only way async wins is if it is free to hit the db whenever it needs to.
- pas 6y ago
  But why async is spending so much CPU if it just waits?
  - fraggle222 6y ago
    Who knows. The point is, if when not restricted you get a ton of db connections, then any restriction on that almost definitely means you are imposing a bottle neck. The only way this would not be the case is if it was trying to create db connections when it didn't need them, unlikely.
dna_polymerase 6y ago
So the CPU and database are the bottlenecks not async Python.
- supermatt 6y ago
  The benchmark is certainly flawed, but I don't see how you can jump to that conclusion.

supermatt 6y ago

Aren't you still capping the throughput by the query rate of your connection pool though? By limiting that, you are limiting the application as a whole - i.e. your benchmark is bound by the speed of your database, and has (almost) nothing to do with the performance of a specific python implementation.

arghwhat 6y ago
Only if there are spare resources left to saturate the connection pools, which didn't seem to be the case. If the system as a whole is well saturated, and the python processes dominate the system load with a DB load proportional to the requests served, then I don't think we would hit any external bottlenecks. The benchmarks performed are not that great (e.g., virtualized, same machine for all components, etc.), but I don't think the errors are enough to throw off the result. Note, of course, that such results are not universal, and some loads might perform better async.
a1a1a1a1a1a1 6y ago
If the benchmark is bound by the database speed, wouldn't the expected result be that all implementations returned roughly the same number of requests per second?

fraggle222 6y ago

>Sidenote here: one thing I found but didn't mention (the reason I put in the pooling, both in Python and pgbouncer) is that otherwise, under load, the async implementions would flood postgres with open connections and everything would just break down. Doesn't this prove that async is waiting for connections when you put a limit on it? The only way async wins is if it is free to hit the db whenever it needs to.

pas 6y ago
But why async is spending so much CPU if it just waits?
- fraggle222 6y ago
  Who knows. The point is, if when not restricted you get a ton of db connections, then any restriction on that almost definitely means you are imposing a bottle neck. The only way this would not be the case is if it was trying to create db connections when it didn't need them, unlikely.

dna_polymerase 6y ago

So the CPU and database are the bottlenecks not async Python.

supermatt 6y ago
The benchmark is certainly flawed, but I don't see how you can jump to that conclusion.