2 ms·

The diagnostic is right, but the remedy given in the article seems worse than the disease. Using backends to make Django pool the connections is sub-optimal. Y

The diagnostic is right, but the remedy given in the article seems worse than the disease. Using backends to make Django pool the connections is sub-optimal. You'll have a different pool per worker process, and they won't persist when you restart your workers - right when you need low latency the most. Use pgbouncer/pgpool instead. On top of that, these poolers have some nice features.

zzzeek 14y ago
if all worker processes have an evenly distributed workload, how is pool-per-worker process a disadvantage? it's still a fixed-size pool, just one that's an aggregate of several smaller pools.
- hcarvalhoalves 14y ago
  Appended to my comment: > You'll have a different pool per worker process, and they won't persist when you restart your workers - right when you need low latency the most
  - zzzeek 14y ago
    So, pooling that has TCP/DB initialize connect overhead during a restart only is "worse than" having TCP/DB initialize connect overhead on restart as well as every subsequent request ? There's pros/cons to in-app pooling vs. middleware pooling, but calling in-app pooling "worse" than no pooling at all is kind of misleading. If I were using PGBouncer, I'd probably leave SQLA's pooling on anyway, so at least you have no TCP latency at all on connect. unless you're dealing with apps that go unevenly idle/active.
    - hcarvalhoalves 14y ago
      > but calling in-app pooling "worse" than no pooling You don't need to get all worked up. Nowhere I affirm it's worse than no polling. I'm talking about not reinventing the wheel, specially since pooling inside the worker is inferior than just setting up pgpool.
      - zzzeek 14y ago
        there's a lot of cases where just running an app and having it talk to the database in a halfway-efficient way is a lot more convenient than having to install and start a whole separate service. As well as, in-app pooling is the easiest way to limit the max simultaneous connection checkouts per application. If PGBouncer were to handle this, I'd assume you'd need to either send distinct usernames or some other instance-specific token over so that it could identify sources of connections.

zzzeek 14y ago

if all worker processes have an evenly distributed workload, how is pool-per-worker process a disadvantage? it's still a fixed-size pool, just one that's an aggregate of several smaller pools.

hcarvalhoalves 14y ago
Appended to my comment: > You'll have a different pool per worker process, and they won't persist when you restart your workers - right when you need low latency the most
- zzzeek 14y ago
  So, pooling that has TCP/DB initialize connect overhead during a restart only is "worse than" having TCP/DB initialize connect overhead on restart as well as every subsequent request ? There's pros/cons to in-app pooling vs. middleware pooling, but calling in-app pooling "worse" than no pooling at all is kind of misleading. If I were using PGBouncer, I'd probably leave SQLA's pooling on anyway, so at least you have no TCP latency at all on connect. unless you're dealing with apps that go unevenly idle/active.
  - hcarvalhoalves 14y ago
    > but calling in-app pooling "worse" than no pooling You don't need to get all worked up. Nowhere I affirm it's worse than no polling. I'm talking about not reinventing the wheel, specially since pooling inside the worker is inferior than just setting up pgpool.
    - zzzeek 14y ago
      there's a lot of cases where just running an app and having it talk to the database in a halfway-efficient way is a lot more convenient than having to install and start a whole separate service. As well as, in-app pooling is the easiest way to limit the max simultaneous connection checkouts per application. If PGBouncer were to handle this, I'd assume you'd need to either send distinct usernames or some other instance-specific token over so that it could identify sources of connections.