6 ms·

Just a heads up that currently Django is not cleaning up open PostgreSQL connections when ran in ASGI mode, leading to too many open connections error: https://

by Ralfp 3y ago

Just a heads up that currently Django is not cleaning up open PostgreSQL connections when ran in ASGI mode, leading to too many open connections error: https://code.djangoproject.com/ticket/33497 https://code.djangoproject.com/ticket/33497

whalesalad 3y ago
that’s kind of a dealbreaker
- thefreeman 3y ago
  could this be addressed by something like pgbouncer or another connection pooler?
  - whalesalad 3y ago
    Perhaps yes, would be worthwhile to benchmark an internal (to your runtime) conn pooler versus external only. I'm using internal pools + pgbouncer but as I write this out I am beginning to wonder if that is even necessary.
  - verandaguy 3y ago
    Going off the pooling mode feature map[0], it might be possible to mitigate that issue using transaction pooling, but not session pooling — which means a restricted feature set and cooperation from the application (which would just not call restricted features up). [0] http://www.pgbouncer.org/features.html#fn:2
pdhborges 3y ago
I'll have zero trust running async Django until all sync_to_async and async_to_sync calls are removed from the code base.
- deleted 3y ago
  [deleted]
- lisasays 3y ago
  Care to elaborate?
  - pdhborges 3y ago
    Look at the intended semantics [1], and then read the implementation [2]. Can you figure out if the implementation is correct? Can you infer the possible limitations of the approach at glance? Can your async library actually handle being called with multiple event loops installed? I have zero trust in this code and I have been bitten by fixes to this library that introduced deadlocks in my own code. [1] https://github.com/django/asgiref#synchronous-code--threads https://github.com/django/asgiref#synchronous-code--threads. [2] https://github.com/django/asgiref/blob/main/asgiref/sync.py#L130 https://github.com/django/asgiref/blob/main/asgiref/sync.py#...
    - lisasays 3y ago
      Doesn't look like something I'd want to have to debug in a pinch, that's for sure. Is there anything else in Python-land you would go with for the use case you have in mind? Or does pretty much all the async stuff turn on you like this, at some point or another?
      - pdhborges 3y ago
        If you need to buy into the async ecosystem for some special reason my recomendation is to: - pick a mature ASGI server - pick a framework and libraries that have been designed from the start with async in mind. Be suspicious of libraries that support sync and async APIs without clear guard rails between the too implementations - avoid setups where you have multiple event loops in a single process.
        lisasays 3y ago
        Thanks - so is there anything in the Python space that ticks those boxes, in your view?
- tomwojcik 3y ago
  Same here, but without these weird utils it doesn't get any better. I have 7 YoE with Django. Its great at so many things. You see some code, like middlewares, and immediately understand what's going on. Now, we also have Starlette. The base of all new, fancy asgi libraries. Here's the base middleware class. https://github.com/encode/starlette/blob/8d7a1cacfb3e1a30cbb5d8e207aa1917bad9a3e2/starlette/middleware/base.py#L92 https://github.com/encode/starlette/blob/8d7a1cacfb3e1a30cbb... In the last couple of years I heard 'we're running fastapi on production. Wanna join us?' so many times... but the reality is that it's still not suitable for prod. Who wants to work with a code like that if you have a readable, stable Django? I'm clueless.
baq 3y ago
Running Postgres without a connection bouncer is a huuuge no-no already, but if this breaks the default mode of pgbouncer it’s basically a showstopper.
- winrid 3y ago
  Your average django app running on one or two servers with four workers each (which use connection polling) does not need pgbouncer.
kolanos 3y ago
Reading that bug report it looks like Django hasn't been isolated as the source of the problem? Looks like Carlton Gibson closed it until it is reproducible. Problem could be in uvicorn, psycopg2, etc
- Ralfp 3y ago
  It was. Its an implementation change made in Django v4 that caused this.
etimberg 3y ago
Yeah, I've definitely seen odd things in Django ASGI. Had at least one instance where it looked like it leaked data between requests but couldn't get a clear enough reproduce. Switching back to WSGI worked fine though

by Ralfp 3y ago

whalesalad 3y ago

that’s kind of a dealbreaker

thefreeman 3y ago
could this be addressed by something like pgbouncer or another connection pooler?
- whalesalad 3y ago
  Perhaps yes, would be worthwhile to benchmark an internal (to your runtime) conn pooler versus external only. I'm using internal pools + pgbouncer but as I write this out I am beginning to wonder if that is even necessary.
- verandaguy 3y ago
  Going off the pooling mode feature map[0], it might be possible to mitigate that issue using transaction pooling, but not session pooling — which means a restricted feature set and cooperation from the application (which would just not call restricted features up). [0] http://www.pgbouncer.org/features.html#fn:2

pdhborges 3y ago

I'll have zero trust running async Django until all sync_to_async and async_to_sync calls are removed from the code base.

deleted 3y ago
[deleted]
lisasays 3y ago
Care to elaborate?
- pdhborges 3y ago
  Look at the intended semantics [1], and then read the implementation [2]. Can you figure out if the implementation is correct? Can you infer the possible limitations of the approach at glance? Can your async library actually handle being called with multiple event loops installed? I have zero trust in this code and I have been bitten by fixes to this library that introduced deadlocks in my own code. [1] https://github.com/django/asgiref#synchronous-code--threads https://github.com/django/asgiref#synchronous-code--threads. [2] https://github.com/django/asgiref/blob/main/asgiref/sync.py#L130 https://github.com/django/asgiref/blob/main/asgiref/sync.py#...
  - lisasays 3y ago
    Doesn't look like something I'd want to have to debug in a pinch, that's for sure. Is there anything else in Python-land you would go with for the use case you have in mind? Or does pretty much all the async stuff turn on you like this, at some point or another?
    - pdhborges 3y ago
      If you need to buy into the async ecosystem for some special reason my recomendation is to: - pick a mature ASGI server - pick a framework and libraries that have been designed from the start with async in mind. Be suspicious of libraries that support sync and async APIs without clear guard rails between the too implementations - avoid setups where you have multiple event loops in a single process.
      - lisasays 3y ago
        Thanks - so is there anything in the Python space that ticks those boxes, in your view?
tomwojcik 3y ago
Same here, but without these weird utils it doesn't get any better. I have 7 YoE with Django. Its great at so many things. You see some code, like middlewares, and immediately understand what's going on. Now, we also have Starlette. The base of all new, fancy asgi libraries. Here's the base middleware class. https://github.com/encode/starlette/blob/8d7a1cacfb3e1a30cbb5d8e207aa1917bad9a3e2/starlette/middleware/base.py#L92 https://github.com/encode/starlette/blob/8d7a1cacfb3e1a30cbb... In the last couple of years I heard 'we're running fastapi on production. Wanna join us?' so many times... but the reality is that it's still not suitable for prod. Who wants to work with a code like that if you have a readable, stable Django? I'm clueless.

baq 3y ago

Running Postgres without a connection bouncer is a huuuge no-no already, but if this breaks the default mode of pgbouncer it’s basically a showstopper.

winrid 3y ago
Your average django app running on one or two servers with four workers each (which use connection polling) does not need pgbouncer.

kolanos 3y ago

Reading that bug report it looks like Django hasn't been isolated as the source of the problem? Looks like Carlton Gibson closed it until it is reproducible. Problem could be in uvicorn, psycopg2, etc

Ralfp 3y ago
It was. Its an implementation change made in Django v4 that caused this.

etimberg 3y ago

Yeah, I've definitely seen odd things in Django ASGI. Had at least one instance where it looked like it leaked data between requests but couldn't get a clear enough reproduce. Switching back to WSGI worked fine though