3 ms·

then you have to rewrite your whole app to use asyncio keywords and colored ORM methods. A gevent monkey patch, or eventually nogil concurrency makes a lot more

then you have to rewrite your whole app to use asyncio keywords and colored ORM methods. A gevent monkey patch, or eventually nogil concurrency makes a lot more practical sense.

simonw 2y ago
You don't have to rewrite your whole app - you can continue using the regular stuff in synchronous view functions, then have a few small async views for your LLM streaming pieces. I've never quite gotten comfortable with gevent patches, but that's more because I don't personally understand them or what their edge cases might be than a commentary on their reliability.
- pphysch 2y ago
  Last time I checked, simply running Django through the ASGI interface dealt you a baseline perf hit.
  - simonw 2y ago
    It's pretty minimal: https://docs.djangoproject.com/en/5.1/topics/async/#performance https://docs.djangoproject.com/en/5.1/topics/async/#performa... > When running in a mode that does not match the view (e.g. an async view under WSGI, or a traditional sync view under ASGI), Django must emulate the other call style to allow your code to run. This context-switch causes a small performance penalty of around a millisecond.
- btown 2y ago
  Gevent IMO is more of a having-discipline footgun than a fundamental one: it makes itself so easy to use, without sprinkling asyncs in your day-to-day code, that you can forget that it has the same problem as explicit Python asyncio… namely, that if you burn CPU in a tight loop without any IO or logging or anything else, you’ll never yield to the event loop, and will block all concurrency until you’re done. Detecting this might require external health checks and the like. And runaway regexes can trigger this too, not just tight loops. But those are also problems that asyncio has! The only other one is that you’ll want to think about forking models with servers like gunicorn, and patch at the right time - but this is a one-time thing to figure out per project, and documentation is quite good now.

then you have to rewrite your whole app to use asyncio keywords and colored ORM methods. A gevent monkey patch, or eventually nogil concurrency makes a lot more practical sense.

simonw 2y ago

You don't have to rewrite your whole app - you can continue using the regular stuff in synchronous view functions, then have a few small async views for your LLM streaming pieces. I've never quite gotten comfortable with gevent patches, but that's more because I don't personally understand them or what their edge cases might be than a commentary on their reliability.

pphysch 2y ago
Last time I checked, simply running Django through the ASGI interface dealt you a baseline perf hit.
- simonw 2y ago
  It's pretty minimal: https://docs.djangoproject.com/en/5.1/topics/async/#performance https://docs.djangoproject.com/en/5.1/topics/async/#performa... > When running in a mode that does not match the view (e.g. an async view under WSGI, or a traditional sync view under ASGI), Django must emulate the other call style to allow your code to run. This context-switch causes a small performance penalty of around a millisecond.
btown 2y ago
Gevent IMO is more of a having-discipline footgun than a fundamental one: it makes itself so easy to use, without sprinkling asyncs in your day-to-day code, that you can forget that it has the same problem as explicit Python asyncio… namely, that if you burn CPU in a tight loop without any IO or logging or anything else, you’ll never yield to the event loop, and will block all concurrency until you’re done. Detecting this might require external health checks and the like. And runaway regexes can trigger this too, not just tight loops. But those are also problems that asyncio has! The only other one is that you’ll want to think about forking models with servers like gunicorn, and patch at the right time - but this is a one-time thing to figure out per project, and documentation is quite good now.