3 ms·

There are so many footguns[1] in async python but it really has stolen the zeitgeist of modern python web. Synchronous django has a lot to commend it. If you de

There are so many footguns[1] in async python but it really has stolen the zeitgeist of modern python web. Synchronous django has a lot to commend it. If you deploy it reasonably well (workers, behind a caching reverse proxy etc) it's easy to operate in production even under duress. It's not going to be the right stack for long lived websocket connections or whatever but for a CRUD-ish or enterprise app, often very productive. [1] buffer bloat is too easy to sleep walk into queue = asyncio.Queue() # oops unbounded concurrency await asyncio.gather(*(fetch(item) for item in items)) # look mum! no outbound sockets left or maybe even no file descriptors accidental blocking data = requests.get(url).json() # oops! we're blocking the event loop and sure you can setup a separate task to measure the event loop latency and alert on that but this is the kinda thing you'll never find in a tutorial, you just need to experience this stuff and figure out a solution you like I can go on and on about async python (cancellation bugs, leaking a task - that has a cataclysmic failure mode where if you forget to hold a reference to your asyncio.create_task() then it's all weak refs in that machinery so your task can get garbage collected before it ran or completed in production! super tricky forensics. Then there's all the obvious stuff like race conditions, new ways of creating deadlocks, blah blah i really can go on for days.

loevborg 2mo ago
Please go on. I also feel like asyncio is a big hack. Even just accidentally blocking the event loop is way too easy. And I couldn't believe it when I read that you need to store a reference to a task in a set to keep it from accidentally getting canceled. How did this become the main stack of AI backends? It's like node but slower AND much easier to mess up because of synchronous IO.
- CraigJPerry 2mo ago
  I'm not sure i see it as a hack, but i do feel unduly burdened as a developer by async in python. I feel like there's a lot that i have to think about and i'd appreciate more help from the language. I don't want to be overly critical, there's languages that people complain about and then there are languages that no one uses... If i compare it to js/ts, some stuff is genuinely better in Python - e.g. if you missed an await. While both ecosystems have lint tools available for this, but the behaviour is just friendlier in python. Structured concurrency is better in python, but even if TaskGroup is nicer to use than AbortController, it still has its own foibles which means i'll usually advocate for AnyIO. But the js/ts ecosystem just generally benefits from being async from the get-go where python you're just a time.sleep() away from a bug that will slip through dev envs and ci pipelines undetected and only rear its head under load in prod. If i have a tip to share its the debug flag for asyncio run: asyncio.run(f(), debug=True) # find some more issues before prod
  - mstaoru 2mo ago
    Or just env PYTHONASYNCIODEBUG=1.
- selfmodruntime 2mo ago
  Well references in python are mutable by default, so you essentially combine an asynchronous model with mutable shared state. Combine that model with unknown caller exceptions in any subfunction and you're in for a world of hurt.
codethief 2mo ago
I would love a blog post on this, in case you ever considered writing one!
marcosdumay 2mo ago
Yep, if you are going to make asynchronous programming, you need either a small, plain, and obvious layer that calls insulated synchronous code or some stack with high quality assurances.
ltbarcly3 2mo ago
async python is always slower than just naively using threads. I don't think there is any application in production anywhere that would not prove me right on this. the complete lack of any kind of scheduling is a disaster. unless you know exactly what I mean by a scheduler please don't reply to this.
- zbentley 2mo ago
  I don’t think there’s anything Python specific about that criticism. Most languages with cooperative concurrency have simple or no schedulers and run coroutines pretty much directly (or unpredictably, if they’re scheduled into parallel executors). Even Java and Tokio’s scheduling, such as it is, isn’t that directly controllable or observable and doesn’t magically fix loop-blocking or cache efficiency or whatever. I think you might just not like cooperative async IO multiplexing as a programming model. Which is fine, but has nothing to do with Python.
  - ltbarcly3 2mo ago
    I don't like it, but I don't like it because it is impossible to make it performant. What if I told you there was one of the greatest engineering achievements of mankind ready to make sure your threads run whenever they can, but also fairly and carefully balancing context switching cost and throughput, and instead you use python async and get 1/10th the performance, weird impossible to avoid pauses an order of magnitude longer than the threaded version's median response time, and just constantly have to fix bugs about callbacks firing on canceled blah blah blah.
    - zbentley 2mo ago
      Those are all fair points! They apply more or less equally to, say, nodejs or Perl's AnyEvent. I'm totally onboard with "use the OS thread scheduler when you can"; it's a brilliant piece of technology. And if you are in a situation where > pauses [are] an order of magnitude longer than the threaded version's median response time ...then fully cooperative, single-thread async programming isn't for you! But async programming was made for the inverse use-case of that: backend functions which spend the vast majority of their time waiting for I/O--slow databases, HTTP requests, and whatnot--and which needed to serve very high concurrency demands on very cheap servers. If, say, your web route handler wants to probe or parallel-fetch from 100 unique-per-request HTTP endpoints, and you expect to handle 100 web requests concurrency, using threads is going to cost a lot of resources (the portion stack space that's allocated eagerly, start/stop time, and so on). For some folks, those resources might be readily available, in which case threads may still be the way to go! But for other folks (tiny servers, or more orders of magnitude than 100/100--use cases like proxies or CDNs) cooperative async might be a better way to go. Similarly, if your concurrent code is sharing a lot of data between routines but is primarily waiting on I/O, single-thread async concurrency might be a less race-condition-bug-prone model to use. I'm generally with you that it's a tool that's prioritized above threads in a lot of cases when it shouldn't be. But it is still definitely valuable for a lot of common challenges. A lot of the above is rehashing the comments I made here: https://news.ycombinator.com/item?id=39247876 https://news.ycombinator.com/item?id=39247876
jorl17 2mo ago
Everytime I have to deal with websockets in Django, I want to rip my eyeballs out. Most of the time what I really want is just push support, and don't need bidirectional communication, i.e. SSE. But it's the same thing: both are a pain to work with in Django. I find django-channels so cumbersome. So, for my latest projects, I've ended up offloading this push functionality to a service like Centrifugo. Django talks to Centrifugo, and clients connect to it as well. It makes Django still be django, without worrying about django-channels, ASGI and all of that complicated setup. I've been very, very, very happy with this decision, even if I do have some additional complexity to articulate the two. It is nowhere near as confusing and convoluted as django-channels and uvicorn/daphne, though. As for async python in general, fully agree. Every couple of years I decide to give it a go with some new tool or framework, thinking surely now it'll finally just click and I'll get with the times. Every time I feel disappointed with footguns and absolutely terrible ergonomics. No, thanks.
- Oxodao 2mo ago
  For lots of applications, Mercure is a great replacement for websockets and it works wonderfully with django
- jkarneges 2mo ago
  You don't need django-channels anymore for SSE: https://docs.djangoproject.com/en/6.0/ref/request-response/#django.http.StreamingHttpResponse.is_async https://docs.djangoproject.com/en/6.0/ref/request-response/#...
whakim 2mo ago
I agree that there's plenty of footguns with async python, but I think the hype is a bit overblown. A few sensible defaults in the standard library would go a long way, e.g. requiring a maxsize argument to Queue (or None if you really need it to be unbounded). The "accidental blocking" thing seems like less of design issue - it has much more to do with python's super long history as a sync-first language. Taking a step back, though, async is popular because it is genuinely extremely useful! If you need to do a bunch of work and that work is mostly IO-bound, then it makes sense to live in an async world.
physicsguy 2mo ago
I think people in the Python world just don't yet have a good mental model of async either. I recently have had conversations with a number of people who have switched to FastAPI because "it's faster" even when they're using it to serve ML models which are compute bound and there's nothing you can await on.
- CraigJPerry 2mo ago
  I think the mental model is the right thing to focus on. I'm not denying there are cases where async at the edge of an app can make sense but there's no free lunch and i that doesn't appear to be well understood. Maybe i will write a blog post, if nothing else than to get my own thoughts in order.

loevborg 2mo ago

Please go on. I also feel like asyncio is a big hack. Even just accidentally blocking the event loop is way too easy. And I couldn't believe it when I read that you need to store a reference to a task in a set to keep it from accidentally getting canceled. How did this become the main stack of AI backends? It's like node but slower AND much easier to mess up because of synchronous IO.

CraigJPerry 2mo ago
I'm not sure i see it as a hack, but i do feel unduly burdened as a developer by async in python. I feel like there's a lot that i have to think about and i'd appreciate more help from the language. I don't want to be overly critical, there's languages that people complain about and then there are languages that no one uses... If i compare it to js/ts, some stuff is genuinely better in Python - e.g. if you missed an await. While both ecosystems have lint tools available for this, but the behaviour is just friendlier in python. Structured concurrency is better in python, but even if TaskGroup is nicer to use than AbortController, it still has its own foibles which means i'll usually advocate for AnyIO. But the js/ts ecosystem just generally benefits from being async from the get-go where python you're just a time.sleep() away from a bug that will slip through dev envs and ci pipelines undetected and only rear its head under load in prod. If i have a tip to share its the debug flag for asyncio run: asyncio.run(f(), debug=True) # find some more issues before prod
- mstaoru 2mo ago
  Or just env PYTHONASYNCIODEBUG=1.
selfmodruntime 2mo ago
Well references in python are mutable by default, so you essentially combine an asynchronous model with mutable shared state. Combine that model with unknown caller exceptions in any subfunction and you're in for a world of hurt.

codethief 2mo ago

I would love a blog post on this, in case you ever considered writing one!

marcosdumay 2mo ago

Yep, if you are going to make asynchronous programming, you need either a small, plain, and obvious layer that calls insulated synchronous code or some stack with high quality assurances.

ltbarcly3 2mo ago

async python is always slower than just naively using threads. I don't think there is any application in production anywhere that would not prove me right on this. the complete lack of any kind of scheduling is a disaster. unless you know exactly what I mean by a scheduler please don't reply to this.

zbentley 2mo ago
I don’t think there’s anything Python specific about that criticism. Most languages with cooperative concurrency have simple or no schedulers and run coroutines pretty much directly (or unpredictably, if they’re scheduled into parallel executors). Even Java and Tokio’s scheduling, such as it is, isn’t that directly controllable or observable and doesn’t magically fix loop-blocking or cache efficiency or whatever. I think you might just not like cooperative async IO multiplexing as a programming model. Which is fine, but has nothing to do with Python.
- ltbarcly3 2mo ago
  I don't like it, but I don't like it because it is impossible to make it performant. What if I told you there was one of the greatest engineering achievements of mankind ready to make sure your threads run whenever they can, but also fairly and carefully balancing context switching cost and throughput, and instead you use python async and get 1/10th the performance, weird impossible to avoid pauses an order of magnitude longer than the threaded version's median response time, and just constantly have to fix bugs about callbacks firing on canceled blah blah blah.
  - zbentley 2mo ago
    Those are all fair points! They apply more or less equally to, say, nodejs or Perl's AnyEvent. I'm totally onboard with "use the OS thread scheduler when you can"; it's a brilliant piece of technology. And if you are in a situation where > pauses [are] an order of magnitude longer than the threaded version's median response time ...then fully cooperative, single-thread async programming isn't for you! But async programming was made for the inverse use-case of that: backend functions which spend the vast majority of their time waiting for I/O--slow databases, HTTP requests, and whatnot--and which needed to serve very high concurrency demands on very cheap servers. If, say, your web route handler wants to probe or parallel-fetch from 100 unique-per-request HTTP endpoints, and you expect to handle 100 web requests concurrency, using threads is going to cost a lot of resources (the portion stack space that's allocated eagerly, start/stop time, and so on). For some folks, those resources might be readily available, in which case threads may still be the way to go! But for other folks (tiny servers, or more orders of magnitude than 100/100--use cases like proxies or CDNs) cooperative async might be a better way to go. Similarly, if your concurrent code is sharing a lot of data between routines but is primarily waiting on I/O, single-thread async concurrency might be a less race-condition-bug-prone model to use. I'm generally with you that it's a tool that's prioritized above threads in a lot of cases when it shouldn't be. But it is still definitely valuable for a lot of common challenges. A lot of the above is rehashing the comments I made here: https://news.ycombinator.com/item?id=39247876 https://news.ycombinator.com/item?id=39247876

jorl17 2mo ago

Everytime I have to deal with websockets in Django, I want to rip my eyeballs out. Most of the time what I really want is just push support, and don't need bidirectional communication, i.e. SSE. But it's the same thing: both are a pain to work with in Django. I find django-channels so cumbersome. So, for my latest projects, I've ended up offloading this push functionality to a service like Centrifugo. Django talks to Centrifugo, and clients connect to it as well. It makes Django still be django, without worrying about django-channels, ASGI and all of that complicated setup. I've been very, very, very happy with this decision, even if I do have some additional complexity to articulate the two. It is nowhere near as confusing and convoluted as django-channels and uvicorn/daphne, though. As for async python in general, fully agree. Every couple of years I decide to give it a go with some new tool or framework, thinking surely now it'll finally just click and I'll get with the times. Every time I feel disappointed with footguns and absolutely terrible ergonomics. No, thanks.

Oxodao 2mo ago
For lots of applications, Mercure is a great replacement for websockets and it works wonderfully with django
jkarneges 2mo ago
You don't need django-channels anymore for SSE: https://docs.djangoproject.com/en/6.0/ref/request-response/#django.http.StreamingHttpResponse.is_async https://docs.djangoproject.com/en/6.0/ref/request-response/#...

whakim 2mo ago

I agree that there's plenty of footguns with async python, but I think the hype is a bit overblown. A few sensible defaults in the standard library would go a long way, e.g. requiring a maxsize argument to Queue (or None if you really need it to be unbounded). The "accidental blocking" thing seems like less of design issue - it has much more to do with python's super long history as a sync-first language. Taking a step back, though, async is popular because it is genuinely extremely useful! If you need to do a bunch of work and that work is mostly IO-bound, then it makes sense to live in an async world.

physicsguy 2mo ago

I think people in the Python world just don't yet have a good mental model of async either. I recently have had conversations with a number of people who have switched to FastAPI because "it's faster" even when they're using it to serve ML models which are compute bound and there's nothing you can await on.

CraigJPerry 2mo ago
I think the mental model is the right thing to focus on. I'm not denying there are cases where async at the edge of an app can make sense but there's no free lunch and i that doesn't appear to be well understood. Maybe i will write a blog post, if nothing else than to get my own thoughts in order.