3 ms·

There are many misconceptions about async python I keep seing again and again. I'd like to offer an alternative point of view: - async python is faster, but fo

There are many misconceptions about async python I keep seing again and again. I'd like to offer an alternative point of view: - async python is faster, but for a very specific niche of workload. Most tasks you do don't fit that workload at all. Which mean most of the time, you should NOT use async Python. And it's ok. Don't make your code complicated when you don't need it. Chose a tech for the need, not the hype. - async python is not just for performances. It helps with making some specific kind of concurrency easy to reason about, because the context switch is explicit, and the chain of event can appear linearly in the code, thanks to await. - it's ok to have some sync processes and some async processes. It's not one or the other. - async is not a replacement for threads or processes. In fact, there are good use cases for having several processes, each with several threads, each with one event loop. Which also means that if you benchmark your WSGI code with 16 workers, you should do so as well with your aWSGI one. The corollary to this is that some web site loads are very well suited for async (e.g: an SPA with a lot of connectivity which delegates long running code to other services), but a lot are not. If each request makes a long SQL query, seeks the hard drive, dynamically performs i18n and adds some calculation on top, it may very well block the event loop for too long, killing any benefit. So when would you use async python ? - For performances, when you need to maintain numerous long connections. E.G: doing websockets ? Use asyncio. Serving static files without nginx ? Use asyncio. Want to create a web crawler and your memory budget does not allow to open 10000 threads ? Use asyncio. - For the interface, use the async/await keywords everywhere you need need inversion of control for I/O. It's not just about perf here, it's also a mechanism to delegate arbitrary parts of your code with a common standard interface around a fancy state machine + scheduler. You can use that to abstract all your I/O and switch backends at will, while offering callback inlining. But again, you have awesome threading and multiprocessing pools with python. Not to mention tools like zeromq, scrapy or celery, which do a lot for you. Don't run to async just because "it's faster". Having async by default in frameworks like fastpi opens up a ton of possibility though. Live settings, pub/sub between processes, websockets...

BMorearty 5y ago
I can’t tell if your first list is misconceptions or your rebuttals to misconceptions. You said it’s misconceptions but I think it’s your rebuttals.
- BiteCode_dev 5y ago
  You are right, I edited it.
dehrmann 5y ago
> It helps with making some specific kind of concurrency easy to reason about, because the context switch is explicit Assuming you're disagreeing with this, yeah, I've seeing "easier concurrency" be a massive footgun because you have code that's safe as long as an await doesn't get added in the wrong place.
- BiteCode_dev 5y ago
  I am not, I've been using the wrong working. I would say it's pretty hard to add an await in the wrong place, although not impossible. I think it's easier to: - forget an await - forget a asyncio.wait or a asyncio.gather The later is very common, and is being addressed by the structured concurrency movement we've seen with Trio and the likes. This should end up in mainline python eventually because it is, indeed, a problem. However, it's still way easier to shot yourself in the foot with concurrency and threads than with asyncio. However, it is harder to startup with asyncio than with threads, because there is more to learn just to boot.
  - dehrmann 5y ago
    > forget an await Do you use a typechecker?
    - BiteCode_dev 5y ago
      Always. But not using an await is not a type error. It's perfectly valid to not call await. There are valid reasons to do it.

BMorearty 5y ago

I can’t tell if your first list is misconceptions or your rebuttals to misconceptions. You said it’s misconceptions but I think it’s your rebuttals.

BiteCode_dev 5y ago
You are right, I edited it.

dehrmann 5y ago

> It helps with making some specific kind of concurrency easy to reason about, because the context switch is explicit Assuming you're disagreeing with this, yeah, I've seeing "easier concurrency" be a massive footgun because you have code that's safe as long as an await doesn't get added in the wrong place.

BiteCode_dev 5y ago
I am not, I've been using the wrong working. I would say it's pretty hard to add an await in the wrong place, although not impossible. I think it's easier to: - forget an await - forget a asyncio.wait or a asyncio.gather The later is very common, and is being addressed by the structured concurrency movement we've seen with Trio and the likes. This should end up in mainline python eventually because it is, indeed, a problem. However, it's still way easier to shot yourself in the foot with concurrency and threads than with asyncio. However, it is harder to startup with asyncio than with threads, because there is more to learn just to boot.
- dehrmann 5y ago
  > forget an await Do you use a typechecker?
  - BiteCode_dev 5y ago
    Always. But not using an await is not a type error. It's perfectly valid to not call await. There are valid reasons to do it.