10 ms·

Async Python is not faster (2020)

89 points by xk3 5y ago

mahoro 5y ago
Django 1.10 from this "benchmark" has been released 5 years ago (the article is from 2020). WAT?
- cors-fls 5y ago
  Django 1.10 appears in Vibora's benchmark which is merely cited at the end of the article. The benchmark of the author does not test Django.
meitros 5y ago
I understand what the article is saying but there’s still a something I’m missing. All I want (for my admittedly simple service) is to be able to send multiple api calls or database calls in parallel rather than one at a time to reduce overall endpoint latency. I guess I should be using a normal synchronous framework that just calls asyncio.run within each api call?
- anonymoushn 5y ago
  At an instant messaging company we did this in 2010 with greenlet. In current_year, everything assumes you will use async, so you should probably use async.
rjbwork 5y ago
Does anyone expect async anything to be faster? AFAIU async enables greater scalability by allowing computation to continue in other contexts while awaiting read/write operations - at the cost of slightly lower single context performance.
- davidkunz 5y ago
  Not sure about Python's async story, but in Node.js it is essential if your program's performance is I/O bound. There's no reason to let a thread stay idle while waiting for a database operation to finish.
  - rjbwork 5y ago
    That doesn't make your code faster though. For a particular bit of single threaded computation the async approach is going to be slower due to context switching overhead. Same in C#, Rust, etc. If your performance is I/O bound, then, as I said, it enables greater scalability by allowing computation to continue on other contexts while waiting for that I/O operation.
    - davidkunz 5y ago
      Yes, this is absolutely correct. I don't think anyone would argue that async code makes CPU-bound computations faster, the opposite is the case because you have naturally more overhead.
  - frenchyatwork 5y ago
    I don't think you disagreed at all with rjbwork at all, it's just you're using words differently. The article, and rjbwork, use slower to mean "slower when the CPU isn't regularly idling". In this context, Node.js async is slower as well, that's why the library better-sqlite3 for NodeJS is synchronous (because you're rarely going to be using sqlite in a context where the CPU/memory isn't very busy).
    - davidkunz 5y ago
      Yes, I don't disagree with rjbwork. I may have read the article a bit differently, it sounded like "don't bother too much with async, it's slower, even for realistic web servers". In your I/O-bound Node.js server, if you would exchange all async calls with synchronous ones, your performance would get a big hit.
  - guywhocodes 5y ago
    Yeah like being nagged on that you should async your fstat calls, which makes them four times slower.
  - __turbobrew__ 5y ago
    Doesn’t your thread just get put to sleep by the kernel when it is blocked on I/O? Unless Node busy waits on I/O I don’t see the problem?
- dangerwill 5y ago
  Yeah, I thought of this as a given. With the real benefit being that your fleet is able to handle a higher and not-hard-limited number of requests in flight, and the trade-off being the increased code complexity
- kayodelycaon 5y ago
  > Does anyone expect async anything to be faster? Judging from what I've read and seen, yes. Async may scale better, but are you working at scales where that pays off?
- cerved 5y ago
  Yes. Anything that does a lot of I/O to slow stuff like disk, network is going to benefit. So in practice, lots of things.
VWWHFSfQ 5y ago
Overall I've been extremely unimpressed with Python's async story. They've spent a decade baking into the core of the language and the result is that it's no better than just using gevent or twisted or any of those other "old school" frameworks. Except now everyone is forced to support it because there are colored functions everywhere and you have to write library code that has to deal with all the myriad ways your stuff can be used depending on if you're in an event loop or not. It's baffling. Async/await at the language level is a total scam. Give me Nginx+Lua and let me just write my regular procedural code and the underlying runtime will handle yielding/resuming for me.
- anonymoushn 5y ago
  It has certainly been depressing, as a greenlet user over a decade ago, to watch people spend weeks rewriting codebases to add "await" in all the right places.
  - moffkalast 5y ago
    Name two things that go better together, Python devs and breaking shit that has no reason to be broken.
- MrBuddyCasino 5y ago
  > async/await at the language level is a total scam This view will magically become mainstream as soon as "having worked with async" stops being a resume inflating selling point. I'm very glad Java is doing the right thing and hiding everything under the same "Thread" abstraction.
- rich_sasha 5y ago
  Most code seems to take sides: it’s either synchronous or not (which I suppose in itself isn’t great). For many good standard libraries/packages, there is now an async variant. I used asyncio for the first time seriously recently and was pleasantly surprised. But, crucially, I didn’t want throughput, I wanted a low-cpu use app with easy-to-write concurrency. I really hope GIL goes away and we can just use threads.
srcreigh 5y ago
The error(?) in this benchmark is putting the DB on the same machine as the server. Async solves the problem of not wanting to block other requests while you go out to expensive external resources. If you benchmark stuff while it's all on the same machine, it masks that cost. I couldn't find the size of the DB. Is it 100GB of data? or more like 10MB? Doing one point query on 10MB might not touch even disk a whole lot. If you were using srcreigh's hosted DB service and queries took 1s to run, the sync servers throughput would suffer greatly, meanwhile async servers would still perform pretty good.
- tempest_ 5y ago
  This was what I noticed as well. The minute your db queries become expensive the sync workers end up slowing down and stopping other functions. If your workers are tied up doing expensive queries you might be holding up cheap queries that might be getting data from a cache or whatever.
- oefrha 5y ago
  > If you were using srcreigh's hosted DB service and queries took 1s to run, your sync server latency would be ~1s :) Or longer if they get queued up. Yep, and your 16-worker gunicorn is going to serve 16 rps. What's shown here (nginx, pg, Python application running on the same low CPU VPS, one extremely fast db query per request) is not a "realistic benchmark".
- stickyricky 5y ago
  If you're at the point where you've got a single application server with 100 waiting connections on a database, you've already blundered. Firstly, you're either overpaying for your database (connections aren't free) or you're underpaying for application servers. Fixing the problem in either dimension reduces the utility of your async server. There's also this myth that Python code is free if the i/o wait is long enough. That's not true. 1). 100 queued connections on an async python server sucks so much momentum out your application. Especially in a resource constrained cloud environment. 2). Any server awaiting a database is going to transform the data to some response object which is going to wreck your overloaded async server. 3). As far as I know, all the async WSGIs are written in Python and will use WAY more CPU than the C-based sync WSGIs.
stickyricky 5y ago
tl;dr: Async (1) is not faster, (2) does not scale, and (3) is not stable under load. Yes and no matter how many times its said people will still opt for the async option even though its slower in the general case. I went so far as to benchmark this myself in 2019 using EC2, digital ocean, and my local machine. I nearly published a blog post but I don't appreciate the attention that brings. Gunicorn is not stable under heavy load. uWSGI delivers better throughput. If we talk about cloud environments exclusively, uWSGI is better full stop. If you have endpoints with long awaits consider a strategy other than holding open the connection. If you need to run a socket server then run it separately from your api server.
deleted 5y ago
[deleted]
djstein 5y ago
The strangest part of this dataset to me was tests being run using Daphne with Starlette, instead of Django. Unclear who would reach for that combo in production, since Daphne is meant for Django.
- sergiomattei 5y ago
  If it's ASGI compliant, there's no real difference. Daphne is written to be a reference implementation of ASGI, not as a Django-specific server. If you're using Django Channels in production you might even be better served by trying out Uvicorn, it's a lot faster.
Uberphallus 5y ago
> Benchmark > Uses a VPS Not saying the results are wrong (though for client side code my experience is very different), but that's not really a good start.
oefrha 5y ago
This was already thoroughly discussed at the time: https://news.ycombinator.com/item?id=23496994 https://news.ycombinator.com/item?id=23496994 And a rebuttal from Python core dev @ambv at the time pointing out how the methodology's flawed: https://twitter.com/llanga/status/1271719778324025349 https://twitter.com/llanga/status/1271719778324025349
- mpweiher 5y ago
  I don't see anything of substance in those tweets. It was slower with the fewer threads, and giving it more threads made it even slower. Celebrating this as awesome efficiency is..."interesting". And saying that "being fast is not the goal" doesn't debunk the result or make its methodology flawed. Quite the contrary, it raises a good discussion about what those goals may be and clears up misconceptions, because apparently many people do believe it is faster (and for many async-style APIs those claims are either made or at least strongly hinted at and not disavowed. Looking at you, GCD/libdispatch).
  - Uberphallus 5y ago
    > It was slower with the fewer threads, and giving it more threads made it even slower. Celebrating this as awesome efficiency is..."interesting". With 1 to 4 process increase in 4 cores there's basically no granularity, at 3 there's not enough concurrency, and at 4 there's too much, choking the PostgreSQL process. Whereas 1 to 16 blocking process increase allow for more granularity ramp up and leave the (almost) exact room for the PSQL process to do its thing before performance degrades. That would be solved having the DB on a separate host to isolate this effect, and what happens in literally any non-dev environment. And that's leaving asid the fact that 16 processes use roughly 4 times as much memory as 4, meaning async is cheaper to scale horizontally overall.
jokull 5y ago
Was already posted and discussed before: https://news.ycombinator.com/item?id=23496994 https://news.ycombinator.com/item?id=23496994
Grimm1 5y ago
Every time this comes up (like every 3 months on here) people just kind of digest it as fact without critically reviewing this. There are multiple posts in the wild that dispute Cal's claims and raise various issues with his benchmarks. Like always consider your work load, does it need a level of optimization? Yes? Then compare various performance options.
huntertwo 5y ago
Every request is expected to be completed in 10ms-20ms because of the nature of this benchmark test. In these cases it is unlikely you're going to beat the scheduling overhead to make async worth it. However, if 10% of your requests take 10s, your 16 workers are very soon going to become 16 stuck workers and you won't be able to fulfill any new requests. This is the problem that async solves. You shouldn't use a fire hose to water your plants but you also shouldn't use a sprinkler system to put out a fire.
- omega3 5y ago
  You might want to use a different analogy: https://en.wikipedia.org/wiki/Fire_sprinkler_system https://en.wikipedia.org/wiki/Fire_sprinkler_system
  - huntertwo 5y ago
    /facepalm
  - remram 5y ago
    It seems exactly backwards
- rebelcoder 5y ago
  This should be obvious to most people agreeing with the article, and I came here to say what you just said. I use async in a webserver environment only, where it does shine, and I use Celery + Rabbit for synchronous tasks (executed asynchronously via webserver). As long as I keep my async code on the webserver end, and my sync code elsewhere, the project seems to stay organized well. Otherwise, I tend to lose sight of proper naming, etc.
- aenis 5y ago
  Very much this. I run prod workloads with async python and indeed when you have some external I/O that can take a long time to complete - async fixes that. One of my systems does around 8 api calls to service a request. It serves 80,000 daily users generating close to 2,000,000 expensive api calls and typically thats served by a single node running on a 2 core cpu. Gunicorn/starlette/fastapi. The real problem with async in python is how easy it is to break it by introducing code or dependency that hogs the cpu every now and then. This usually means debugging weird timeouts that only happen every few days and are super hard to trace. Not sure I'd like to do async python again but it sure is efficient for I/O heavy workloads.
marginalia_nu 5y ago
Every developer that does not know about it will eventually come to discover Amdahl's Law on their own.
deleted 5y ago
[deleted]
umvi 5y ago
Try fetching json from 100 HTTP endpoints with async vs. non-async.
tankenmate 5y ago
I suspect that these benchmarks largely prove that the Linux kernel is better at multi-processing and scheduling than any of the Async Python offerings so far.
davidw 5y ago
Erlang - which does async very well for a lot of tasks - is not particularly fast. What it is good at is juggling a lot of things at the same time in a fairly predictable amount of time.
captainmuon 5y ago
I don't have much experience with modern async python, but I used Twisted coroutines (with `yield` instead of `await`) a lot. I found the greatest benefit was responsiveness. You could write networking code for a GUI in imperative style, and the GUI would still remain responsive. You do have to make sure to not do any big computations, but only IO on the main thread, as async only helps with the latter. Actually, I found that in some places the code wasn't perfectly "async", or there were some hidden CPU intensive parts, so that every now and then the GUI still had responsiveness issues. I then moved all the network code to a separate thread. Then I had two event loops, one from the toolkit (Gtk) on the main thread, and one from Twisted on the background thread. For communication, I had functions to "submit" a function to run on either thread (a form of message passing). Personally, I found this to be the best way to architecture a complex GUI app. For such an app, the most imporant things are that it is responsive and correct, so this works pretty well.
3pt14159 5y ago
I did a project for a major financial company everyone on HN has heard of. I used async python with just a crazy amount of data coming in mostly all at once (think end of market) and then had to pipe it to different places with a huge emphasis on prioritization. It worked really, really, really well. I'm not saying I'd rather use Async over web workers, but when you have the right task for this tool and you architect the solution around it intelligently it really does shine.
BiteCode_dev 5y ago
There are many misconceptions about async python I keep seing again and again. I'd like to offer an alternative point of view: - async python is faster, but for a very specific niche of workload. Most tasks you do don't fit that workload at all. Which mean most of the time, you should NOT use async Python. And it's ok. Don't make your code complicated when you don't need it. Chose a tech for the need, not the hype. - async python is not just for performances. It helps with making some specific kind of concurrency easy to reason about, because the context switch is explicit, and the chain of event can appear linearly in the code, thanks to await. - it's ok to have some sync processes and some async processes. It's not one or the other. - async is not a replacement for threads or processes. In fact, there are good use cases for having several processes, each with several threads, each with one event loop. Which also means that if you benchmark your WSGI code with 16 workers, you should do so as well with your aWSGI one. The corollary to this is that some web site loads are very well suited for async (e.g: an SPA with a lot of connectivity which delegates long running code to other services), but a lot are not. If each request makes a long SQL query, seeks the hard drive, dynamically performs i18n and adds some calculation on top, it may very well block the event loop for too long, killing any benefit. So when would you use async python ? - For performances, when you need to maintain numerous long connections. E.G: doing websockets ? Use asyncio. Serving static files without nginx ? Use asyncio. Want to create a web crawler and your memory budget does not allow to open 10000 threads ? Use asyncio. - For the interface, use the async/await keywords everywhere you need need inversion of control for I/O. It's not just about perf here, it's also a mechanism to delegate arbitrary parts of your code with a common standard interface around a fancy state machine + scheduler. You can use that to abstract all your I/O and switch backends at will, while offering callback inlining. But again, you have awesome threading and multiprocessing pools with python. Not to mention tools like zeromq, scrapy or celery, which do a lot for you. Don't run to async just because "it's faster". Having async by default in frameworks like fastpi opens up a ton of possibility though. Live settings, pub/sub between processes, websockets...
- BMorearty 5y ago
  I can’t tell if your first list is misconceptions or your rebuttals to misconceptions. You said it’s misconceptions but I think it’s your rebuttals.
dehrmann 5y ago
> A bigger worry is that async frameworks go a bit wobbly under load. Async should generally be better for throughput because you can complete work items without interruption, whereas context switching will give all threads a chance to make progress. Async will start to have issues once you do any sort of non-trivial processing in the event loop. The irony is most cases employing async are latency-sensitive, not throughput-sensitive.
ceronman 5y ago
I think the point of Async IO is not to go faster, but to be able to server more requests in a single process. This is only possible when request time is mostly spent waiting for I/O. So if you have some server and you want to handle hundreds of thousands of concurrent requests, and each requests takes a few hundred milliseconds and most of this time is waiting for I/O (e.g, waiting for other micro services to reply), then using Async IO you could do this in one single box. Yo probably won't be able to do that with a typical balanced process pool such ash uwsgi or gunicorn.

89 points by xk3 5y ago

mahoro 5y ago

Django 1.10 from this "benchmark" has been released 5 years ago (the article is from 2020). WAT?

cors-fls 5y ago
Django 1.10 appears in Vibora's benchmark which is merely cited at the end of the article. The benchmark of the author does not test Django.

meitros 5y ago

I understand what the article is saying but there’s still a something I’m missing. All I want (for my admittedly simple service) is to be able to send multiple api calls or database calls in parallel rather than one at a time to reduce overall endpoint latency. I guess I should be using a normal synchronous framework that just calls asyncio.run within each api call?

anonymoushn 5y ago
At an instant messaging company we did this in 2010 with greenlet. In current_year, everything assumes you will use async, so you should probably use async.

rjbwork 5y ago

Does anyone expect async anything to be faster? AFAIU async enables greater scalability by allowing computation to continue in other contexts while awaiting read/write operations - at the cost of slightly lower single context performance.

davidkunz 5y ago
Not sure about Python's async story, but in Node.js it is essential if your program's performance is I/O bound. There's no reason to let a thread stay idle while waiting for a database operation to finish.
- rjbwork 5y ago
  That doesn't make your code faster though. For a particular bit of single threaded computation the async approach is going to be slower due to context switching overhead. Same in C#, Rust, etc. If your performance is I/O bound, then, as I said, it enables greater scalability by allowing computation to continue on other contexts while waiting for that I/O operation.
  - davidkunz 5y ago
    Yes, this is absolutely correct. I don't think anyone would argue that async code makes CPU-bound computations faster, the opposite is the case because you have naturally more overhead.
- frenchyatwork 5y ago
  I don't think you disagreed at all with rjbwork at all, it's just you're using words differently. The article, and rjbwork, use slower to mean "slower when the CPU isn't regularly idling". In this context, Node.js async is slower as well, that's why the library better-sqlite3 for NodeJS is synchronous (because you're rarely going to be using sqlite in a context where the CPU/memory isn't very busy).
  - davidkunz 5y ago
    Yes, I don't disagree with rjbwork. I may have read the article a bit differently, it sounded like "don't bother too much with async, it's slower, even for realistic web servers". In your I/O-bound Node.js server, if you would exchange all async calls with synchronous ones, your performance would get a big hit.
- guywhocodes 5y ago
  Yeah like being nagged on that you should async your fstat calls, which makes them four times slower.
- __turbobrew__ 5y ago
  Doesn’t your thread just get put to sleep by the kernel when it is blocked on I/O? Unless Node busy waits on I/O I don’t see the problem?
dangerwill 5y ago
Yeah, I thought of this as a given. With the real benefit being that your fleet is able to handle a higher and not-hard-limited number of requests in flight, and the trade-off being the increased code complexity
kayodelycaon 5y ago
> Does anyone expect async anything to be faster? Judging from what I've read and seen, yes. Async may scale better, but are you working at scales where that pays off?
cerved 5y ago
Yes. Anything that does a lot of I/O to slow stuff like disk, network is going to benefit. So in practice, lots of things.

VWWHFSfQ 5y ago

Overall I've been extremely unimpressed with Python's async story. They've spent a decade baking into the core of the language and the result is that it's no better than just using gevent or twisted or any of those other "old school" frameworks. Except now everyone is forced to support it because there are colored functions everywhere and you have to write library code that has to deal with all the myriad ways your stuff can be used depending on if you're in an event loop or not. It's baffling. Async/await at the language level is a total scam. Give me Nginx+Lua and let me just write my regular procedural code and the underlying runtime will handle yielding/resuming for me.

anonymoushn 5y ago
It has certainly been depressing, as a greenlet user over a decade ago, to watch people spend weeks rewriting codebases to add "await" in all the right places.
- moffkalast 5y ago
  Name two things that go better together, Python devs and breaking shit that has no reason to be broken.
MrBuddyCasino 5y ago
> async/await at the language level is a total scam This view will magically become mainstream as soon as "having worked with async" stops being a resume inflating selling point. I'm very glad Java is doing the right thing and hiding everything under the same "Thread" abstraction.
rich_sasha 5y ago
Most code seems to take sides: it’s either synchronous or not (which I suppose in itself isn’t great). For many good standard libraries/packages, there is now an async variant. I used asyncio for the first time seriously recently and was pleasantly surprised. But, crucially, I didn’t want throughput, I wanted a low-cpu use app with easy-to-write concurrency. I really hope GIL goes away and we can just use threads.

srcreigh 5y ago

The error(?) in this benchmark is putting the DB on the same machine as the server. Async solves the problem of not wanting to block other requests while you go out to expensive external resources. If you benchmark stuff while it's all on the same machine, it masks that cost. I couldn't find the size of the DB. Is it 100GB of data? or more like 10MB? Doing one point query on 10MB might not touch even disk a whole lot. If you were using srcreigh's hosted DB service and queries took 1s to run, the sync servers throughput would suffer greatly, meanwhile async servers would still perform pretty good.

tempest_ 5y ago
This was what I noticed as well. The minute your db queries become expensive the sync workers end up slowing down and stopping other functions. If your workers are tied up doing expensive queries you might be holding up cheap queries that might be getting data from a cache or whatever.
oefrha 5y ago
> If you were using srcreigh's hosted DB service and queries took 1s to run, your sync server latency would be ~1s :) Or longer if they get queued up. Yep, and your 16-worker gunicorn is going to serve 16 rps. What's shown here (nginx, pg, Python application running on the same low CPU VPS, one extremely fast db query per request) is not a "realistic benchmark".
stickyricky 5y ago
If you're at the point where you've got a single application server with 100 waiting connections on a database, you've already blundered. Firstly, you're either overpaying for your database (connections aren't free) or you're underpaying for application servers. Fixing the problem in either dimension reduces the utility of your async server. There's also this myth that Python code is free if the i/o wait is long enough. That's not true. 1). 100 queued connections on an async python server sucks so much momentum out your application. Especially in a resource constrained cloud environment. 2). Any server awaiting a database is going to transform the data to some response object which is going to wreck your overloaded async server. 3). As far as I know, all the async WSGIs are written in Python and will use WAY more CPU than the C-based sync WSGIs.

stickyricky 5y ago

tl;dr: Async (1) is not faster, (2) does not scale, and (3) is not stable under load. Yes and no matter how many times its said people will still opt for the async option even though its slower in the general case. I went so far as to benchmark this myself in 2019 using EC2, digital ocean, and my local machine. I nearly published a blog post but I don't appreciate the attention that brings. Gunicorn is not stable under heavy load. uWSGI delivers better throughput. If we talk about cloud environments exclusively, uWSGI is better full stop. If you have endpoints with long awaits consider a strategy other than holding open the connection. If you need to run a socket server then run it separately from your api server.

deleted 5y ago

[deleted]

djstein 5y ago

The strangest part of this dataset to me was tests being run using Daphne with Starlette, instead of Django. Unclear who would reach for that combo in production, since Daphne is meant for Django.

sergiomattei 5y ago
If it's ASGI compliant, there's no real difference. Daphne is written to be a reference implementation of ASGI, not as a Django-specific server. If you're using Django Channels in production you might even be better served by trying out Uvicorn, it's a lot faster.

Uberphallus 5y ago

> Benchmark > Uses a VPS Not saying the results are wrong (though for client side code my experience is very different), but that's not really a good start.

oefrha 5y ago

This was already thoroughly discussed at the time: https://news.ycombinator.com/item?id=23496994 https://news.ycombinator.com/item?id=23496994 And a rebuttal from Python core dev @ambv at the time pointing out how the methodology's flawed: https://twitter.com/llanga/status/1271719778324025349 https://twitter.com/llanga/status/1271719778324025349

mpweiher 5y ago
I don't see anything of substance in those tweets. It was slower with the fewer threads, and giving it more threads made it even slower. Celebrating this as awesome efficiency is..."interesting". And saying that "being fast is not the goal" doesn't debunk the result or make its methodology flawed. Quite the contrary, it raises a good discussion about what those goals may be and clears up misconceptions, because apparently many people do believe it is faster (and for many async-style APIs those claims are either made or at least strongly hinted at and not disavowed. Looking at you, GCD/libdispatch).
- Uberphallus 5y ago
  > It was slower with the fewer threads, and giving it more threads made it even slower. Celebrating this as awesome efficiency is..."interesting". With 1 to 4 process increase in 4 cores there's basically no granularity, at 3 there's not enough concurrency, and at 4 there's too much, choking the PostgreSQL process. Whereas 1 to 16 blocking process increase allow for more granularity ramp up and leave the (almost) exact room for the PSQL process to do its thing before performance degrades. That would be solved having the DB on a separate host to isolate this effect, and what happens in literally any non-dev environment. And that's leaving asid the fact that 16 processes use roughly 4 times as much memory as 4, meaning async is cheaper to scale horizontally overall.

jokull 5y ago

Was already posted and discussed before: https://news.ycombinator.com/item?id=23496994 https://news.ycombinator.com/item?id=23496994

Grimm1 5y ago

Every time this comes up (like every 3 months on here) people just kind of digest it as fact without critically reviewing this. There are multiple posts in the wild that dispute Cal's claims and raise various issues with his benchmarks. Like always consider your work load, does it need a level of optimization? Yes? Then compare various performance options.

huntertwo 5y ago

Every request is expected to be completed in 10ms-20ms because of the nature of this benchmark test. In these cases it is unlikely you're going to beat the scheduling overhead to make async worth it. However, if 10% of your requests take 10s, your 16 workers are very soon going to become 16 stuck workers and you won't be able to fulfill any new requests. This is the problem that async solves. You shouldn't use a fire hose to water your plants but you also shouldn't use a sprinkler system to put out a fire.

omega3 5y ago
You might want to use a different analogy: https://en.wikipedia.org/wiki/Fire_sprinkler_system https://en.wikipedia.org/wiki/Fire_sprinkler_system
- huntertwo 5y ago
  /facepalm
- remram 5y ago
  It seems exactly backwards
rebelcoder 5y ago
This should be obvious to most people agreeing with the article, and I came here to say what you just said. I use async in a webserver environment only, where it does shine, and I use Celery + Rabbit for synchronous tasks (executed asynchronously via webserver). As long as I keep my async code on the webserver end, and my sync code elsewhere, the project seems to stay organized well. Otherwise, I tend to lose sight of proper naming, etc.
aenis 5y ago
Very much this. I run prod workloads with async python and indeed when you have some external I/O that can take a long time to complete - async fixes that. One of my systems does around 8 api calls to service a request. It serves 80,000 daily users generating close to 2,000,000 expensive api calls and typically thats served by a single node running on a 2 core cpu. Gunicorn/starlette/fastapi. The real problem with async in python is how easy it is to break it by introducing code or dependency that hogs the cpu every now and then. This usually means debugging weird timeouts that only happen every few days and are super hard to trace. Not sure I'd like to do async python again but it sure is efficient for I/O heavy workloads.

marginalia_nu 5y ago

Every developer that does not know about it will eventually come to discover Amdahl's Law on their own.

deleted 5y ago

[deleted]

umvi 5y ago

Try fetching json from 100 HTTP endpoints with async vs. non-async.

tankenmate 5y ago

I suspect that these benchmarks largely prove that the Linux kernel is better at multi-processing and scheduling than any of the Async Python offerings so far.

davidw 5y ago

Erlang - which does async very well for a lot of tasks - is not particularly fast. What it is good at is juggling a lot of things at the same time in a fairly predictable amount of time.

captainmuon 5y ago

I don't have much experience with modern async python, but I used Twisted coroutines (with `yield` instead of `await`) a lot. I found the greatest benefit was responsiveness. You could write networking code for a GUI in imperative style, and the GUI would still remain responsive. You do have to make sure to not do any big computations, but only IO on the main thread, as async only helps with the latter. Actually, I found that in some places the code wasn't perfectly "async", or there were some hidden CPU intensive parts, so that every now and then the GUI still had responsiveness issues. I then moved all the network code to a separate thread. Then I had two event loops, one from the toolkit (Gtk) on the main thread, and one from Twisted on the background thread. For communication, I had functions to "submit" a function to run on either thread (a form of message passing). Personally, I found this to be the best way to architecture a complex GUI app. For such an app, the most imporant things are that it is responsive and correct, so this works pretty well.

3pt14159 5y ago

I did a project for a major financial company everyone on HN has heard of. I used async python with just a crazy amount of data coming in mostly all at once (think end of market) and then had to pipe it to different places with a huge emphasis on prioritization. It worked really, really, really well. I'm not saying I'd rather use Async over web workers, but when you have the right task for this tool and you architect the solution around it intelligently it really does shine.

BiteCode_dev 5y ago

There are many misconceptions about async python I keep seing again and again. I'd like to offer an alternative point of view: - async python is faster, but for a very specific niche of workload. Most tasks you do don't fit that workload at all. Which mean most of the time, you should NOT use async Python. And it's ok. Don't make your code complicated when you don't need it. Chose a tech for the need, not the hype. - async python is not just for performances. It helps with making some specific kind of concurrency easy to reason about, because the context switch is explicit, and the chain of event can appear linearly in the code, thanks to await. - it's ok to have some sync processes and some async processes. It's not one or the other. - async is not a replacement for threads or processes. In fact, there are good use cases for having several processes, each with several threads, each with one event loop. Which also means that if you benchmark your WSGI code with 16 workers, you should do so as well with your aWSGI one. The corollary to this is that some web site loads are very well suited for async (e.g: an SPA with a lot of connectivity which delegates long running code to other services), but a lot are not. If each request makes a long SQL query, seeks the hard drive, dynamically performs i18n and adds some calculation on top, it may very well block the event loop for too long, killing any benefit. So when would you use async python ? - For performances, when you need to maintain numerous long connections. E.G: doing websockets ? Use asyncio. Serving static files without nginx ? Use asyncio. Want to create a web crawler and your memory budget does not allow to open 10000 threads ? Use asyncio. - For the interface, use the async/await keywords everywhere you need need inversion of control for I/O. It's not just about perf here, it's also a mechanism to delegate arbitrary parts of your code with a common standard interface around a fancy state machine + scheduler. You can use that to abstract all your I/O and switch backends at will, while offering callback inlining. But again, you have awesome threading and multiprocessing pools with python. Not to mention tools like zeromq, scrapy or celery, which do a lot for you. Don't run to async just because "it's faster". Having async by default in frameworks like fastpi opens up a ton of possibility though. Live settings, pub/sub between processes, websockets...

BMorearty 5y ago
I can’t tell if your first list is misconceptions or your rebuttals to misconceptions. You said it’s misconceptions but I think it’s your rebuttals.

dehrmann 5y ago

> A bigger worry is that async frameworks go a bit wobbly under load. Async should generally be better for throughput because you can complete work items without interruption, whereas context switching will give all threads a chance to make progress. Async will start to have issues once you do any sort of non-trivial processing in the event loop. The irony is most cases employing async are latency-sensitive, not throughput-sensitive.

ceronman 5y ago

I think the point of Async IO is not to go faster, but to be able to server more requests in a single process. This is only possible when request time is mostly spent waiting for I/O. So if you have some server and you want to handle hundreds of thousands of concurrent requests, and each requests takes a few hundred milliseconds and most of this time is waiting for I/O (e.g, waiting for other micro services to reply), then using Async IO you could do this in one single box. Yo probably won't be able to do that with a typical balanced process pool such ash uwsgi or gunicorn.