3 ms·

There is very little in everyday Python usage that benefits from Asyncio. Two in webdev, are long running request (Websockets, SSE, long polling), and processin

There is very little in everyday Python usage that benefits from Asyncio. Two in webdev, are long running request (Websockets, SSE, long polling), and processing multiple backend IO processes in parallel. However the later is very rare, you may think you have multiple DB request that could use asyncio, but most of the time they are dependent on each other. Almost all of the time a normal multithreaded Python server is perfectly sufficient and much easer to code. My recommendation is only use it where it is really REALY required.

nicolaslem 4y ago
Agreed, the SaaS Python application I maintain does SSE with threads. It is not the prettiest thing but it works and threads are cheaper than rewriting the whole thing to be async.
- dividuum 4y ago
  I never understood why gevent wasn't integrated more into default Python. I think at the moment it still requires monkey patching, so non-aware functions use the gevent variants. But once you've done that, you can gevent.spawn() a normal python function and it also runs concurrent (though not parallel) and is suspended when blocking I/O happens. No function coloring required. I'm sure there is some reason why that wasn't done. Anyone knows?
kissgyorgy 4y ago
Fully agree with this! Just stick to Flask/Django if you doing a simple website/API. Async also makes DB connections slower, here is a great piece from Mike Bayer from years ago: https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a...
usrbinbash 4y ago
Completely agree with this. I have several python services running as glue between some of our components. They all use threading. I find it alot easier to reason about.
jsmith45 4y ago
Sure. As a general rule the general style of coding using in async-await approaches is primarily about one of two things The first purpose is allowing more throughput at the expense of per request latency (Typically each request will take longer than with equivalent sync code). The main scenario where an async version could potentially complete sooner than a sync version is when the the code is able to start multiple async tasks and then await then as a group. For example if your task needs to make 10 http requests, and make those requests sequentially like one would in sync code, it will be slower. If one starts all ten calls and then awaits the results, then you might be able to a speedup on this overall request. Other main purpose is when working with a UI framework where there is a main thread, and certain operations can only occur on the main thread. Use of async/await pattern helps avoid accidentally blocking the main thread, which can kill application responsiveness. This is why the pattern is used in javascript, and was one of the headline scenarios when C# first introduced this pattern. (The alternative being other methods of asynchrony which typically include use of callbacks, which can make the code harder to develop or understand). But basically, unless you have UI blocking problems, or are concerned about the number of requests per second you can handle, async-await patterns may be better avoided. It being even more costly in python than it is in some other languages does not really help.
jacob019 4y ago
I don't know what you use Python for every day. Sure for some utility scripts it doesn't matter. I use it to run my ecommerce business, and for a variety of other plumbing, mostly for passing messages around between users, APIs, databases, printers, etc. Async programming is a must for just about everything. I guess the alternative would be thread pools, or process pools, like back in the day; but that has a lot of downsides. It is slower, way more resource intensive, and state sharing/synchronization becomes a major issue, you can only handle thread number of tasks concurrently and you're going to use that memory all the time; not to mention all the code needs to be thread safe. Most of our systems use gevent, but we've starting using asyncio for new projects.
- samwillis 4y ago
  Same as you, e-commerce store, along with other mostly web projects, quite a bit of real-time and long running requests. I also use Gevent extensively (I prefer it to asyncio). We use Gevent or Asyncio in a small number of routes that either have long running requests, SSE in our case, or have a very large number of rear facing io requests we can parallelise (a few back office screens and processes). The complexity that asyncio would add to the the code for the 95% of routs that don't need it would add a lot of unnecessary overhead to development. My point is, I don't believe it adds any value 95% of the time, but does, sometimes significantly, 5% of the time. I think it's better to only use it where it's really needed and stick to old fashioned, none concurrent, code everywhere else.
  - jacob019 4y ago
    Ok, I can agree with that. Gevent doesn't really add any cognitive overhead or complexity at all. Asyncio sure does, and makes the code hard to read too. Long ago I settled on a stack that includes using gevent for anything that listens on a socket, never regretted it.

nicolaslem 4y ago

Agreed, the SaaS Python application I maintain does SSE with threads. It is not the prettiest thing but it works and threads are cheaper than rewriting the whole thing to be async.

dividuum 4y ago
I never understood why gevent wasn't integrated more into default Python. I think at the moment it still requires monkey patching, so non-aware functions use the gevent variants. But once you've done that, you can gevent.spawn() a normal python function and it also runs concurrent (though not parallel) and is suspended when blocking I/O happens. No function coloring required. I'm sure there is some reason why that wasn't done. Anyone knows?

kissgyorgy 4y ago

Fully agree with this! Just stick to Flask/Django if you doing a simple website/API. Async also makes DB connections slower, here is a great piece from Mike Bayer from years ago: https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a...

usrbinbash 4y ago

Completely agree with this. I have several python services running as glue between some of our components. They all use threading. I find it alot easier to reason about.

jsmith45 4y ago

Sure. As a general rule the general style of coding using in async-await approaches is primarily about one of two things The first purpose is allowing more throughput at the expense of per request latency (Typically each request will take longer than with equivalent sync code). The main scenario where an async version could potentially complete sooner than a sync version is when the the code is able to start multiple async tasks and then await then as a group. For example if your task needs to make 10 http requests, and make those requests sequentially like one would in sync code, it will be slower. If one starts all ten calls and then awaits the results, then you might be able to a speedup on this overall request. Other main purpose is when working with a UI framework where there is a main thread, and certain operations can only occur on the main thread. Use of async/await pattern helps avoid accidentally blocking the main thread, which can kill application responsiveness. This is why the pattern is used in javascript, and was one of the headline scenarios when C# first introduced this pattern. (The alternative being other methods of asynchrony which typically include use of callbacks, which can make the code harder to develop or understand). But basically, unless you have UI blocking problems, or are concerned about the number of requests per second you can handle, async-await patterns may be better avoided. It being even more costly in python than it is in some other languages does not really help.

jacob019 4y ago

I don't know what you use Python for every day. Sure for some utility scripts it doesn't matter. I use it to run my ecommerce business, and for a variety of other plumbing, mostly for passing messages around between users, APIs, databases, printers, etc. Async programming is a must for just about everything. I guess the alternative would be thread pools, or process pools, like back in the day; but that has a lot of downsides. It is slower, way more resource intensive, and state sharing/synchronization becomes a major issue, you can only handle thread number of tasks concurrently and you're going to use that memory all the time; not to mention all the code needs to be thread safe. Most of our systems use gevent, but we've starting using asyncio for new projects.

samwillis 4y ago
Same as you, e-commerce store, along with other mostly web projects, quite a bit of real-time and long running requests. I also use Gevent extensively (I prefer it to asyncio). We use Gevent or Asyncio in a small number of routes that either have long running requests, SSE in our case, or have a very large number of rear facing io requests we can parallelise (a few back office screens and processes). The complexity that asyncio would add to the the code for the 95% of routs that don't need it would add a lot of unnecessary overhead to development. My point is, I don't believe it adds any value 95% of the time, but does, sometimes significantly, 5% of the time. I think it's better to only use it where it's really needed and stick to old fashioned, none concurrent, code everywhere else.
- jacob019 4y ago
  Ok, I can agree with that. Gevent doesn't really add any cognitive overhead or complexity at all. Asyncio sure does, and makes the code hard to read too. Long ago I settled on a stack that includes using gevent for anything that listens on a socket, never regretted it.