4 ms·

This is one of many reasons I'm sceptical of the current trend in Python to "async all the things". The nuance to how it operates is often opaque to the develop

This is one of many reasons I'm sceptical of the current trend in Python to "async all the things". The nuance to how it operates is often opaque to the developer, particularly those less experienced. GUI toolkits (like Textual) however are a really good use case for Asyncio. Human interaction with a program is inherently asynchronous, using async/await so that you can more cleanly specify your control flow is so much better than complicated callbacks. Using async/await in front end JS code for example is a delight. Where I'm particularly unconvinced of their use is in server side view and api end point processing. The majority of the time you have maybe a couple of IO opps that depend on each other. There is often little than can be parallelised (within a request) and so there are few performance gains to be a made. Traditional synchronous imperative code run with a multithreaded server is proven, scalable and much easier to debug. There are always places where it's useful though, things such as long running requests (websockets, long polling), or those very rare occurrences where you do have many easily parallelizable IO opps within one short request.

michael_j_x 4y ago
I am not sure I agree that the GUI is a good use case for async. A human interaction with the program must almost always pre-empt whatever the program was running, so I can not see how a cooperative multi-threading runtime like async Python can work in such a scenario.
- harpiaharpyja 4y ago
  Well it does work so "it can't work" isn't a very substantial criticism. Is there some nuance or detail that you meant to include?
heavyset_go 4y ago
> Where I'm particularly unconvinced of their use is in server side view and api end point processing. The majority of the time you have maybe a couple of IO opps that depend on each other. There is often little than can be parallelised (within a request) and so there are few performance gains to be a made. Traditional synchronous imperative code run with a multithreaded server is proven, scalable and much easier to debug. Traditional synchronous imperative code run with a multithreaded server is proven, scalable and much easier to debug. Python doesn't have multithreading that scales or supports real parallelism. asyncio has very measurable performance benefits for exactly that use case you've mentioned versus threaded servers.
- zzzeek 4y ago
  Sorry that's not accurate. Asyncio and threading offer the same variety of "parallelism" , which is that both can wait on multiple io streams at once (the gil is released waiting on io). Neither offer CPU parallelism, unless lots of your CPU work is in native extensions that release the gil. In that unusual case, threading would offer parallelism where asyncio wouldn't. Asyncio's single advantage is you can wait on lots of io streams, like many thousands, very cheaply without having to roll non blocking IO queueing code directly.
  - heavyset_go 4y ago
    I didn't say that asyncio offered parallelism, I'm pointing out that normal assumptions about multithreading you'd make with other languages don't always apply to Python. You'd typically assume that threads offer parallelism, a property you might choose to use them for over something like single-threaded asyncio. I've found that for even IO bound workloads, the amount of throughput plateaus when using a relatively small amount of threads despite the GIL being released on IO.
    - zzzeek 4y ago
      sorry for misreading that from your post! my own benching with threads vs. asyncio has never found any performance difference between the two approaches (asyncio slightly slower). if you need very wide throughput, then yes asyncio is better. otherwise, it's very difficult to create equitable comparisons between threaded and asyncio code.
      - heavyset_go 4y ago
        Haha, it's cool. I once had a problem where I needed to scrape a bunch of sites at once, and ended up using it as a benchmark for threads vs asyncio. The overhead with threads was high even when using a handful of threads, and quickly hit a plateau after scaling to 4-8 threads on a 16 core machine. I was a big fan of the concurrent.futures executor model and generally thought it was good performance anyway, but figured I'd try to implement it in asyncio, and rate limited both for a fairer comparison. It was so shockingly fast that it was the last project I reached for threads again with on Python. I can't say that it was an equitable comparison, but it certainly left an impression on me.
Topgamer7 4y ago
These days with graphql, or complex microservices architectures, you could have multiple hops to fulfil l the original request. Flask sync will hold that thread hostage until the request is done. Where async with properly used async libs will allow other requests to process. We often have medium sized reports take seconds. That is a lot of time to wait. And would just end up bloating your service scaling to handle more connections. Any service with decently long lived network requests will benefit from event loop handled scheduling.
- jsyolo 4y ago
  I don't think he was referring to python when he mentioned server-side, but to real multi-threaded runtimes.
nbadg 4y ago
The thing is, there's a lot more nuance to it than this. Async/await is part of the language syntax in python, but asyncio is only one particular implementation of an event loop framework to power it. But really what async/await provides is a general-purpose cooperative multitasking syntax. This allows other libraries to implement their own event loop frameworks, each with their own different semantics and considerations (the two best-known alternatives being Curio and Trio). At a language level, there's nothing even forcing you to use async/await for ascync IO -- you could, if you really wanted, probably write a library that used it to start threads and await their completion. So you have, from highest-level to lowest-level: application code, async/await language syntax, the event loop framework, and then the implementation of the event loop itself. The OP article concerns a peculiar implementation detail in the lowest level that makes it very easy to write bugs at the highest level. But that means that even if you do "async all the things", you'll only encounter this situation if you write your application code in a particular way. It just so happens that "in a particular way" is, in this case, the overwhelming majority of how people write it, which is, of course, why the OP article is relevant.
- heavyset_go 4y ago
  > The OP article concerns a peculiar implementation detail in the lowest level that makes it very easy to write bugs at the highest level. Are other async implementations using the asyncio.Task abstraction? I haven't looked into it, but I assumed that asyncio.Task was tied to the asyncio implementation and event loop.
  - nbadg 4y ago
    asyncio.Task is part of the asyncio event loop framework. So any event loop implementation that conforms to that framework will have to have one (including the default event loop implementation that ships with asyncio). So for example, uvloop, which is an alternative event loop implementation that works with the asyncio event loop framework, also uses asyncio Tasks. Other event loop frameworks can do whatever they want, and presumably, wouldn't be importing from asyncio. Whether they have a similar abstraction is completely up to the framework itself. Trio, for example, doesn't have a concept of a task object at all, because it enforces a strict tree structure for tasks.
pdonis 4y ago
> GUI toolkits (like Textual) however are a really good use case for Asyncio. Only if the GUI toolkit is explicitly written to be asyncio-aware and use asyncio's event loop. Textual appears to be written specifically to do that. However, other GUI toolkits that I'm aware of that have Python bindings aren't written that way. Qt, for example, uses its own event loop, and if you want anything other than a GUI event to be fed into Qt's event loop so your event-driven code can process it, you have to do that by hand and make sure it works. There is no point in even trying to use another event loop, such as Python's asyncio event loop, since that loop will never run while Qt's event loop is running.
- harpiaharpyja 4y ago
  There's a qasync library that I've used to bridge asyncio and Qt, it works pretty well. But yes, you need some extra support.
samsquire 4y ago
I am a huge fan of parallel and async code. I spend a lot of time researching it and trying to design software that is easily parallelisable. Many GUIs use the event/message pump pattern, such as Windows 32 API. Qt does something with its event loop (QEventLoop) Threads are a rather low level instrument to get background tasks going because the interface between the main thread and the threads is rather omitted. In Java you could use a ConcurrentLinkedQueue. And in Python you can use JoinableQueue. I am heavily interested in this space because I want to write understandable software that anybody can pick up and work with. I worked on a JMS log viewer that used threads but would crash with ConcurrentModificationException due to not being thread safe. I changed it to be thread safe but its performance dropped through the floor. In my learnings since then I should hast sharded each JMS connection topic to its own thread or multiplexed multiple JMS topics per thread and loop over them. The main thread can interrogate the thread with a lock, that should be faster than every thread trying to acquire the lock. It would be driven by the main thread but the work is done in the background. The threads can keep the fetched messages in memory until the main thread is ready for them. I think with the right abstraction, thread safety can be achieved and concurrency shouldn't be something to be afraid of. It is very difficult and challenging working at the low levels of concurrency such as a concurrent browser engine. (I've not done that though.) This is why languages such as Pony lang, Inko, Cyber and Erlang, Elixir are so promising. We can build high performance systems that parallelise. Writing an async/await pipeline that looks synchronous is far easier to understand and maintain than nested callbacks. So I can see where async is useful. I just hope we can design async software to be simpler to maintain and extend.
whoopdeepoo 4y ago
I don't write any colored function code in python, I'd much rather work with process/thread pools
- Animats 4y ago
  Me too, but threading is botched in Python. Not just the Global Interpreter Lock. Some Python packages are not thread-safe, and it's not documented which ones are not. Years ago I discovered that CPickle was not thread safe, and that wasn't considered a problem.
  - jsyolo 4y ago
    Since the GIL is there and it restricts multithreaded issues to happen, why would python packages be thread-safe?
    - Jorge1o1 4y ago
      You can still have thread safety issues with the GIL in place, because globals and other data is shared between threads. For example, you can put a dictionary at the module level, thread A can set a key in that dictionary like “name”, thread B can overwrite it, and then thread A comes back, does dct[“name”] and gets an unexpected answer. This is a relatively easy mistake to make, a lot of python code has module level variables.
- hgomersall 4y ago
  I really don't get the whole coloured function thing. How's it not just the function signature? You might as well claim a new function argument makes a new colour. Granted, all of my use of async is in Rust in which the compiler picks this stuff up, so maybe in python there are other concerns I'm missing.
  - harpiaharpyja 4y ago
    Honestly, I think the whole colored function thing is BS. I've never run into an issue with it.
traverseda 4y ago
>Where I'm particularly unconvinced of their use is in server side view and api end point processing. Sure, performance isn't going to get better, but for websockets and server sent events the occasional long-lived async task can be great. Especially when you need to poll something, or check in on a subprocess.

michael_j_x 4y ago

I am not sure I agree that the GUI is a good use case for async. A human interaction with the program must almost always pre-empt whatever the program was running, so I can not see how a cooperative multi-threading runtime like async Python can work in such a scenario.

harpiaharpyja 4y ago
Well it does work so "it can't work" isn't a very substantial criticism. Is there some nuance or detail that you meant to include?

heavyset_go 4y ago

> Where I'm particularly unconvinced of their use is in server side view and api end point processing. The majority of the time you have maybe a couple of IO opps that depend on each other. There is often little than can be parallelised (within a request) and so there are few performance gains to be a made. Traditional synchronous imperative code run with a multithreaded server is proven, scalable and much easier to debug. Traditional synchronous imperative code run with a multithreaded server is proven, scalable and much easier to debug. Python doesn't have multithreading that scales or supports real parallelism. asyncio has very measurable performance benefits for exactly that use case you've mentioned versus threaded servers.

zzzeek 4y ago
Sorry that's not accurate. Asyncio and threading offer the same variety of "parallelism" , which is that both can wait on multiple io streams at once (the gil is released waiting on io). Neither offer CPU parallelism, unless lots of your CPU work is in native extensions that release the gil. In that unusual case, threading would offer parallelism where asyncio wouldn't. Asyncio's single advantage is you can wait on lots of io streams, like many thousands, very cheaply without having to roll non blocking IO queueing code directly.
- heavyset_go 4y ago
  I didn't say that asyncio offered parallelism, I'm pointing out that normal assumptions about multithreading you'd make with other languages don't always apply to Python. You'd typically assume that threads offer parallelism, a property you might choose to use them for over something like single-threaded asyncio. I've found that for even IO bound workloads, the amount of throughput plateaus when using a relatively small amount of threads despite the GIL being released on IO.
  - zzzeek 4y ago
    sorry for misreading that from your post! my own benching with threads vs. asyncio has never found any performance difference between the two approaches (asyncio slightly slower). if you need very wide throughput, then yes asyncio is better. otherwise, it's very difficult to create equitable comparisons between threaded and asyncio code.
    - heavyset_go 4y ago
      Haha, it's cool. I once had a problem where I needed to scrape a bunch of sites at once, and ended up using it as a benchmark for threads vs asyncio. The overhead with threads was high even when using a handful of threads, and quickly hit a plateau after scaling to 4-8 threads on a 16 core machine. I was a big fan of the concurrent.futures executor model and generally thought it was good performance anyway, but figured I'd try to implement it in asyncio, and rate limited both for a fairer comparison. It was so shockingly fast that it was the last project I reached for threads again with on Python. I can't say that it was an equitable comparison, but it certainly left an impression on me.

Topgamer7 4y ago

These days with graphql, or complex microservices architectures, you could have multiple hops to fulfil l the original request. Flask sync will hold that thread hostage until the request is done. Where async with properly used async libs will allow other requests to process. We often have medium sized reports take seconds. That is a lot of time to wait. And would just end up bloating your service scaling to handle more connections. Any service with decently long lived network requests will benefit from event loop handled scheduling.

jsyolo 4y ago
I don't think he was referring to python when he mentioned server-side, but to real multi-threaded runtimes.

nbadg 4y ago

The thing is, there's a lot more nuance to it than this. Async/await is part of the language syntax in python, but asyncio is only one particular implementation of an event loop framework to power it. But really what async/await provides is a general-purpose cooperative multitasking syntax. This allows other libraries to implement their own event loop frameworks, each with their own different semantics and considerations (the two best-known alternatives being Curio and Trio). At a language level, there's nothing even forcing you to use async/await for ascync IO -- you could, if you really wanted, probably write a library that used it to start threads and await their completion. So you have, from highest-level to lowest-level: application code, async/await language syntax, the event loop framework, and then the implementation of the event loop itself. The OP article concerns a peculiar implementation detail in the lowest level that makes it very easy to write bugs at the highest level. But that means that even if you do "async all the things", you'll only encounter this situation if you write your application code in a particular way. It just so happens that "in a particular way" is, in this case, the overwhelming majority of how people write it, which is, of course, why the OP article is relevant.

heavyset_go 4y ago
> The OP article concerns a peculiar implementation detail in the lowest level that makes it very easy to write bugs at the highest level. Are other async implementations using the asyncio.Task abstraction? I haven't looked into it, but I assumed that asyncio.Task was tied to the asyncio implementation and event loop.
- nbadg 4y ago
  asyncio.Task is part of the asyncio event loop framework. So any event loop implementation that conforms to that framework will have to have one (including the default event loop implementation that ships with asyncio). So for example, uvloop, which is an alternative event loop implementation that works with the asyncio event loop framework, also uses asyncio Tasks. Other event loop frameworks can do whatever they want, and presumably, wouldn't be importing from asyncio. Whether they have a similar abstraction is completely up to the framework itself. Trio, for example, doesn't have a concept of a task object at all, because it enforces a strict tree structure for tasks.

pdonis 4y ago

> GUI toolkits (like Textual) however are a really good use case for Asyncio. Only if the GUI toolkit is explicitly written to be asyncio-aware and use asyncio's event loop. Textual appears to be written specifically to do that. However, other GUI toolkits that I'm aware of that have Python bindings aren't written that way. Qt, for example, uses its own event loop, and if you want anything other than a GUI event to be fed into Qt's event loop so your event-driven code can process it, you have to do that by hand and make sure it works. There is no point in even trying to use another event loop, such as Python's asyncio event loop, since that loop will never run while Qt's event loop is running.

harpiaharpyja 4y ago
There's a qasync library that I've used to bridge asyncio and Qt, it works pretty well. But yes, you need some extra support.

samsquire 4y ago

I am a huge fan of parallel and async code. I spend a lot of time researching it and trying to design software that is easily parallelisable. Many GUIs use the event/message pump pattern, such as Windows 32 API. Qt does something with its event loop (QEventLoop) Threads are a rather low level instrument to get background tasks going because the interface between the main thread and the threads is rather omitted. In Java you could use a ConcurrentLinkedQueue. And in Python you can use JoinableQueue. I am heavily interested in this space because I want to write understandable software that anybody can pick up and work with. I worked on a JMS log viewer that used threads but would crash with ConcurrentModificationException due to not being thread safe. I changed it to be thread safe but its performance dropped through the floor. In my learnings since then I should hast sharded each JMS connection topic to its own thread or multiplexed multiple JMS topics per thread and loop over them. The main thread can interrogate the thread with a lock, that should be faster than every thread trying to acquire the lock. It would be driven by the main thread but the work is done in the background. The threads can keep the fetched messages in memory until the main thread is ready for them. I think with the right abstraction, thread safety can be achieved and concurrency shouldn't be something to be afraid of. It is very difficult and challenging working at the low levels of concurrency such as a concurrent browser engine. (I've not done that though.) This is why languages such as Pony lang, Inko, Cyber and Erlang, Elixir are so promising. We can build high performance systems that parallelise. Writing an async/await pipeline that looks synchronous is far easier to understand and maintain than nested callbacks. So I can see where async is useful. I just hope we can design async software to be simpler to maintain and extend.

whoopdeepoo 4y ago

I don't write any colored function code in python, I'd much rather work with process/thread pools

Animats 4y ago
Me too, but threading is botched in Python. Not just the Global Interpreter Lock. Some Python packages are not thread-safe, and it's not documented which ones are not. Years ago I discovered that CPickle was not thread safe, and that wasn't considered a problem.
- jsyolo 4y ago
  Since the GIL is there and it restricts multithreaded issues to happen, why would python packages be thread-safe?
  - Jorge1o1 4y ago
    You can still have thread safety issues with the GIL in place, because globals and other data is shared between threads. For example, you can put a dictionary at the module level, thread A can set a key in that dictionary like “name”, thread B can overwrite it, and then thread A comes back, does dct[“name”] and gets an unexpected answer. This is a relatively easy mistake to make, a lot of python code has module level variables.
hgomersall 4y ago
I really don't get the whole coloured function thing. How's it not just the function signature? You might as well claim a new function argument makes a new colour. Granted, all of my use of async is in Rust in which the compiler picks this stuff up, so maybe in python there are other concerns I'm missing.
- harpiaharpyja 4y ago
  Honestly, I think the whole colored function thing is BS. I've never run into an issue with it.

traverseda 4y ago

>Where I'm particularly unconvinced of their use is in server side view and api end point processing. Sure, performance isn't going to get better, but for websockets and server sent events the occasional long-lived async task can be great. Especially when you need to poll something, or check in on a subprocess.