3 ms·

Async Python is still confusing af - when do I need it, what happens under the hood, does it actually help with performance, sometimes the GIL comes into play a

by jstx1 4y ago

Async Python is still confusing af - when do I need it, what happens under the hood, does it actually help with performance, sometimes the GIL comes into play and sometimes it doesn't, why do we ever use threads at all if there's a GIL, why is it called asyncio if we can use it for anything. My mind is kind of scattered and people seems to be using a lot of async Python for some reason. Any good resources to clear things up?

bombolo 4y ago
It's basically a fancy way to do epoll() around file descriptors, but hide the need to keep a main loop and state, and keep it hidden as functions that run in fake concurrency, stopping whenever they block, and being executed again when their file descriptor has activity. It doesn't necessarily improve performance. It's just a much easier way to do non-blocking I/O (note that blocking and threaded I/O is easier to do but much much heavier).
323 4y ago
It makes concurrent programming much simpler than using threads. Very few locking and care is needed with asyncio, as opposed to using threads. Race conditions are basically not a thing if you write reasonably idiomatic code. It might be (or not) faster than using threads, but that's not the main benefit in my view - this easiness of use is.
philote 4y ago
It's great for when you can do concurrent I/O tasks. For example running a web backend, web scraping, or many API calls. FastAPI (and Starlette that it's built off of) is an async web framework and in my experience performs well. Basically, your program will normally halt when doing I/O, and won't proceed until that I/O is done. During that halt, your program is doing nothing (no cpu being used). With asyncio, you can schedule multiple tasks to run, so if one is halted doing I/O, another can run. Edit: And AFAIK, the GIL does not come into play at all with async. Only when multithreading.
- morelisp 4y ago
  > the GIL does not come into play at all with async. In the sense that the GIL is still held and you can have at most one path of Python code executing at a time regardless of whether you use async or threads, sure. Most blocking I/O was already releasing the GIL so the difference is purely in how you can design your modules; for any reasoning about performance the GIL behaves the same way whether you use asyncio or not.
uniqueuid 4y ago
I agree that especially within the standard context of python and its syntax, async seems weird (because it's sprinkled into an existing paradigm). The best mental model for me always was to think: Here's an await, that means "interpreter, go and do something else that's currently waiting while this is not done yet." And that's all about IO, because what you can wait on is essentially IO. By the way, I really wish there was a better story to executors in async python. To think we still have the same queue/pickle-based multiprocessing to send async tasks to another core is kind of sad. Hoping for 3.12 and beyond there. [edit] one really neat example that helped me get asyncio was Guido van Rossum's crawler in 500 lines of python [1]. A lot of the syntax is deprecated now, but it's still a great walk-through [1] http://aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html http://aosabook.org/en/500L/a-web-crawler-with-asyncio-corou...
scrlk 4y ago
I've found SuperFastPython to be helpful for understanding Python concurrency: https://superfastpython.com/python-concurrency-choose-api/ https://superfastpython.com/python-concurrency-choose-api/
ActorNightly 4y ago
Async looks like parallelism, but its just smart scheduling. The core concept of async is saying "hey, im waiting for something to complete thats not under my control (like waiting for data on a socket to be able to be read), go ahead and do other things in the mean time". If you ever coded in sockets in C (and its a good exercise to do so), you probably have at some point ran across `select` which is essentially a non blocking way to check which sockets have data available to read, and then sequentially read the data. This gives the ability for a program to appear parallelized in the sense that it can handle multiple client connections, but its not truly parallel. Different clients can be handled at different time depending on which order they connect, which is asynchronous in nature (versus processing each client in sequence and waiting on each one to connect and disconnect before moving on to the next one) Async in Python is basically this concept, with a core fundamental feature of time limited execution. Functions can say that they are pausing for x seconds, allowing other functions to run, or functions can say that they give a certain function x seconds to run before resuming execution. If you async code (along with any library you may use) doesn't contain any sleeps or timeouts, its exactly equivalent to synchronous code (since the event loop never really recieves a message that it can suspend a routine or cancel it). With sleeps and timeouts, you gain control over things that can potentially block, both from a caller perspective of not having a function call block your own, and from a callee perspective of not making your function blocking. The use case is for it is that it is good for I/O bound operations like Threading is, but with the addition that you don't have to worry about synchronization or race conditions, since by design your code will have predictable access patterns. The downside is that your code and any libraries that you use within your code has to be implemented as async libraries, and any library that is async has to have async wrappers around the calls to its methods, which in turn means that your entire code has to be async. Threading with Python is generally not useful, as its not true parallelism because of GIL. GIL allows only one thread in Python to run. Threading is safer in Python because of this, however it obviously has drawbacks. In general its best used if you want asyncio like performance with a library that is not written with async, since GIL is smart enough to detect when a thread is waiting for input and switch context. True parallelism in Python is achieved with multiprocessing, however the use case is a little different. Rather than spinning off processes, you generally launch a bunch of worker processes up front (to avoid the larger overhead), then use smart scheduling to distribute work between these processes. Here though you do have to worry about race conditions and synchronization, and use things like locks and mutexes.
x-complexity 4y ago
> Async Python is still confusing af - when do I need it, It's needed when you're spending a lot of time waiting for an I/O request to complete (network/HTTP requests, disk reads/writes, database reads/writes) > what happens under the hood, https://tenthousandmeters.com/blog/python-behind-the-scenes-12-how-asyncawait-works-in-python/#:~:text=likely%20to%20use.-,asyncio,-asyncio%20came%20to https://tenthousandmeters.com/blog/python-behind-the-scenes-... (Please read the entire blog post - It goes through the necessary concepts like generators, event loops, & coroutines) > does it actually help with performance, Refer to the first answer: You'll see improved performances if your workloads are mainly comprised of waiting for other stuff to complete. If you're compute-heavy, it'll be better to use the 'multiprocessing' library instead. > sometimes the GIL comes into play and sometimes it doesn't, The GIL comes into play when you have a lot of compute-heavy tasks: Otherwise, you'll rarely encounter it. It's only when you have that many compute-heavy tasks that you start to use the 'multiprocessing' library. > why do we ever use threads at all if there's a GIL, Threads exist because it was there before asyncio & event loops came into Python. > why is it called asyncio if we can use it for anything. Its name came from PEP 3156, proposing the asyncio library back in 2012. https://peps.python.org/pep-3156/ https://peps.python.org/pep-3156/ As for why, asynchronous I/O stands in contrast to synchronous I/O, where the program/thread had to wait for the I/O request to complete before it can do anything else. Making tasks asynchronous allows it to do other stuff while it waits for a task's request to complete, increasing CPU & I/O utilization. > My mind is kind of scattered and people seems to be using a lot of async Python for some reason. Any good resources to clear things up? Highly recommend this video from mcoding: It's fairly simple & goes through a sample implementation. https://www.youtube.com/watch?v=ftmdDlwMwwQ https://www.youtube.com/watch?v=ftmdDlwMwwQ Also, this article: https://realpython.com/async-io-python/ https://realpython.com/async-io-python/
franga2000 4y ago
When working in a fully async context, it becomes very logical and natural. A great example of this is the FastAPI web framework. You never write any top-level code, only functions that the framework calls, so you never have to deal with the event loop directly. You basically just sprinkle some async and await keywords around your IO bottlenecks and things suddenly run smoother.
loa_in_ 4y ago
In personal experience, when you write anything more complicated using threads in python, I end up writing lots of boilerplate code that I wished I could just shove somewhere - Events, 1 element Queues, ContextVar contexts, all of which don't benefit for being named and make appearance only in two spots in code. Async removed a lot of this for me while also unifying the future ecosystem and making my code more composable and easy to integrate.

by jstx1 4y ago

bombolo 4y ago

It's basically a fancy way to do epoll() around file descriptors, but hide the need to keep a main loop and state, and keep it hidden as functions that run in fake concurrency, stopping whenever they block, and being executed again when their file descriptor has activity. It doesn't necessarily improve performance. It's just a much easier way to do non-blocking I/O (note that blocking and threaded I/O is easier to do but much much heavier).

323 4y ago

It makes concurrent programming much simpler than using threads. Very few locking and care is needed with asyncio, as opposed to using threads. Race conditions are basically not a thing if you write reasonably idiomatic code. It might be (or not) faster than using threads, but that's not the main benefit in my view - this easiness of use is.

philote 4y ago

It's great for when you can do concurrent I/O tasks. For example running a web backend, web scraping, or many API calls. FastAPI (and Starlette that it's built off of) is an async web framework and in my experience performs well. Basically, your program will normally halt when doing I/O, and won't proceed until that I/O is done. During that halt, your program is doing nothing (no cpu being used). With asyncio, you can schedule multiple tasks to run, so if one is halted doing I/O, another can run. Edit: And AFAIK, the GIL does not come into play at all with async. Only when multithreading.

morelisp 4y ago
> the GIL does not come into play at all with async. In the sense that the GIL is still held and you can have at most one path of Python code executing at a time regardless of whether you use async or threads, sure. Most blocking I/O was already releasing the GIL so the difference is purely in how you can design your modules; for any reasoning about performance the GIL behaves the same way whether you use asyncio or not.

uniqueuid 4y ago

I agree that especially within the standard context of python and its syntax, async seems weird (because it's sprinkled into an existing paradigm). The best mental model for me always was to think: Here's an await, that means "interpreter, go and do something else that's currently waiting while this is not done yet." And that's all about IO, because what you can wait on is essentially IO. By the way, I really wish there was a better story to executors in async python. To think we still have the same queue/pickle-based multiprocessing to send async tasks to another core is kind of sad. Hoping for 3.12 and beyond there. [edit] one really neat example that helped me get asyncio was Guido van Rossum's crawler in 500 lines of python [1]. A lot of the syntax is deprecated now, but it's still a great walk-through [1] http://aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html http://aosabook.org/en/500L/a-web-crawler-with-asyncio-corou...

scrlk 4y ago

I've found SuperFastPython to be helpful for understanding Python concurrency: https://superfastpython.com/python-concurrency-choose-api/ https://superfastpython.com/python-concurrency-choose-api/

ActorNightly 4y ago

Async looks like parallelism, but its just smart scheduling. The core concept of async is saying "hey, im waiting for something to complete thats not under my control (like waiting for data on a socket to be able to be read), go ahead and do other things in the mean time". If you ever coded in sockets in C (and its a good exercise to do so), you probably have at some point ran across `select` which is essentially a non blocking way to check which sockets have data available to read, and then sequentially read the data. This gives the ability for a program to appear parallelized in the sense that it can handle multiple client connections, but its not truly parallel. Different clients can be handled at different time depending on which order they connect, which is asynchronous in nature (versus processing each client in sequence and waiting on each one to connect and disconnect before moving on to the next one) Async in Python is basically this concept, with a core fundamental feature of time limited execution. Functions can say that they are pausing for x seconds, allowing other functions to run, or functions can say that they give a certain function x seconds to run before resuming execution. If you async code (along with any library you may use) doesn't contain any sleeps or timeouts, its exactly equivalent to synchronous code (since the event loop never really recieves a message that it can suspend a routine or cancel it). With sleeps and timeouts, you gain control over things that can potentially block, both from a caller perspective of not having a function call block your own, and from a callee perspective of not making your function blocking. The use case is for it is that it is good for I/O bound operations like Threading is, but with the addition that you don't have to worry about synchronization or race conditions, since by design your code will have predictable access patterns. The downside is that your code and any libraries that you use within your code has to be implemented as async libraries, and any library that is async has to have async wrappers around the calls to its methods, which in turn means that your entire code has to be async. Threading with Python is generally not useful, as its not true parallelism because of GIL. GIL allows only one thread in Python to run. Threading is safer in Python because of this, however it obviously has drawbacks. In general its best used if you want asyncio like performance with a library that is not written with async, since GIL is smart enough to detect when a thread is waiting for input and switch context. True parallelism in Python is achieved with multiprocessing, however the use case is a little different. Rather than spinning off processes, you generally launch a bunch of worker processes up front (to avoid the larger overhead), then use smart scheduling to distribute work between these processes. Here though you do have to worry about race conditions and synchronization, and use things like locks and mutexes.

x-complexity 4y ago

> Async Python is still confusing af - when do I need it, It's needed when you're spending a lot of time waiting for an I/O request to complete (network/HTTP requests, disk reads/writes, database reads/writes) > what happens under the hood, https://tenthousandmeters.com/blog/python-behind-the-scenes-12-how-asyncawait-works-in-python/#:~:text=likely%20to%20use.-,asyncio,-asyncio%20came%20to https://tenthousandmeters.com/blog/python-behind-the-scenes-... (Please read the entire blog post - It goes through the necessary concepts like generators, event loops, & coroutines) > does it actually help with performance, Refer to the first answer: You'll see improved performances if your workloads are mainly comprised of waiting for other stuff to complete. If you're compute-heavy, it'll be better to use the 'multiprocessing' library instead. > sometimes the GIL comes into play and sometimes it doesn't, The GIL comes into play when you have a lot of compute-heavy tasks: Otherwise, you'll rarely encounter it. It's only when you have that many compute-heavy tasks that you start to use the 'multiprocessing' library. > why do we ever use threads at all if there's a GIL, Threads exist because it was there before asyncio & event loops came into Python. > why is it called asyncio if we can use it for anything. Its name came from PEP 3156, proposing the asyncio library back in 2012. https://peps.python.org/pep-3156/ https://peps.python.org/pep-3156/ As for why, asynchronous I/O stands in contrast to synchronous I/O, where the program/thread had to wait for the I/O request to complete before it can do anything else. Making tasks asynchronous allows it to do other stuff while it waits for a task's request to complete, increasing CPU & I/O utilization. > My mind is kind of scattered and people seems to be using a lot of async Python for some reason. Any good resources to clear things up? Highly recommend this video from mcoding: It's fairly simple & goes through a sample implementation. https://www.youtube.com/watch?v=ftmdDlwMwwQ https://www.youtube.com/watch?v=ftmdDlwMwwQ Also, this article: https://realpython.com/async-io-python/ https://realpython.com/async-io-python/

franga2000 4y ago

When working in a fully async context, it becomes very logical and natural. A great example of this is the FastAPI web framework. You never write any top-level code, only functions that the framework calls, so you never have to deal with the event loop directly. You basically just sprinkle some async and await keywords around your IO bottlenecks and things suddenly run smoother.

loa_in_ 4y ago

In personal experience, when you write anything more complicated using threads in python, I end up writing lots of boilerplate code that I wished I could just shove somewhere - Events, 1 element Queues, ContextVar contexts, all of which don't benefit for being named and make appearance only in two spots in code. Async removed a lot of this for me while also unifying the future ecosystem and making my code more composable and easy to integrate.