22 ms·

Sync vs. Async Python: What Is the Difference?

151 points by gilad 6y ago

doubleunplussed 6y ago
Not mentioned in TFA, but I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent threads/coroutines. You can't do this with threads. Async lets you do this whilst still storing the state of unfinished things in a stack, i.e. not having to make a million callbacks. Making it look like you're using threads even though you're not. The better async code and interfaces become, the more it looks like regular old multithreaded code, except there aren't actual threads underlying it. You still need to make sure you serialise access to shared resources, don't share data that shouldn't be shared, etc. All the same considerations as multithreaded code. Almost ends up looking like the underlying mechanism: threads with a GIL vs async ought to be an implementation detail that doesn't require you to modify your entire programming model. Extension code not holding the GIL can still run in true parallel with real threads, so that's a meaningful difference but is usually not relevant for IO where async is usually used.
- nurettin 6y ago
  This explanation works great as long as the underlying implementation of threads in your particular python implementation is ultimately concurrent, but not parallel.
- matheusmoreira 6y ago
  > I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread Yes. The point is to use a single thread with one stack to process several tasks. This consumes less memory. For example, Go currently has a minimum stack size of 2 KiB so a machine with 4 GiB of memory will be able to process less than 2 million goroutines. An event loop uses a single thread with a single stack, reducing memory usage at the cost of complexity. Asynchronous functions are just like coroutines. The difference is they return to the awaiting caller instead of yielding to another function. The order of execution is determined by the underlying loop.
- dorfsmay 6y ago
  Also there is a point where managing many threads become a load on the CPU, which my inderstanding is why Rust is not provinding green threads anymore.
- calpaterson 6y ago
  Why are you "utterly convinced" of that though? If I run 20 threads in Python, unix top reports resident memory usage to me as 14mb. Why do I see that number instead of 160mb? Code sample: https://gist.github.com/calpaterson/ab35377da9275ca3af7072dbd03ec3a6 https://gist.github.com/calpaterson/ab35377da9275ca3af7072db...
  - Scarbutt 6y ago
    You mean 14MB?
    - calpaterson 6y ago
      Htop shows me "14016" as resident memory and I don't think it is using kilobytes as the unit.
      - anonymoushn 6y ago
        It is using kilobytes as the unit.
        calpaterson 6y ago
        Yes you're right. What a silly mistake. Normal Python interpreter = 9mb, with 20 threads = 14mb. Edited the original post.
        mlyle 6y ago
        Answer to your question was already discussed right here: https://news.ycombinator.com/item?id=24429221 https://news.ycombinator.com/item?id=24429221
      - mlyle 6y ago
        That's embarrassing.
        6y ago
- harikb 6y ago
  There are also other language implementation styles that get roughly the same benefit without writing async and await all over the codebase. If the implied yield at defined synchronization points coupled with a decent scheduler like in Go would make this all a non-issue [1]. [1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
- lmm 6y ago
  > avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent threads/coroutines. You can't do this with threads. It's 8 K B per thread, so you can scale a thousand times further than you thought. One dark secret of the async movement is that if your goal is C10K (10,000 concurrent clients) then actually bog standard threading will handle that fine these days. > The better async code and interfaces become, the more it looks like regular old multithreaded code, except there aren't actual threads underlying it. You still need to make sure you serialise access to shared resources, don't share data that shouldn't be shared, etc. All the same considerations as multithreaded code. Depends what approach you're using. I prefer making an explicit distinction between sync and async functions ( https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html ), so you effectively invert the notion of a "critical section" - instead of marking which sections can't yield, you mark which sections can yield, so your code is safe by default and you can introduce concurrency explicitly as and when you need it for performance, rather than your code being fast-but-unsafe by default and you're expected to fix a bunch of rare nondeterministic bugs with minimal support from your tools, which is how it works in a multithreading world.
  - mlyle 6y ago
    > It's 8 K B per thread, so you can scale a thousand times further than you thought. One dark secret of the async movement is that if your goal is C10K (10,000 concurrent clients) then actually bog standard threading will handle that fine these days. Default virtual memory allocation for threads on Linux distributions tends to be 8 megabytes. Actual memory used is the peak stack depth used, rounded up a bit. It'd be pretty unusual to only use as little as 8 kilobytes per thread; just the standard per-thread libc context information for concurrency is a few kilobytes, plus at least one page of stack, plus the kernel's information about the thread (which isn't counted against the process)... Yes, you can spawn thousands of threads on relatively modest hardware; I was spawning thousands of threads a decade ago. Spawning 5000 bare-minimal python threads that do nil seems to use about 300 megs of ram on my system; real threads that do anything substantial will use a whole lot more, even if their use of the stack depth is intermittent. Not to mention allocators that cache part of freed heap per-thread, etc.
- otabdeveloper4 6y ago
  The "8 megabytes" is virtual memory. You're only incrementing a counter in a table, nothing is actually allocated until you actually start using that stack.
- jnwatson 6y ago
  In general, it is far easier to reason about locking with async code, as the number of preemption points is far far lower. As a result, you get very low overhead inter-task communication.
- laurencerowe 6y ago
  > I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent threads/coroutines. If it's possible to avoid shared state then I tend to prefer threads, but in the presence of shared state there are good reasons to think that explicit coroutines are easier to reason about than threads or green-threads: https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html
  - gpderetta 6y ago
    but then you need to forfeit parallelism. If you add parallel execution to (shared memory) async, the advantage disappear.
    - laurencerowe 6y ago
      With Python you forfeit parallelism either way due to the GIL.
- justsomeuser 6y ago
  It’s not the only reason. For me, async code is easier to debug and understand because it is composed of regular functions (tagged with async). This means if you have a function that is 10 layers deep, you can pause the debugger and see the stack context. Same with your IDE, you can jump to each function. With threads that 10 function stack would be 10 threads, each requiring some sort of tooling at both software write time and runtime to get the context. In summary composing a system of just “functions (sync + async)” is easier than vs “functions + threads”
  - gpderetta 6y ago
    hum, I'm missing something. The same call stack you get with async would map to exactly one thread call stack. Sure each thread will get its own call stack, but all related continuations that participate in an async call stack would be owned by the same thread (Assuming the same application design).
    - justsomeuser 6y ago
      My point is that with async both the runtime and the dev tools try to make async call stacks look exactly like sync ones. This keeps the context of how your code gets into certain states. If you have 10 threads, it’s like you have 10 different processes (without the context of how they are related - which you get with composing functions) which is harder to understand. Also assuming that each thread does not have an event loop.
  - arethuza 6y ago
    "With threads that 10 function stack would be 10 threads" Presumably only if you have one-thread per function - which I guess you could do but not sure if anyone actually writes multi-threaded code that way.
    - justsomeuser 6y ago
      If you wanted each function to wait for IO or an event without blocking a thread you need an event loop, or one thread per function that needs to block to wait on incoming events.
- ynik 6y ago
  In a multi-threaded context yes, reduced memory usage is the main benefit. But async/await can also be used for other things! In a C# Windows GUI application, it's normal to use async/await on the UI thread. Your UI can await multiple tasks at the same time, yet you don't need any locks when accessing the UI state; because all your code runs on the UI thread. This is a really useful programming model made possible by cooperative task-switching via `await` on a single thread. Here the "await" being explicit is a crucial feature, it allows the programmer to reason about when the shared state might be mutated by other tasks (or maybe by the user clicking cancel while the current task is waiting). Any pre-emptive task switching adds a lot of additional complexity and isn't really suitable for UI code.
  - spaetzleesser 6y ago
    “Your UI can await multiple tasks at the same time, yet you don't need any locks when accessing the UI state; because all your code runs on the UI thread.” Is that true? I thought the code after each await runs on a different thread from the code before the await. At least that’s what I have observed when debugging things.
- heeen2 6y ago
  > I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread I needed to write an interface to a SSE API (tldr: long running socket connection with occasional message arriving as content) There simply was no way to poll a http connection (e.g. requests) if it had new data. It would block no matter what. So I would have to start writing threaded code, or just use asyncio which seemed much more ergonomic.
calpaterson 6y ago
> If these hundred tasks make heavy use of the CPU, then the sync and async solutions would have similar performance, since the speed at which the CPU runs is fixed, Python's speed of executing code is always the same and the work to be done by the application is also equal. I think one of the things the author has not discussed is that if you have high CPU then you are going to get starvation as async does not have a fairness mechanism - timesharing is co-operative and based on FIFO in all the implementations I've seen. If you start to saturate CPU (even a bit) you get uneven distribution of cpu time between your various async threads. This translates into chronic timeouts. For async to be applicable you really need to have a) high io load and b) lowish cpu load.
- sltkr 6y ago
  > ... you are going to get starvation as async does not have a fairness mechanism - timesharing is co-operative and based on FIFO in all the implementations I've seen. If it's FIFO based, why would you get starvation?
  - euiq 6y ago
    What would the CPU-heavy code await on?
  - calpaterson 6y ago
    FIFO systems can have starvation when dealing with time bound things like web requests - slow request handlers can cause other (waiting) requests to timeout. In a thread-per-request or process-per-request model where you run say 4 * cpu workers the operating system scheduler is able to interrupt stuff that's hogging the CPU but that isn't the case in a typical 1 * cpu worker async model. When there's little cpu load this is not a real threat but when there is material load you can easily run into problems.
    - sltkr 6y ago
      I see; it sounds like latency would become very unpredictable under that model. I can imagine that's a serious problem in practice, but it's not typically what's meant by starvation. https://en.wikipedia.org/wiki/Starvation_(computer_science) https://en.wikipedia.org/wiki/Starvation_(computer_science)
holografix 6y ago
I’m very uninformed when it comes to async in Python so please bear with me. Given a “web API type” python app, where about 30% of the APIs also reach out to other, external APIs, before responds to the incoming requests... Is that _the_ classic example where async would provide a solid benefit? Is that why Node has become so popular = it’s all json + non blocking API responses?
- hansvm 6y ago
  That's where it _could_ provide a solid benefit (though you could probably usually do better by instantiating an appropriately sized thread pool and avoiding async entirely), but that's missing the real power of async which is that it makes concurrency easier to reason about. Except at an await the application behaves serially; you can't get interrupted in between lines or in the middle of an operation that it turns out isn't atomic.
  - arethuza 6y ago
    "it makes concurrency easier to reason about" Doesn't it do that by not really having concurrency? There is only a single locus of control so a whole class of problems should simply cease to exist.
    - hansvm 6y ago
      We might be nit-picking definitions, but I don't think that's a fair characterization. From the perspective of the user programming with async, when you gather the results of two coroutines you don't care at all the order used for each stack frame. The mental model is one of concurrent operations which might suspend at yield points.
- divbzero 6y ago
  Yes, the “web API type” app you describe is a classic example. Asynchronous Python is similar to async/await in C# or the non-blocking patterns in JS.
boardwaalk 6y ago
I just want to rep Trio, mentioned in the article. I'm using it to prototype a system in a different language before doing another iteration of said system and it's is quite nice to use, at least compared to what I remember asyncio being like.
- florimondmanca 6y ago
  > it's is quite nice to use, at least compared to what I remember asyncio being like. Yes, I agree. Fairly recently there's been anyio [0], which brings ideas from trio to other async libraries, in particular asyncio. E.g. it has equivalent of trio nurseries (what anyio calls "task groups") for implementing structured concurrency ideas in asyncio environments (saving you from headaches due to having to deal with tasks manually). Very neat way to use trio ideas when stuck on asyncio (e.g. for web apps). :) anyio was also discussed in the latest Python Bytes episode [1]. [0]: https://github.com/agronholm/anyio https://github.com/agronholm/anyio [1]: https://pythonbytes.fm/episodes/show/197/structured-concurrency-in-pythonB https://pythonbytes.fm/episodes/show/197/structured-concurre...
jacquesm 6y ago
All this async stuff should die as soon as possible. It exposes the limitations of the underlying tech and forces the application developer to deal with things that the systems level people couldn't bother with. Look at the Erlang eco-system to see how this is done the right way. Threads, asynchronicity, locks and so on should be dealt with at the OS or the framework level, not at the application level. Javascript also suffers from this and it makes JS ugly and hard to read. Callback hell will be yours or you end up with crutches such as promises. At the application level things should be as deterministic as possible and the default should be that statements executed in sequence will have their side effects updated in sequence as well.
- speedgoose 6y ago
  Thanks to the async/await keywords, modern javascript is not a callback hell anymore.
  - jacquesm 6y ago
    No, now you get it all mixed up. One library will use callback functions you supply, another will use callbacks but inline them and yet another will use async/await. And you, the application developer are left somewhere in the middle. Oh, and you can't use 'await' in the main thread of your JS, you can only use it from within another async function, so when you need it most - for instance during application start-up - it isn't available.
    - speedgoose 6y ago
      You can use promisify for the first one. For the second one I think these libraries doing that but still not supporting promises are not that common anymore but nothing you can't do with promises yourself. Javascript has a single thread, but I see what you mean and it has been fixed recently. To support older versions you can simply declare an anonymous async function and execute it immediately.
  - doubleunplussed 6y ago
    I think of it the other way around: Thanks to javascript being single-threaded, we now have async/await keywords strewn throughout the software ecosystem where they provide no benefit.
andrewstuart 6y ago
I absolutely love async Python. It lets me do stuff that would be hard with ordinary Python. No doubt it takes some effort and practice to grasp, but once you have it worked out you'll find it's a powerful tool in the toolbelt. Many languages now have async and await - it's not exclusive to Python. The reason so many languages have gopne this way is that so far it's the easiest and most sensible solution to writing concurrent code in a way that won't become incomprehensible. And if you are trying to get your head around it, the easiest analogy is the web browser and JavaScript. If you have grasped JavaScript and the event loop then essentially you have grasped async Python, or not far from it. Some people misunderstand async Python - they think the point is to make Python faster - it isn't. The point is to enable concurrent programming. I do understand the haters - at first, until you grasp it, async Python so incredibly foreign and so different from ordinary synchronous Python that you might think "AAAARGH why did they do this and make it so hard"? But really it's not hard once you grasp the mental model of what is going on.
- nurettin 6y ago
  I had so much trouble getting the python tcp client to work like I wanted it to. Then I tried my hand on the async tcp client and was pleasantly surprised how simple it was. reader, writer = await asyncio.wait_for(asyncio.open_connection(), connect_timeout) line = "" while True: line = await asyncio.wait_for(reader.readline(), read_timeout) if line == b"": break yield line
  - andrewstuart 6y ago
    I've written a bunch of things that were really well suited to async Python including an mpjeg server, a queueing server and a process management server that monitors the stdout and stderr of various processes and reads them in real time and acts upon the messages it reads. Each one of those applications was made possible comprehensible and even easy by async Python.
    - nurettin 6y ago
      I made a python clone of multitail in just under a day thanks to being able to read multiple files simultaneously with async. It really is overpowered.
pansa2 6y ago
> Greenlets are similar to coroutines [...] the async ecosystem in Python is fractured in two big groups. Async in software development in general is fractured in two big groups. On the one hand languages like C#, C++ and Dart support (stackless) coroutines like Python’s - and on the other hand, Go, Java and Lua support stackful coroutines that are more like Python’s greenlets. There are pros and cons to each approach. I wonder if one or the other will eventually become dominant.
- BorisTheBrave 6y ago
  What is Java's support for "stackful coroutines"?
  - pansa2 6y ago
    Project Loom: https://wiki.openjdk.java.net/display/loom/Main https://wiki.openjdk.java.net/display/loom/Main
    - fnord123 6y ago
      You used the present tense. Loom is not available yet.
- gpderetta 6y ago
  to be fair, stackless coroutines are in the C++ language because they require language support. There are also options for stackful coroutines and of course good old threads.
shermanmccoy 6y ago
Any emacs users know of a linter that will tell me when I forget to await a coroutine?
- war1025 6y ago
  Pretty sure I added a lint rule to pylint the other day that will do just that. I imagine emacs has a pylint plugin. Basically you add a linter that overrides `visit_callfunc()` and make sure it's not an async function being called.
  - shermanmccoy 6y ago
    Cheers.
jgehrcke 6y ago
If you're curious about this topic then I would love to link you to https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a... written by Mike Bayer (author of SQLAlchemy) -- it's an article that I keep referring to as a must-read, incredibly well written and all about real-world implications (totally transferable to other non-Python ecosystems). If you like to, maybe also have a read into https://gehrcke.de/gipc/background.html https://gehrcke.de/gipc/background.html. This is where I tried to put into words what I learned a couple of years ago when I wrote a library for the gevent ecosystem. You'll find some paragraphs about "Cooperative scheduling vs. preemptive scheduling", about "Asynchronous vs. synchronous", and about a few more simple and yet important topics in the world of event-driven architectures.
ds0 6y ago
tangentially, I'd like to give thanks to Miguel Grinberg for getting me started on flask and sockets in python via his posts.
zwieback 6y ago
What this really highlights is that you need to learn the basics to make the right decision for your application and platform. There's no one solution for everyone but if you don't take the time to understand context switching and the role of your OS then you will be doomed to tilting at windmills.
luord 6y ago
Didn't know that about Flask. One learns something new every day, even with tools one's been using for years. Actually, I gotta take a closer look to greenlets in general.

151 points by gilad 6y ago

Not mentioned in TFA, but I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent threads/coroutines. You can't do this with threads. Async lets you do this whilst still storing the state of unfinished things in a stack, i.e. not having to make a million callbacks. Making it look like you're using threads even though you're not. The better async code and interfaces become, the more it looks like regular old multithreaded code, except there aren't actual threads underlying it. You still need to make sure you serialise access to shared resources, don't share data that shouldn't be shared, etc. All the same considerations as multithreaded code. Almost ends up looking like the underlying mechanism: threads with a GIL vs async ought to be an implementation detail that doesn't require you to modify your entire programming model. Extension code not holding the GIL can still run in true parallel with real threads, so that's a meaningful difference but is usually not relevant for IO where async is usually used.

nurettin 6y ago
This explanation works great as long as the underlying implementation of threads in your particular python implementation is ultimately concurrent, but not parallel.
matheusmoreira 6y ago
> I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread Yes. The point is to use a single thread with one stack to process several tasks. This consumes less memory. For example, Go currently has a minimum stack size of 2 KiB so a machine with 4 GiB of memory will be able to process less than 2 million goroutines. An event loop uses a single thread with a single stack, reducing memory usage at the cost of complexity. Asynchronous functions are just like coroutines. The difference is they return to the awaiting caller instead of yielding to another function. The order of execution is determined by the underlying loop.
dorfsmay 6y ago
Also there is a point where managing many threads become a load on the CPU, which my inderstanding is why Rust is not provinding green threads anymore.
calpaterson 6y ago
Why are you "utterly convinced" of that though? If I run 20 threads in Python, unix top reports resident memory usage to me as 14mb. Why do I see that number instead of 160mb? Code sample: https://gist.github.com/calpaterson/ab35377da9275ca3af7072dbd03ec3a6 https://gist.github.com/calpaterson/ab35377da9275ca3af7072db...
- Scarbutt 6y ago
  You mean 14MB?
  - calpaterson 6y ago
    Htop shows me "14016" as resident memory and I don't think it is using kilobytes as the unit.
    - anonymoushn 6y ago
      It is using kilobytes as the unit.
      - calpaterson 6y ago
        Yes you're right. What a silly mistake. Normal Python interpreter = 9mb, with 20 threads = 14mb. Edited the original post.
        mlyle 6y ago
        Answer to your question was already discussed right here: https://news.ycombinator.com/item?id=24429221 https://news.ycombinator.com/item?id=24429221
    - mlyle 6y ago
      That's embarrassing.
      - 6y ago
harikb 6y ago
There are also other language implementation styles that get roughly the same benefit without writing async and await all over the codebase. If the implied yield at defined synchronization points coupled with a decent scheduler like in Go would make this all a non-issue [1]. [1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
lmm 6y ago
> avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent threads/coroutines. You can't do this with threads. It's 8 K B per thread, so you can scale a thousand times further than you thought. One dark secret of the async movement is that if your goal is C10K (10,000 concurrent clients) then actually bog standard threading will handle that fine these days. > The better async code and interfaces become, the more it looks like regular old multithreaded code, except there aren't actual threads underlying it. You still need to make sure you serialise access to shared resources, don't share data that shouldn't be shared, etc. All the same considerations as multithreaded code. Depends what approach you're using. I prefer making an explicit distinction between sync and async functions ( https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html ), so you effectively invert the notion of a "critical section" - instead of marking which sections can't yield, you mark which sections can yield, so your code is safe by default and you can introduce concurrency explicitly as and when you need it for performance, rather than your code being fast-but-unsafe by default and you're expected to fix a bunch of rare nondeterministic bugs with minimal support from your tools, which is how it works in a multithreading world.
- mlyle 6y ago
  > It's 8 K B per thread, so you can scale a thousand times further than you thought. One dark secret of the async movement is that if your goal is C10K (10,000 concurrent clients) then actually bog standard threading will handle that fine these days. Default virtual memory allocation for threads on Linux distributions tends to be 8 megabytes. Actual memory used is the peak stack depth used, rounded up a bit. It'd be pretty unusual to only use as little as 8 kilobytes per thread; just the standard per-thread libc context information for concurrency is a few kilobytes, plus at least one page of stack, plus the kernel's information about the thread (which isn't counted against the process)... Yes, you can spawn thousands of threads on relatively modest hardware; I was spawning thousands of threads a decade ago. Spawning 5000 bare-minimal python threads that do nil seems to use about 300 megs of ram on my system; real threads that do anything substantial will use a whole lot more, even if their use of the stack depth is intermittent. Not to mention allocators that cache part of freed heap per-thread, etc.
otabdeveloper4 6y ago
The "8 megabytes" is virtual memory. You're only incrementing a counter in a table, nothing is actually allocated until you actually start using that stack.
jnwatson 6y ago
In general, it is far easier to reason about locking with async code, as the number of preemption points is far far lower. As a result, you get very low overhead inter-task communication.
laurencerowe 6y ago
> I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent threads/coroutines. If it's possible to avoid shared state then I tend to prefer threads, but in the presence of shared state there are good reasons to think that explicit coroutines are easier to reason about than threads or green-threads: https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html
- gpderetta 6y ago
  but then you need to forfeit parallelism. If you add parallel execution to (shared memory) async, the advantage disappear.
  - laurencerowe 6y ago
    With Python you forfeit parallelism either way due to the GIL.
justsomeuser 6y ago
It’s not the only reason. For me, async code is easier to debug and understand because it is composed of regular functions (tagged with async). This means if you have a function that is 10 layers deep, you can pause the debugger and see the stack context. Same with your IDE, you can jump to each function. With threads that 10 function stack would be 10 threads, each requiring some sort of tooling at both software write time and runtime to get the context. In summary composing a system of just “functions (sync + async)” is easier than vs “functions + threads”
- gpderetta 6y ago
  hum, I'm missing something. The same call stack you get with async would map to exactly one thread call stack. Sure each thread will get its own call stack, but all related continuations that participate in an async call stack would be owned by the same thread (Assuming the same application design).
  - justsomeuser 6y ago
    My point is that with async both the runtime and the dev tools try to make async call stacks look exactly like sync ones. This keeps the context of how your code gets into certain states. If you have 10 threads, it’s like you have 10 different processes (without the context of how they are related - which you get with composing functions) which is harder to understand. Also assuming that each thread does not have an event loop.
- arethuza 6y ago
  "With threads that 10 function stack would be 10 threads" Presumably only if you have one-thread per function - which I guess you could do but not sure if anyone actually writes multi-threaded code that way.
  - justsomeuser 6y ago
    If you wanted each function to wait for IO or an event without blocking a thread you need an event loop, or one thread per function that needs to block to wait on incoming events.
ynik 6y ago
In a multi-threaded context yes, reduced memory usage is the main benefit. But async/await can also be used for other things! In a C# Windows GUI application, it's normal to use async/await on the UI thread. Your UI can await multiple tasks at the same time, yet you don't need any locks when accessing the UI state; because all your code runs on the UI thread. This is a really useful programming model made possible by cooperative task-switching via `await` on a single thread. Here the "await" being explicit is a crucial feature, it allows the programmer to reason about when the shared state might be mutated by other tasks (or maybe by the user clicking cancel while the current task is waiting). Any pre-emptive task switching adds a lot of additional complexity and isn't really suitable for UI code.
- spaetzleesser 6y ago
  “Your UI can await multiple tasks at the same time, yet you don't need any locks when accessing the UI state; because all your code runs on the UI thread.” Is that true? I thought the code after each await runs on a different thread from the code before the await. At least that’s what I have observed when debugging things.
heeen2 6y ago
> I'm utterly convinced that the only compelling reason for async is to avoid the per-thread stack memory allocation of 8MB per thread I needed to write an interface to a SSE API (tldr: long running socket connection with occasional message arriving as content) There simply was no way to poll a http connection (e.g. requests) if it had new data. It would block no matter what. So I would have to start writing threaded code, or just use asyncio which seemed much more ergonomic.

calpaterson 6y ago

> If these hundred tasks make heavy use of the CPU, then the sync and async solutions would have similar performance, since the speed at which the CPU runs is fixed, Python's speed of executing code is always the same and the work to be done by the application is also equal. I think one of the things the author has not discussed is that if you have high CPU then you are going to get starvation as async does not have a fairness mechanism - timesharing is co-operative and based on FIFO in all the implementations I've seen. If you start to saturate CPU (even a bit) you get uneven distribution of cpu time between your various async threads. This translates into chronic timeouts. For async to be applicable you really need to have a) high io load and b) lowish cpu load.

sltkr 6y ago
> ... you are going to get starvation as async does not have a fairness mechanism - timesharing is co-operative and based on FIFO in all the implementations I've seen. If it's FIFO based, why would you get starvation?
- euiq 6y ago
  What would the CPU-heavy code await on?
- calpaterson 6y ago
  FIFO systems can have starvation when dealing with time bound things like web requests - slow request handlers can cause other (waiting) requests to timeout. In a thread-per-request or process-per-request model where you run say 4 * cpu workers the operating system scheduler is able to interrupt stuff that's hogging the CPU but that isn't the case in a typical 1 * cpu worker async model. When there's little cpu load this is not a real threat but when there is material load you can easily run into problems.
  - sltkr 6y ago
    I see; it sounds like latency would become very unpredictable under that model. I can imagine that's a serious problem in practice, but it's not typically what's meant by starvation. https://en.wikipedia.org/wiki/Starvation_(computer_science) https://en.wikipedia.org/wiki/Starvation_(computer_science)

holografix 6y ago

I’m very uninformed when it comes to async in Python so please bear with me. Given a “web API type” python app, where about 30% of the APIs also reach out to other, external APIs, before responds to the incoming requests... Is that _the_ classic example where async would provide a solid benefit? Is that why Node has become so popular = it’s all json + non blocking API responses?

hansvm 6y ago
That's where it _could_ provide a solid benefit (though you could probably usually do better by instantiating an appropriately sized thread pool and avoiding async entirely), but that's missing the real power of async which is that it makes concurrency easier to reason about. Except at an await the application behaves serially; you can't get interrupted in between lines or in the middle of an operation that it turns out isn't atomic.
- arethuza 6y ago
  "it makes concurrency easier to reason about" Doesn't it do that by not really having concurrency? There is only a single locus of control so a whole class of problems should simply cease to exist.
  - hansvm 6y ago
    We might be nit-picking definitions, but I don't think that's a fair characterization. From the perspective of the user programming with async, when you gather the results of two coroutines you don't care at all the order used for each stack frame. The mental model is one of concurrent operations which might suspend at yield points.
divbzero 6y ago
Yes, the “web API type” app you describe is a classic example. Asynchronous Python is similar to async/await in C# or the non-blocking patterns in JS.

boardwaalk 6y ago

I just want to rep Trio, mentioned in the article. I'm using it to prototype a system in a different language before doing another iteration of said system and it's is quite nice to use, at least compared to what I remember asyncio being like.

florimondmanca 6y ago
> it's is quite nice to use, at least compared to what I remember asyncio being like. Yes, I agree. Fairly recently there's been anyio [0], which brings ideas from trio to other async libraries, in particular asyncio. E.g. it has equivalent of trio nurseries (what anyio calls "task groups") for implementing structured concurrency ideas in asyncio environments (saving you from headaches due to having to deal with tasks manually). Very neat way to use trio ideas when stuck on asyncio (e.g. for web apps). :) anyio was also discussed in the latest Python Bytes episode [1]. [0]: https://github.com/agronholm/anyio https://github.com/agronholm/anyio [1]: https://pythonbytes.fm/episodes/show/197/structured-concurrency-in-pythonB https://pythonbytes.fm/episodes/show/197/structured-concurre...

jacquesm 6y ago

All this async stuff should die as soon as possible. It exposes the limitations of the underlying tech and forces the application developer to deal with things that the systems level people couldn't bother with. Look at the Erlang eco-system to see how this is done the right way. Threads, asynchronicity, locks and so on should be dealt with at the OS or the framework level, not at the application level. Javascript also suffers from this and it makes JS ugly and hard to read. Callback hell will be yours or you end up with crutches such as promises. At the application level things should be as deterministic as possible and the default should be that statements executed in sequence will have their side effects updated in sequence as well.

speedgoose 6y ago
Thanks to the async/await keywords, modern javascript is not a callback hell anymore.
- jacquesm 6y ago
  No, now you get it all mixed up. One library will use callback functions you supply, another will use callbacks but inline them and yet another will use async/await. And you, the application developer are left somewhere in the middle. Oh, and you can't use 'await' in the main thread of your JS, you can only use it from within another async function, so when you need it most - for instance during application start-up - it isn't available.
  - speedgoose 6y ago
    You can use promisify for the first one. For the second one I think these libraries doing that but still not supporting promises are not that common anymore but nothing you can't do with promises yourself. Javascript has a single thread, but I see what you mean and it has been fixed recently. To support older versions you can simply declare an anonymous async function and execute it immediately.
- doubleunplussed 6y ago
  I think of it the other way around: Thanks to javascript being single-threaded, we now have async/await keywords strewn throughout the software ecosystem where they provide no benefit.

andrewstuart 6y ago

I absolutely love async Python. It lets me do stuff that would be hard with ordinary Python. No doubt it takes some effort and practice to grasp, but once you have it worked out you'll find it's a powerful tool in the toolbelt. Many languages now have async and await - it's not exclusive to Python. The reason so many languages have gopne this way is that so far it's the easiest and most sensible solution to writing concurrent code in a way that won't become incomprehensible. And if you are trying to get your head around it, the easiest analogy is the web browser and JavaScript. If you have grasped JavaScript and the event loop then essentially you have grasped async Python, or not far from it. Some people misunderstand async Python - they think the point is to make Python faster - it isn't. The point is to enable concurrent programming. I do understand the haters - at first, until you grasp it, async Python so incredibly foreign and so different from ordinary synchronous Python that you might think "AAAARGH why did they do this and make it so hard"? But really it's not hard once you grasp the mental model of what is going on.

nurettin 6y ago
I had so much trouble getting the python tcp client to work like I wanted it to. Then I tried my hand on the async tcp client and was pleasantly surprised how simple it was. reader, writer = await asyncio.wait_for(asyncio.open_connection(), connect_timeout) line = "" while True: line = await asyncio.wait_for(reader.readline(), read_timeout) if line == b"": break yield line
- andrewstuart 6y ago
  I've written a bunch of things that were really well suited to async Python including an mpjeg server, a queueing server and a process management server that monitors the stdout and stderr of various processes and reads them in real time and acts upon the messages it reads. Each one of those applications was made possible comprehensible and even easy by async Python.
  - nurettin 6y ago
    I made a python clone of multitail in just under a day thanks to being able to read multiple files simultaneously with async. It really is overpowered.

pansa2 6y ago

> Greenlets are similar to coroutines [...] the async ecosystem in Python is fractured in two big groups. Async in software development in general is fractured in two big groups. On the one hand languages like C#, C++ and Dart support (stackless) coroutines like Python’s - and on the other hand, Go, Java and Lua support stackful coroutines that are more like Python’s greenlets. There are pros and cons to each approach. I wonder if one or the other will eventually become dominant.

BorisTheBrave 6y ago
What is Java's support for "stackful coroutines"?
- pansa2 6y ago
  Project Loom: https://wiki.openjdk.java.net/display/loom/Main https://wiki.openjdk.java.net/display/loom/Main
  - fnord123 6y ago
    You used the present tense. Loom is not available yet.
gpderetta 6y ago
to be fair, stackless coroutines are in the C++ language because they require language support. There are also options for stackful coroutines and of course good old threads.

shermanmccoy 6y ago

Any emacs users know of a linter that will tell me when I forget to await a coroutine?

war1025 6y ago
Pretty sure I added a lint rule to pylint the other day that will do just that. I imagine emacs has a pylint plugin. Basically you add a linter that overrides `visit_callfunc()` and make sure it's not an async function being called.
- shermanmccoy 6y ago
  Cheers.

jgehrcke 6y ago

If you're curious about this topic then I would love to link you to https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a... written by Mike Bayer (author of SQLAlchemy) -- it's an article that I keep referring to as a must-read, incredibly well written and all about real-world implications (totally transferable to other non-Python ecosystems). If you like to, maybe also have a read into https://gehrcke.de/gipc/background.html https://gehrcke.de/gipc/background.html. This is where I tried to put into words what I learned a couple of years ago when I wrote a library for the gevent ecosystem. You'll find some paragraphs about "Cooperative scheduling vs. preemptive scheduling", about "Asynchronous vs. synchronous", and about a few more simple and yet important topics in the world of event-driven architectures.

ds0 6y ago

tangentially, I'd like to give thanks to Miguel Grinberg for getting me started on flask and sockets in python via his posts.

zwieback 6y ago

What this really highlights is that you need to learn the basics to make the right decision for your application and platform. There's no one solution for everyone but if you don't take the time to understand context switching and the role of your OS then you will be doomed to tilting at windmills.

luord 6y ago

Didn't know that about Flask. One learns something new every day, even with tools one's been using for years. Actually, I gotta take a closer look to greenlets in general.