4 ms·

The two don't really compete, because async/await is primarily about parallelizing IO. If I want to (say) probe a dozen URLs for liveness in parallel, or write

The two don't really compete, because async/await is primarily about parallelizing IO. If I want to (say) probe a dozen URLs for liveness in parallel, or write data to/from thousands of client sockets from my webserver, doing that with threads--especially free-threaded Python threads, which are still quite lock-happy inside the interpreter, GIL or not--has a very quickly-noticeable performance and resource cost. Async/await's primary utility is that of a capable utility interface for making I/O concurrent (and parallel as well, in many cases), regardless of whether threads are in use. Hell, even golang multiplexes concurrent goroutines' threads onto concurrent IO schedulers behind the scenes, as does Java's NIO, Erlang/BEAM, and many many similar systems.

PaulHoule 1y ago
People who talk about there being a hard line between parallelism and concurrency are always writing code with race conditions that they deny exist or writing code with performance bottlenecks they can't understand because they deny they exist. I like working in Java because you can use the same threading primitives for both and have systems that work well in both IO-dominated and CPU-dominated regimes which sometimes happen in the same application under different conditions. Personally there are enough details to work out that we might be up to Python 3.24 when you can really count on all your dependncies to be thread safe. One of the reasons Java has been successful is the extreme xenophobia (not to mention painful JNI) which meant we re-implemented stuff to be thread-safe in pure Java as opposed to sucking in a lot of C/C++ stuff which will never be thread safe.
- zelphirkalt 1y ago
  Using multiple OS level threads is not very efficient, when the only problem is waiting for some IO and wanting to do something else in the meantime. A more lightweight primitive of concurrency is needed. I am not saying that async is necessarily it.
  - PaulHoule 1y ago
    It's not efficient if you need 100,000 threads. 1,000 threads are almost free... what's expensive is thread switching in cases where parallelism matters. I have pair programmed embarrassingly parallel problems so many times and every time my partner wanted to argue about the need to batch the work and every time we failed to get a speedup the first time, and after adding batching we got close to ideal speedup. People are still hung up on this 1999 article https://www.kegel.com/c10k.html https://www.kegel.com/c10k.html but hardware has moved on. I'm typing this on a machine with 16 CPU cores and I'm more worried about leaving easy parallelism on the table (even when it only gets say a 40% overall speedup) more than I am in 5% overhead from OS threads. I've seen so much buggy code that tries to get it correct with select() and io_uring() and all that but doesn't quite. Google has to handle enough questions to care, and you probably don't. If you worry about these things (right or wrong) Java has your back with virtual threads.
    - zbentley 1y ago
      You're mostly right; computers are fast at core/thread parallelism now and leaving capacity on the table is bad. But there are a lot of common problems where the overhead of threads shows up a lot earlier in the scaling process than you'd think. Probing sets of ~100 URLs to see which are live? Even in a fast, well-threaded language, the overhead of creating/probing/shutting down all those threads adds up to noticeable latency very quickly. Creating lingering threadpools requires pre-planning available thread-parallelism and scheduling (say) concurrent web requests' workloads onto those shared pools with care. Compared to saying "I know these concurrent computations are 99.9% slow IO-wait-bound while probing URLs, let the event loop's epoll take care of it", that's a hassle--and just as hard to get right as manually messing about with select/poll/epoll multiplexers. You're right that there are a lot of buggy IO multiplexer implementations out there; that kind of stuff is best farmed out to a battle-hardened event loop, ideally one that's distributed with your language runtime. But such systems aren't "google scale only", they're needful in a lot of places for ordinary programs' work--whether or not they're ever coded manually or interacted with directly in those programs. If you don't often see problems like the hypothetical URL liveness checker, consider the main socket read/write components of a webserver harness: do you really want to spin up a thread as soon as bytes arrive on a socket to be read/written? Or would you rather allocate a precious request-handler thread (or process, or whatever) only after an efficient IO-multiplexed event loop has parsed enough of a request off of the socket to begin handling it? Plenty of popular webservers take both approaches. The ones that prefer evented I/O for their core request flow are less vulnerable to SlowLoris. There are more benefits to async/await concurrency (and costs, too), including cancellation/timeout semantics, prioritization, and more, but even without getting into those, near-automatic IO multiplexing is pretty huge.
OutOfHere 1y ago
Go lang will happily use multiple cores if the load calls for it, multiple cores are available, and GOMAXPROCS is not restricted to 1. With Python's asyncio, yes, you can read from many client sockets in a core, but you really can't do much work with them in the core, otherwise even the IO will slow down. It is not future-proof at all.
- zbentley 1y ago
  Not for I/O. If many goroutines all request a socket read, the golang runtime shoves them all onto an IO multiplexing event loop very similar to Python's asyncio stdlib event loops: https://www.sobyte.net/post/2022-01/go-netpoller/ https://www.sobyte.net/post/2022-01/go-netpoller/ And sure, Golang's much better than a cooperative concurrency system at giving work out to multiple cores once the IO finishes, no argument there. But again . . . async/await in Python (and JavaScript, Java NIO, and many more) is not about using multiple cores for computations; it's about efficiently making IO concurrent (and, if possible, parallel) for unpredictably "IO-wide" workloads, of which there are many.
  - OutOfHere 1y ago
    Today you want just IO, but tomorrow you will inevitably want some computation too, and later a lot of computation, and then everything will slow down, even the IO, because everything is still running in a single thread. This is how it goes, not just typically, but inevitably always.
  - wahern 1y ago
    > at giving work out to multiple cores once the IO finishes The IO (read, write) doesn't need to finish, just the poll call. That's a very different thing in terms of core utilization, and while it's technically a serialization point, it's not the only serialization point in Go's architecture; it falls out from Go having a single, global run queue, requiring all (idle) threads to serialize on dequeueing tasks. But thank you for pointing that out. TIL, there's a single epoll queue for the whole Go process: https://github.com/golang/go/issues/65064 https://github.com/golang/go/issues/65064
amelius 1y ago
Async is nice in theory, but then some manager asks to do not only IO but also some computation, and there goes the entire plan. Better to use threads from the start because they can be used to manage both types of resource (IO and CPU) in a uniform way.
- OutOfHere 1y ago
  That is precisely what happens. People then jump to absurdities like microservices which severely compound the architecture.

PaulHoule 1y ago

People who talk about there being a hard line between parallelism and concurrency are always writing code with race conditions that they deny exist or writing code with performance bottlenecks they can't understand because they deny they exist. I like working in Java because you can use the same threading primitives for both and have systems that work well in both IO-dominated and CPU-dominated regimes which sometimes happen in the same application under different conditions. Personally there are enough details to work out that we might be up to Python 3.24 when you can really count on all your dependncies to be thread safe. One of the reasons Java has been successful is the extreme xenophobia (not to mention painful JNI) which meant we re-implemented stuff to be thread-safe in pure Java as opposed to sucking in a lot of C/C++ stuff which will never be thread safe.

zelphirkalt 1y ago
Using multiple OS level threads is not very efficient, when the only problem is waiting for some IO and wanting to do something else in the meantime. A more lightweight primitive of concurrency is needed. I am not saying that async is necessarily it.
- PaulHoule 1y ago
  It's not efficient if you need 100,000 threads. 1,000 threads are almost free... what's expensive is thread switching in cases where parallelism matters. I have pair programmed embarrassingly parallel problems so many times and every time my partner wanted to argue about the need to batch the work and every time we failed to get a speedup the first time, and after adding batching we got close to ideal speedup. People are still hung up on this 1999 article https://www.kegel.com/c10k.html https://www.kegel.com/c10k.html but hardware has moved on. I'm typing this on a machine with 16 CPU cores and I'm more worried about leaving easy parallelism on the table (even when it only gets say a 40% overall speedup) more than I am in 5% overhead from OS threads. I've seen so much buggy code that tries to get it correct with select() and io_uring() and all that but doesn't quite. Google has to handle enough questions to care, and you probably don't. If you worry about these things (right or wrong) Java has your back with virtual threads.
  - zbentley 1y ago
    You're mostly right; computers are fast at core/thread parallelism now and leaving capacity on the table is bad. But there are a lot of common problems where the overhead of threads shows up a lot earlier in the scaling process than you'd think. Probing sets of ~100 URLs to see which are live? Even in a fast, well-threaded language, the overhead of creating/probing/shutting down all those threads adds up to noticeable latency very quickly. Creating lingering threadpools requires pre-planning available thread-parallelism and scheduling (say) concurrent web requests' workloads onto those shared pools with care. Compared to saying "I know these concurrent computations are 99.9% slow IO-wait-bound while probing URLs, let the event loop's epoll take care of it", that's a hassle--and just as hard to get right as manually messing about with select/poll/epoll multiplexers. You're right that there are a lot of buggy IO multiplexer implementations out there; that kind of stuff is best farmed out to a battle-hardened event loop, ideally one that's distributed with your language runtime. But such systems aren't "google scale only", they're needful in a lot of places for ordinary programs' work--whether or not they're ever coded manually or interacted with directly in those programs. If you don't often see problems like the hypothetical URL liveness checker, consider the main socket read/write components of a webserver harness: do you really want to spin up a thread as soon as bytes arrive on a socket to be read/written? Or would you rather allocate a precious request-handler thread (or process, or whatever) only after an efficient IO-multiplexed event loop has parsed enough of a request off of the socket to begin handling it? Plenty of popular webservers take both approaches. The ones that prefer evented I/O for their core request flow are less vulnerable to SlowLoris. There are more benefits to async/await concurrency (and costs, too), including cancellation/timeout semantics, prioritization, and more, but even without getting into those, near-automatic IO multiplexing is pretty huge.

OutOfHere 1y ago

Go lang will happily use multiple cores if the load calls for it, multiple cores are available, and GOMAXPROCS is not restricted to 1. With Python's asyncio, yes, you can read from many client sockets in a core, but you really can't do much work with them in the core, otherwise even the IO will slow down. It is not future-proof at all.

zbentley 1y ago
Not for I/O. If many goroutines all request a socket read, the golang runtime shoves them all onto an IO multiplexing event loop very similar to Python's asyncio stdlib event loops: https://www.sobyte.net/post/2022-01/go-netpoller/ https://www.sobyte.net/post/2022-01/go-netpoller/ And sure, Golang's much better than a cooperative concurrency system at giving work out to multiple cores once the IO finishes, no argument there. But again . . . async/await in Python (and JavaScript, Java NIO, and many more) is not about using multiple cores for computations; it's about efficiently making IO concurrent (and, if possible, parallel) for unpredictably "IO-wide" workloads, of which there are many.
- OutOfHere 1y ago
  Today you want just IO, but tomorrow you will inevitably want some computation too, and later a lot of computation, and then everything will slow down, even the IO, because everything is still running in a single thread. This is how it goes, not just typically, but inevitably always.
- wahern 1y ago
  > at giving work out to multiple cores once the IO finishes The IO (read, write) doesn't need to finish, just the poll call. That's a very different thing in terms of core utilization, and while it's technically a serialization point, it's not the only serialization point in Go's architecture; it falls out from Go having a single, global run queue, requiring all (idle) threads to serialize on dequeueing tasks. But thank you for pointing that out. TIL, there's a single epoll queue for the whole Go process: https://github.com/golang/go/issues/65064 https://github.com/golang/go/issues/65064

amelius 1y ago

Async is nice in theory, but then some manager asks to do not only IO but also some computation, and there goes the entire plan. Better to use threads from the start because they can be used to manage both types of resource (IO and CPU) in a uniform way.

OutOfHere 1y ago
That is precisely what happens. People then jump to absurdities like microservices which severely compound the architecture.