7 ms·

I think coroutines are one of those abstractions that are easy to fall in love with, but in reality they do more harm than good. There are extremely few circums

I think coroutines are one of those abstractions that are easy to fall in love with, but in reality they do more harm than good. There are extremely few circumstances where coroutines are useful, and those are not the ones most people use them for. Coroutines are not cheap threads, and if they are used as such, your software will end up either leaking resources or contending on resources under any considerable load. A very simple demonstrative is: yes you can have a million coroutines, but you have 10 database connections. No matter how smart you make the scheduler, you will be bound by your IO resources in almost all applications. (This also hints at the few instances where coroutines can be useful: pure compute). The above problem is exacerbated by coroutines literally undoing one of the most powerful abstractions for resource management: RAII. Your resource allocations stop being tied to lexical scope, and a misplaced yield will leak the resource very easily. And even if you know about this, it is quite difficult to do correct resource management or provide proper backpressure, and the resulting code will be unintuitive and difficult to maintain. You'll need to introduce explicit queues/semaphore-like structures in places where a simple threadpool would have sufficed for all your resource management needs. So what are the examples where coroutines are useful? Pure compute. This is very very rare, but for example one could use coroutines to fuse stream processors and create bounded-memory compute. There was also one paper that described how software transactional memory was used to essentially race coroutines to find optimal circuitry layouts. This kind of compute is kind of it. And this is most definitely not what people are using coroutines for. Instead, they throw around "threads are expensive" and treat Rust async like Java futures, completely missing the point. Sidenote: the particular way that Rust async was adopted, and specifically the tokio family of libraries is a whole other hellhole which sidetracked the entire Rust ecosystem, but admittedly this is not the coroutine concept's fault.

tadfisher 3y ago
Coroutines and concurrency are orthogonal. Coroutines can be used as a primitive in concurrent programs, but the core concept is that of a suspended computation; think replacing callbacks with 'async', not replacing threads with coroutines. For example, coroutines work great in GUI code because typically you have an event loop, and you want to schedule work to return a value (or send an event) later on during some future execution of the loop. This can all be done on a single thread without bringing concurrency into the mix. As far as resource contention, I can provide a couple of counter-examples. In Kotlin, if you know you are performing I/O-bound work, you use the "IO" scheduler, which uses an unbounded thread pool to block a thread per-task. In Java, Project Loom makes this altogether unnecessary by yielding IO-blocked virtual threads for you.
- wglb 3y ago
  Your understanding of the use of coroutines is much superior to the parent post. Think "cooperating sequential processes". If you don't have that, coroutines are likely not fit for the problem.
- aatd86 3y ago
  Is it perhaps a bit more nuanced? Green threads à la Go with channels would be a way to deal with several communicating "event loops" at the same time via coroutines, hence the name goroutines?
- riwsky 3y ago
  You’re confusing concurrency with parallelism. A single-threaded event loop a la node is indeed concurrent. Multiple threads a la the Kotlin IO pool you mention would be both concurrent and parallel. Concurrency isn’t about different parts of the code actually executing at the same time, merely about being able to structure and think about code as if it were executing that way.
- exfalso 3y ago
  I'm not talking about whether the scheduler can block the task on IO. This is trivial. Let's see a couple of examples. Database connection contention: async function() { let connection = getConnection().await; // Allocate resource connection.execute("SELECT 'Hello World!'").await; // <- yield to the scheduler, connection is held onto until continuation is scheduled connection.close().await; // Release resource } This is an extremely trivial example, and this already leaks resources. If you launch 100 of these then there is a chance that there will be 100 open connections at the same time. Had you used a simple thread pool, the size of that pool limits the number of open connections. Memory leak: async function() { let object = stream.readLargeObject().await; <- Memory allocated process1(object).await; <- Memory leaked to scheduler until continuation is scheduled return process2(object).await; } Again, very trivial example, already leaking. process1 and process2 are not tied together with direct control flow edges but rather the flow dispatches through the coroutine scheduler, and they are holding onto the resources while sitting in the scheduler's bookkeeping. Again, yields disrupt the allocation's scope, and if you launch a lot of these coroutines you'll run out of memory. This issue is made worse if you need to use combinations of resource allocations. For example DB connection + memory allocation or network + disk or even network+network are common combos. Depending on the complexity of the allocation nesting you can run into very nasty contention issues or even deadlocks, where - again - a threadpool would have worked well. N threads, N number of resources. Done. I want to stress that there are ways to mitigate the above (queues/resource pools/semaphores), however they are not nearly as intuitive as using a threadpool, and you need to be constantly aware of these leaks when you're writing async code.
verdagon 3y ago
I believe coroutines are mostly unrelated to RAII. One can still have coroutines in a single ownership world and guarantee destructors are called. (If not, I have a lot of redesigning to do for Vale!) Also, doesn't your mentioned problem also happen for full threads? I'd imagine they'd contend for those 10 connections no matter how you did your concurrency.
- exfalso 3y ago
  They do happen with threads indeed, but it's a lot easier to manage and the code is straightforward. You can essentially tie your resources to your threads, and your threadpool's size becomes your resources' allocation limit. In fact one common strategy is to use thread local storage to cache certain resources like RNG or regex statemachines/parsers, and you have essentially a guarantee that you'll be fine(ofc as long as the code doesn't launch threads left and right). Again, I'm not saying that you can't manage this with coroutines, but it is harder, and I've seen this problem pop up in very different contexts/languages(Kotlin/Java, Rust, Haskell) whenever coroutines are used. Large churn or an unexpected bottleneck(e.g. network saturated or disrupted) causing blowup of suspended coroutines that in turn leak resources without applying proper backpressure. When this happens coroutines tend to be harder to debug as well. Plain threads work for disk IO, network IO, memory usage etc etc, and the resource allocations also compose well. E.g. classic flipped order resource alloc: coroutine 1 allocates bounded resource A, then tries to allocate B and yields. Coroutine 2 allocates B, then tries to allocate A, bam, deadlock once A's resource pool is depleted. Essentially suspended coroutines have hidden dependency edges on one another through resources. With threads, the thread pool size aligns precisely with the resource pool size, under high churn it will be the resource's actual slowness that blocks the pool temporarily, but there's no way to deadlock on resource allocations.
  - adrianmsmith 3y ago
    > With threads, the thread pool size aligns precisely with the resource pool size, under high churn it will be the resource's actual slowness that blocks the pool temporarily, but there's no way to deadlock on resource allocations. I'm not sure I see the argument that that's definitely the case. I mean I agree it could be the case. At a previous job we did lots of "enterprise Java" type stuff. Spring Boot (tends to hide what's going on a bit), Tomcat (fixed number of threads to process incoming HTTP requests, I think 200 by default), database connection pool (I think 10 by default in the connection pool that Spring chooses by default). The average programmer on this team did not know about any of these pool sizes. They just wrote code and, for the most part, it worked. But then there was the situation where, for example, one test hung on my machine, but on nobody else's. Turns out (if I remember correctly) that stream.parallel() was used, which processes things on a default thread pool, which is sized to the number of CPU cores by default. My machine had 20 cores, other people had 8 or 10 cores. So they were processing fewer items at once. On my machine this then requested more connections simultaneously than were available in the database connection pool, and I think due to having locked some other resource (again not really obviously if you read the code) then deadlocked on my machine only. As you can imagine it took me a whole afternoon to diagnose this! So what I'm saying is, I agree with everything you've said, but I think these problems can happen just as easily with threads, at least the way there're commonly used in e.g. Java Spring Boot.

tadfisher 3y ago

Coroutines and concurrency are orthogonal. Coroutines can be used as a primitive in concurrent programs, but the core concept is that of a suspended computation; think replacing callbacks with 'async', not replacing threads with coroutines. For example, coroutines work great in GUI code because typically you have an event loop, and you want to schedule work to return a value (or send an event) later on during some future execution of the loop. This can all be done on a single thread without bringing concurrency into the mix. As far as resource contention, I can provide a couple of counter-examples. In Kotlin, if you know you are performing I/O-bound work, you use the "IO" scheduler, which uses an unbounded thread pool to block a thread per-task. In Java, Project Loom makes this altogether unnecessary by yielding IO-blocked virtual threads for you.

wglb 3y ago
Your understanding of the use of coroutines is much superior to the parent post. Think "cooperating sequential processes". If you don't have that, coroutines are likely not fit for the problem.
aatd86 3y ago
Is it perhaps a bit more nuanced? Green threads à la Go with channels would be a way to deal with several communicating "event loops" at the same time via coroutines, hence the name goroutines?
riwsky 3y ago
You’re confusing concurrency with parallelism. A single-threaded event loop a la node is indeed concurrent. Multiple threads a la the Kotlin IO pool you mention would be both concurrent and parallel. Concurrency isn’t about different parts of the code actually executing at the same time, merely about being able to structure and think about code as if it were executing that way.
exfalso 3y ago
I'm not talking about whether the scheduler can block the task on IO. This is trivial. Let's see a couple of examples. Database connection contention: async function() { let connection = getConnection().await; // Allocate resource connection.execute("SELECT 'Hello World!'").await; // <- yield to the scheduler, connection is held onto until continuation is scheduled connection.close().await; // Release resource } This is an extremely trivial example, and this already leaks resources. If you launch 100 of these then there is a chance that there will be 100 open connections at the same time. Had you used a simple thread pool, the size of that pool limits the number of open connections. Memory leak: async function() { let object = stream.readLargeObject().await; <- Memory allocated process1(object).await; <- Memory leaked to scheduler until continuation is scheduled return process2(object).await; } Again, very trivial example, already leaking. process1 and process2 are not tied together with direct control flow edges but rather the flow dispatches through the coroutine scheduler, and they are holding onto the resources while sitting in the scheduler's bookkeeping. Again, yields disrupt the allocation's scope, and if you launch a lot of these coroutines you'll run out of memory. This issue is made worse if you need to use combinations of resource allocations. For example DB connection + memory allocation or network + disk or even network+network are common combos. Depending on the complexity of the allocation nesting you can run into very nasty contention issues or even deadlocks, where - again - a threadpool would have worked well. N threads, N number of resources. Done. I want to stress that there are ways to mitigate the above (queues/resource pools/semaphores), however they are not nearly as intuitive as using a threadpool, and you need to be constantly aware of these leaks when you're writing async code.

verdagon 3y ago

I believe coroutines are mostly unrelated to RAII. One can still have coroutines in a single ownership world and guarantee destructors are called. (If not, I have a lot of redesigning to do for Vale!) Also, doesn't your mentioned problem also happen for full threads? I'd imagine they'd contend for those 10 connections no matter how you did your concurrency.

exfalso 3y ago
They do happen with threads indeed, but it's a lot easier to manage and the code is straightforward. You can essentially tie your resources to your threads, and your threadpool's size becomes your resources' allocation limit. In fact one common strategy is to use thread local storage to cache certain resources like RNG or regex statemachines/parsers, and you have essentially a guarantee that you'll be fine(ofc as long as the code doesn't launch threads left and right). Again, I'm not saying that you can't manage this with coroutines, but it is harder, and I've seen this problem pop up in very different contexts/languages(Kotlin/Java, Rust, Haskell) whenever coroutines are used. Large churn or an unexpected bottleneck(e.g. network saturated or disrupted) causing blowup of suspended coroutines that in turn leak resources without applying proper backpressure. When this happens coroutines tend to be harder to debug as well. Plain threads work for disk IO, network IO, memory usage etc etc, and the resource allocations also compose well. E.g. classic flipped order resource alloc: coroutine 1 allocates bounded resource A, then tries to allocate B and yields. Coroutine 2 allocates B, then tries to allocate A, bam, deadlock once A's resource pool is depleted. Essentially suspended coroutines have hidden dependency edges on one another through resources. With threads, the thread pool size aligns precisely with the resource pool size, under high churn it will be the resource's actual slowness that blocks the pool temporarily, but there's no way to deadlock on resource allocations.
- adrianmsmith 3y ago
  > With threads, the thread pool size aligns precisely with the resource pool size, under high churn it will be the resource's actual slowness that blocks the pool temporarily, but there's no way to deadlock on resource allocations. I'm not sure I see the argument that that's definitely the case. I mean I agree it could be the case. At a previous job we did lots of "enterprise Java" type stuff. Spring Boot (tends to hide what's going on a bit), Tomcat (fixed number of threads to process incoming HTTP requests, I think 200 by default), database connection pool (I think 10 by default in the connection pool that Spring chooses by default). The average programmer on this team did not know about any of these pool sizes. They just wrote code and, for the most part, it worked. But then there was the situation where, for example, one test hung on my machine, but on nobody else's. Turns out (if I remember correctly) that stream.parallel() was used, which processes things on a default thread pool, which is sized to the number of CPU cores by default. My machine had 20 cores, other people had 8 or 10 cores. So they were processing fewer items at once. On my machine this then requested more connections simultaneously than were available in the database connection pool, and I think due to having locked some other resource (again not really obviously if you read the code) then deadlocked on my machine only. As you can imagine it took me a whole afternoon to diagnose this! So what I'm saying is, I agree with everything you've said, but I think these problems can happen just as easily with threads, at least the way there're commonly used in e.g. Java Spring Boot.