4 ms·

This is how I understand the two approaches to concurrency: 1) Use preemptive system threads that can execute in parallel. A task requiring simultaneous waitin

This is how I understand the two approaches to concurrency: 1) Use preemptive system threads that can execute in parallel. A task requiring simultaneous waiting is given an operating system thread of its own so it can block without stopping the entire program. But threads require significant memory and other resources per thread. Also, the operating system can arbitrarily interleave the execution of system threads, requiring the programmer to carefully protect shared resources with locks and condition variables, which is exceedingly error-prone. 2) Have a single-threaded program, where that single thread runs an event loop whose job is to react to external events by invoking a callback function that has been registered. While it doesn't require the same kind of complex synchronization that preemptive threads do, the inverted control structure of this approach requires your own control flow to thread awkwardly through the system's event loop, leading to a maze of event callbacks. Coroutines work with the latter style in an attempt to tame its complexity. They are themselves complex, however, and, in my opinion, that complexity doesn't pull its weight when you consider in the end you only have one thread. Can anyone tell me what I'm missing or how these problems with these approaches have been solved in places?

snissn 9y ago
You can have 2) with the additional complexity / upside of multiple threads
maxpolun 9y ago
You can have multiple threads with coroutines. However you generally only have one thread per CPU/hardware thread, and use async io within that thread. This is the general architecture of nginx for example (though it doesn't use coroutines, just callback based async io).
strictfp 9y ago
Or 3) you do like golang and erlang and build really lightweight green threads so you get the programming model of approach 1 and nearly the same speed as number 2. This of course requires extra effort to prevent blocking of OS threads and in the case where the OS expects thread locality. But IMHO it's the superior approach. The best solution would be if OSes themselves would provide really lightweight threads. And I suspect that they will, eventually.
- nickbauman 9y ago
  I haven't done Erlang, but in my experience Golang has much the same problems as 1) so I'm not sure I'd agree. +1 to lightweight threads, tho.
  - simplify 9y ago
    Phoenix/Elixir can handle 2 million websocket connections on a single box. I would agree myself :) http://www.phoenixframework.org/blog/the-road-to-2-million-websocket-connections http://www.phoenixframework.org/blog/the-road-to-2-million-w...
    - grogenaut 9y ago
      Wouldn't you need something like 40 NICs to do that?
      - phamilton 9y ago
        I believe you are referring to running out of ports, as 65k available ports * 40 ~= 2M. The limitation is actually on a quad tuple: {src_ip, src_port, dest_ip, dest_port} As such, the actual limit is outrageously high and isn't really a problem. Interestingly enough, the referenced Phoenix benchmark overlooked the dest component and hit a limit as they had only 40 or so machines to use in the test. Had they used multiple ips on the server it would have given them a lot more breathing room. EDIT: Multiple IPs != Multiple NICs. You can assign multiple IPs to a single physical interface. Also, they could have used multiple ports to accomplish the same effect.
        grogenaut 9y ago
        Good to know. I thought you were always limited by the outgoing port range to 65k - (used ports) good to know. So they were limited by the testing cluster or the tested server?
        9y ago
KayEss 9y ago
The implementation of the coroutines might be complex, but their use is very straightforward -- the code looks almost exactly the same as the blocking code would, just with the awaits in there. As for the number of threads, when I use Boost ASIO with coroutines I often end up with multiple threads servicing a pool of coroutines, so if there is shared state between them then there is still synchronisation. I use channels implemented on top of eventfd to help with that. When I converted some network code from callbacks to coroutines the code ended up about 1/3 as long and was far simpler and easier to understand. It also fixed several bugs I simply couldn't find. The reality is that the callbacks are far more complex to use than the coroutines.
- vvanders 9y ago
  Yup, we used to let our designers loose on coroutines as a way to script Game AI. If designers can use them safely then I'm sure the average programmer can too :).
butterisgood 9y ago
Concurrency is about how you organize a solution really more than anything. Whether those concurrent tasks execute in parallel or not is a different consideration. Many authors and experts conflate these terms. You can write concurrent code with pthreads. It may happen to execute in parallel but it doesn't have to. In fact, I believe the early SunOS/Solaris pthreads had a 'green' scheduler where it could all be done in userspace (this is from my own memory nearly 20 years ago). This made them basically "stack switchers" which is what some lightweight coroutine libraries end up doing. Having written heavily callback-driven code. Threading contexts manually, thinking about object lifetimes, I can say that coroutines have a much cleaner flow to them and can make it easier to write and later read and debug programs than crazy callbacks and contexts. Object lifetimes are one thing coroutines help with in a big way. I think it's why Rust is so attractive to so many. Look at plan 9's libthread for example: https://github.com/9fans/plan9port/tree/master/src/libthread https://github.com/9fans/plan9port/tree/master/src/libthread It allows one to take advantage of nonblocking sockets as a single "kernel execution entity" mechanism across many userspace threads. This approach works pretty well. The Mongrel2 webserver clearly borrows this approach in their task library: https://github.com/9fans/plan9port/blob/master/src/libthread/386-ucontext.h https://github.com/9fans/plan9port/blob/master/src/libthread... https://github.com/mongrel2/mongrel2/blob/master/src/task/386-ucontext.h https://github.com/mongrel2/mongrel2/blob/master/src/task/38...
- mhink 9y ago
  > Having written heavily callback-driven code. Threading contexts manually, thinking about object lifetimes, I can say that coroutines have a much cleaner flow to them and can make it easier to write and later read and debug programs than crazy callbacks and contexts. This is the biggest reason I vastly prefer using coroutines in my Javascript projects (by way of `redux-saga` if there's heavy lifting, or just async/await otherwise). > Concurrency is about how you organize a solution really more than anything. You hit the nail on the head here. IMHO, business logic should be expressed procedurally, whereas driver code in an event-loop driven environment (like V8) is more suited to a functional style. Coroutines let you do both, quite cleanly. Thanks for the links, btw, I've been wanting to dig through other code like this in more established languages. :)
missblit 9y ago
I found them useful when using the Boost Asio C++ networking library. I needed to read some bytes, make a decision, read some more bytes, make another decision, etc. This started out as callback hell since every read was asynchronous, which was very painful, but after using the coroutine support I was able to write the code as if it was synchronous and the logic became a lot easier to follow. And the coroutines weren't that hard to use, even lacking proper language support. I could basically write something like async_read(socket, buffer, yield_context);. The coroutine yielded behind the scenes and the buffer was populated once control flow was returned to the next line (from who knows where!)
CupOfJava 9y ago
The fundamental difference between single threading, coroutines, and multi threading: Single Thread: 1 thread of execution, 1 execution stack Coroutine: 1 thread of execution, multiple execution stacks Multi threading: multiple threads of execution, multiple execution stacks
- gpderetta 9y ago
  FYI each coroutine is normally considered its own thread of execution (which is a distinct thing from a kernel scheduled thread). Coroutines are distinct from threads as they normally lack a scheduler (the decision to which coroutine run next is either made statically ot explicitly by the coroutine); coroutines meant to be used on top of an userspace scheduler are often called fibers (or simply userspace, or green, threads).
zackmorris 9y ago
I'll take a crack at this - coroutines are not about performance. They are for converting callbacks to a single thread of execution that's easier to reason about (which could themselves be embedded in other threads). Imperative logic is equivalent to a state machine but again much easier for humans to reason about, that's why developers got so much more done in the past with synchronous execution and macros than they do today as we struggle to build even the simplest user interfaces. I'm hopeful that we can get all of the benefits of 1980s cooperative threading but that the decades of experience since allow us to avoid the pitfalls.
com2kid 9y ago
> 2) Have a single-threaded program, where that single thread runs an event loop whose job is to react to external events by invoking a callback function that has been registered. While it doesn't require the same kind of complex synchronization that preemptive threads do, the inverted control structure of this approach requires your own control flow to thread awkwardly through the system's event loop, leading to a maze of event callbacks. Co-routines and callback based programming are great for dealing with I/O and for offloading "work" to black boxes, those black boxes can very well be other threads, CPU cores, hardware, or another computer. When the work is complete, your callback is called, or the co-routine continues. It is perfectly possible to be juggling multiple tasks this way, but have all the results appear in sequential order on one thread, and not have to deal with the complexity of real multi-threading. This only works for tasks that are easily offloadable, you need VERY clean data boundaries, minimal shared state, and well defined points where you hand off control. In other words, a well engineered program. :)
bonzini 9y ago
Coroutines may also require mutexes. Usually they are very coarse-grained, but the usecases for coroutine mutexes and wait queues exist. These are not pthread mutexes, of course, they would be part of the coroutine runtime or hand coded. For example, you could have coroutines that write to a shared resource (such as a socket) where interleaving the data from two sources would be a problem.
int_19h 9y ago
Event loop + callbacks allows you to fully utilize any given thread (meaning that while there's outstanding I/O or other long-running operation, the thread isn't idling waiting for it to complete, but instead performing some useful work). This is completely orthogonal to multithreading - if you have multiple threads, there's still benefit in fully utilizing them, and so each thread gets event loop and callbacks. It's true that some popular frameworks in this vein (notably, Node.js) combines event loop with single thread per process, but it's not a prerequisite. In C#, async/await is commonly used with multiple threads, for example in ASP.NET web apps.
dom0 9y ago
2) is concurrency, 1) is parallelism; there are various programming models for multiple threads. The basic approach of mutual exclusion (=shared memory, locks, barriers) is just one of them, another approach would be message passing / actors (queues, ZeroMQ etc.). You can mix the two, and the latter typically is implemented on top of the former, because hardware does not have message passing facilities. Some other approaches don't really fit that scheme (e.g. TBB). Choosing an approach in this field is not always obvious, though sometimes it is. E.g. many stupid server workloads are bound on I/O concurrency, so green threads are the best fit. However, as soon as CPU scaling is required, multiple independently schedulable kernel tasks are required -- you either have to fork/CreateProcess or spawn threads. This does not necessarily imply that larger parts of the application have to be aware of this. E.g. it's entirely possible to plug a behind-the-scenes CPU thread into an event loop.
Matthias247 9y ago
You can also have mixtures of those approaches in languages which support multiple threads. E.g. you could have one thread which runs an eventloop and async operations and possibly coroutines. And other threads where a pure blocking programming model is used. From these threads you could e.g. post some work to the eventloop thread and wait for it to complete. The Future/Task abstractions that are found in multithreaded programming languages (like C# or Java) also reflect this possibilities: E.g. they allow to specify on which thread a continuation should be run instead of just assuming the current thread. They also need synchronisation internally, because multiple threads might want to use the Future object. From my personal experience the things that work well are pure singlethreaded models with an async style (like in Javascript/node.js) or multithreaded but pure synchronous models without callbacks (as popular in Go). The mixture (multithreading with callbacks, which might be invoked on any thread) often leads to confusion how an API should be used correctly.

snissn 9y ago

You can have 2) with the additional complexity / upside of multiple threads

maxpolun 9y ago

You can have multiple threads with coroutines. However you generally only have one thread per CPU/hardware thread, and use async io within that thread. This is the general architecture of nginx for example (though it doesn't use coroutines, just callback based async io).

strictfp 9y ago

Or 3) you do like golang and erlang and build really lightweight green threads so you get the programming model of approach 1 and nearly the same speed as number 2. This of course requires extra effort to prevent blocking of OS threads and in the case where the OS expects thread locality. But IMHO it's the superior approach. The best solution would be if OSes themselves would provide really lightweight threads. And I suspect that they will, eventually.

nickbauman 9y ago
I haven't done Erlang, but in my experience Golang has much the same problems as 1) so I'm not sure I'd agree. +1 to lightweight threads, tho.
- simplify 9y ago
  Phoenix/Elixir can handle 2 million websocket connections on a single box. I would agree myself :) http://www.phoenixframework.org/blog/the-road-to-2-million-websocket-connections http://www.phoenixframework.org/blog/the-road-to-2-million-w...
  - grogenaut 9y ago
    Wouldn't you need something like 40 NICs to do that?
    - phamilton 9y ago
      I believe you are referring to running out of ports, as 65k available ports * 40 ~= 2M. The limitation is actually on a quad tuple: {src_ip, src_port, dest_ip, dest_port} As such, the actual limit is outrageously high and isn't really a problem. Interestingly enough, the referenced Phoenix benchmark overlooked the dest component and hit a limit as they had only 40 or so machines to use in the test. Had they used multiple ips on the server it would have given them a lot more breathing room. EDIT: Multiple IPs != Multiple NICs. You can assign multiple IPs to a single physical interface. Also, they could have used multiple ports to accomplish the same effect.
      - grogenaut 9y ago
        Good to know. I thought you were always limited by the outgoing port range to 65k - (used ports) good to know. So they were limited by the testing cluster or the tested server?
        9y ago

KayEss 9y ago

The implementation of the coroutines might be complex, but their use is very straightforward -- the code looks almost exactly the same as the blocking code would, just with the awaits in there. As for the number of threads, when I use Boost ASIO with coroutines I often end up with multiple threads servicing a pool of coroutines, so if there is shared state between them then there is still synchronisation. I use channels implemented on top of eventfd to help with that. When I converted some network code from callbacks to coroutines the code ended up about 1/3 as long and was far simpler and easier to understand. It also fixed several bugs I simply couldn't find. The reality is that the callbacks are far more complex to use than the coroutines.

vvanders 9y ago
Yup, we used to let our designers loose on coroutines as a way to script Game AI. If designers can use them safely then I'm sure the average programmer can too :).

butterisgood 9y ago

Concurrency is about how you organize a solution really more than anything. Whether those concurrent tasks execute in parallel or not is a different consideration. Many authors and experts conflate these terms. You can write concurrent code with pthreads. It may happen to execute in parallel but it doesn't have to. In fact, I believe the early SunOS/Solaris pthreads had a 'green' scheduler where it could all be done in userspace (this is from my own memory nearly 20 years ago). This made them basically "stack switchers" which is what some lightweight coroutine libraries end up doing. Having written heavily callback-driven code. Threading contexts manually, thinking about object lifetimes, I can say that coroutines have a much cleaner flow to them and can make it easier to write and later read and debug programs than crazy callbacks and contexts. Object lifetimes are one thing coroutines help with in a big way. I think it's why Rust is so attractive to so many. Look at plan 9's libthread for example: https://github.com/9fans/plan9port/tree/master/src/libthread https://github.com/9fans/plan9port/tree/master/src/libthread It allows one to take advantage of nonblocking sockets as a single "kernel execution entity" mechanism across many userspace threads. This approach works pretty well. The Mongrel2 webserver clearly borrows this approach in their task library: https://github.com/9fans/plan9port/blob/master/src/libthread/386-ucontext.h https://github.com/9fans/plan9port/blob/master/src/libthread... https://github.com/mongrel2/mongrel2/blob/master/src/task/386-ucontext.h https://github.com/mongrel2/mongrel2/blob/master/src/task/38...

mhink 9y ago
> Having written heavily callback-driven code. Threading contexts manually, thinking about object lifetimes, I can say that coroutines have a much cleaner flow to them and can make it easier to write and later read and debug programs than crazy callbacks and contexts. This is the biggest reason I vastly prefer using coroutines in my Javascript projects (by way of `redux-saga` if there's heavy lifting, or just async/await otherwise). > Concurrency is about how you organize a solution really more than anything. You hit the nail on the head here. IMHO, business logic should be expressed procedurally, whereas driver code in an event-loop driven environment (like V8) is more suited to a functional style. Coroutines let you do both, quite cleanly. Thanks for the links, btw, I've been wanting to dig through other code like this in more established languages. :)

missblit 9y ago

I found them useful when using the Boost Asio C++ networking library. I needed to read some bytes, make a decision, read some more bytes, make another decision, etc. This started out as callback hell since every read was asynchronous, which was very painful, but after using the coroutine support I was able to write the code as if it was synchronous and the logic became a lot easier to follow. And the coroutines weren't that hard to use, even lacking proper language support. I could basically write something like async_read(socket, buffer, yield_context);. The coroutine yielded behind the scenes and the buffer was populated once control flow was returned to the next line (from who knows where!)

CupOfJava 9y ago

The fundamental difference between single threading, coroutines, and multi threading: Single Thread: 1 thread of execution, 1 execution stack Coroutine: 1 thread of execution, multiple execution stacks Multi threading: multiple threads of execution, multiple execution stacks

gpderetta 9y ago
FYI each coroutine is normally considered its own thread of execution (which is a distinct thing from a kernel scheduled thread). Coroutines are distinct from threads as they normally lack a scheduler (the decision to which coroutine run next is either made statically ot explicitly by the coroutine); coroutines meant to be used on top of an userspace scheduler are often called fibers (or simply userspace, or green, threads).

zackmorris 9y ago

I'll take a crack at this - coroutines are not about performance. They are for converting callbacks to a single thread of execution that's easier to reason about (which could themselves be embedded in other threads). Imperative logic is equivalent to a state machine but again much easier for humans to reason about, that's why developers got so much more done in the past with synchronous execution and macros than they do today as we struggle to build even the simplest user interfaces. I'm hopeful that we can get all of the benefits of 1980s cooperative threading but that the decades of experience since allow us to avoid the pitfalls.

com2kid 9y ago

> 2) Have a single-threaded program, where that single thread runs an event loop whose job is to react to external events by invoking a callback function that has been registered. While it doesn't require the same kind of complex synchronization that preemptive threads do, the inverted control structure of this approach requires your own control flow to thread awkwardly through the system's event loop, leading to a maze of event callbacks. Co-routines and callback based programming are great for dealing with I/O and for offloading "work" to black boxes, those black boxes can very well be other threads, CPU cores, hardware, or another computer. When the work is complete, your callback is called, or the co-routine continues. It is perfectly possible to be juggling multiple tasks this way, but have all the results appear in sequential order on one thread, and not have to deal with the complexity of real multi-threading. This only works for tasks that are easily offloadable, you need VERY clean data boundaries, minimal shared state, and well defined points where you hand off control. In other words, a well engineered program. :)

bonzini 9y ago

Coroutines may also require mutexes. Usually they are very coarse-grained, but the usecases for coroutine mutexes and wait queues exist. These are not pthread mutexes, of course, they would be part of the coroutine runtime or hand coded. For example, you could have coroutines that write to a shared resource (such as a socket) where interleaving the data from two sources would be a problem.

int_19h 9y ago

Event loop + callbacks allows you to fully utilize any given thread (meaning that while there's outstanding I/O or other long-running operation, the thread isn't idling waiting for it to complete, but instead performing some useful work). This is completely orthogonal to multithreading - if you have multiple threads, there's still benefit in fully utilizing them, and so each thread gets event loop and callbacks. It's true that some popular frameworks in this vein (notably, Node.js) combines event loop with single thread per process, but it's not a prerequisite. In C#, async/await is commonly used with multiple threads, for example in ASP.NET web apps.

dom0 9y ago

2) is concurrency, 1) is parallelism; there are various programming models for multiple threads. The basic approach of mutual exclusion (=shared memory, locks, barriers) is just one of them, another approach would be message passing / actors (queues, ZeroMQ etc.). You can mix the two, and the latter typically is implemented on top of the former, because hardware does not have message passing facilities. Some other approaches don't really fit that scheme (e.g. TBB). Choosing an approach in this field is not always obvious, though sometimes it is. E.g. many stupid server workloads are bound on I/O concurrency, so green threads are the best fit. However, as soon as CPU scaling is required, multiple independently schedulable kernel tasks are required -- you either have to fork/CreateProcess or spawn threads. This does not necessarily imply that larger parts of the application have to be aware of this. E.g. it's entirely possible to plug a behind-the-scenes CPU thread into an event loop.

Matthias247 9y ago

You can also have mixtures of those approaches in languages which support multiple threads. E.g. you could have one thread which runs an eventloop and async operations and possibly coroutines. And other threads where a pure blocking programming model is used. From these threads you could e.g. post some work to the eventloop thread and wait for it to complete. The Future/Task abstractions that are found in multithreaded programming languages (like C# or Java) also reflect this possibilities: E.g. they allow to specify on which thread a continuation should be run instead of just assuming the current thread. They also need synchronisation internally, because multiple threads might want to use the Future object. From my personal experience the things that work well are pure singlethreaded models with an async style (like in Javascript/node.js) or multithreaded but pure synchronous models without callbacks (as popular in Go). The mixture (multithreading with callbacks, which might be invoked on any thread) often leads to confusion how an API should be used correctly.