5 ms·

Coroutines are awesome, don't get me wrong. BUT, contrary to what I believed when I was still working with Lua, making the coroutine explicit (like async/await

Coroutines are awesome, don't get me wrong. BUT, contrary to what I believed when I was still working with Lua, making the coroutine explicit (like async/await in C# and JavaScript ES2017) is actually even better. In particular, when your coroutines are explicit, you can say "queue up this task, this task, and this task, and continue here when they're ALL done." So when you're dealing with several tasks that might take different amounts of time to complete, they can do the work in parallel. I went from being a major Lua fanboy and posting frequently to the mailing list to drifting away from involvement to abandoning the language, in a relatively short period of time. I even wrote a blog entry about it. [1] I still think Lua has better "bones" than JavaScript, but with TypeScript, async/await, and JavaScript JIT engines everywhere, TypeScript just wins for me. [1] https://realmensch.org/2016/05/28/goodbye-lua/ https://realmensch.org/2016/05/28/goodbye-lua/

daurnimator 10y ago
The problem with making coroutines (like async/await in JS) is http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... This is one of the reasons I love lua's coroutines.
- unscaled 10y ago
  I assume this is exactly what SomeCallMeTim meant. Yes, explicitly awaitable futures/promises are slightly less ergonomic than coroutines, but this is a very reasonable trade-off. You give up: 1. Extra cognitive load every time you need to await. 2. You've got to be more careful when designing some generic higher-order functions (you need to decide which color their function arguments should be). In return you get: 1. Explicit concurrency. No function that you call unexpectedly suspends behind your back. In some cases this explicitness could be actually important (e.g. when using per-thread/process shared memory). 2. Futures/promises/tasks are first-class values. You could pass them along, save them for later, await on all or any of them and so forth. 3. Theoretically speaking, promises can be more memory efficient than coroutines since you don't need to allocate a full stack for each of them - just the state you need. "What Color Is Your Function" is misleading: there is no strictly superior solution for this tradeoff. Futures/promises encode concurrency as a monad and like every monad they are viral and once you start propagating them around, they can be a pain. But the alternative for not encoding some concern of your program as a monad is to let it pass through an implicit side-channel (I/O, error-handling, global state, built-in scheduler) that is harder to control and can't be treated as first-class object in the language. Go itself chose to go the explicit with error-handling. It doesn't even uses Monads, so it gets the worst of both worlds in my opinion, but the designers' main concern with exceptions which they managed to avert is still valid: with (unchecked) exceptions you never know which function you call is going to raise an error on you somewhere down the line. So yes, with Go you'll never have with unexpected throws, but you'll have to deal with unexpected goroutine suspension. In Go all of your functions will be happily colorless with regards to concurrency but will be colored with regards to error-handling. Again, both choices here are valid, but they are not so brain-dead simple or universal as this article is trying to put them.
  - SomeCallMeTim 10y ago
    Very well put, thanks. Explicit promises have very nice, composable behaviors that I'm exploiting all the time, and that have no equivalent in the Lua coroutine world. And once you have propagated them around, I think the ergonomics are just fine; I've taken some pretty hairy callback logic that no one could fully understand without careful study and turned it into async/await code that pretty much anyone could follow. I think the implicitness of "this could suspend" does frequently cause serious problems; in Lua this shows up as the dreaded "Attempt to yield across a metamethod C call boundary" [1] bug. Talk about cognitive load! If you're calling C from Lua, and that C code then calls back into Lua, and the latter function yields, you get this error. So you have to be very careful about what is allowed to yield -- and since any function in Lua, at any point in the call stack, can yield, it's easy to accidentally shoot yourself in the foot. [1] http://stackoverflow.com/questions/8459459/lua-coroutine-error-tempt-to-yield-across-metamethod-c-call-boundary http://stackoverflow.com/questions/8459459/lua-coroutine-err...
  - anonymoushn 10y ago
    > 2. Futures/promises/tasks are first-class values. You could pass them along, save them for later, await on all or any of them and so forth. Can you explain why this is unachievable other than by tagging your yield-capable call sites?
    - unscaled 10y ago
      You can acheive some composability (e.g. easily awaiting multiple events) by having first-class values for the coroutines/green-threads themselves. This is what greenlet/gevent does, and it's rather pleasant to use. Try doing something like gevent.joinall() on Go though. https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/ https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-a... Not fun. The other problem is that even with gevent, what you get is a first-class _coroutine_, not a first class class _continuation_. You have to actually spawn new coroutines (creating new stacks and going through sometimes unnecessary scheduler hoops) to get a first-class coroutine out of a possibly blocking function. Again, I'm not saying you can't do something like gevent.joinall(gevent.spawn(foo()), gevent.spawn(bar()), but: 1. Many languages don't support this (including Go and Lua, AFAIK). 2. It comes with its own tradeoffs again.
anonymoushn 10y ago
The async/await model and the coroutine model are equally capable of expressing "start these 3 tasks and wake me up when they're all done." I've written this before in Python using Greenlet.
- SomeCallMeTim 10y ago
  Lua coroutines don't have that ability out of the box. JavaScript promises do (Promise.all).
  - deleted 10y ago
    [deleted]
  - anonymoushn 10y ago
    This is an interesting characterization of the situation! I can save the 20 LOC involved in implementing this if I just give up the ability to write my own event loops and pepper my code with a bunch of additional "await"s every time I add an asynchronous operation to a function used by other functions.
    - unscaled 10y ago
      You can also implement your own event loops with promises, but it's a matter of tradeoffs again: You can't control the way functions written by someone else will schedule their internal continuations - you can only control the way you schedule your own continuation. I'm not sure if Javascript gives you a lot of control there, but C# allows you to define your own TaskScheduler and specify it on ContinueWith.

daurnimator 10y ago

The problem with making coroutines (like async/await in JS) is http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... This is one of the reasons I love lua's coroutines.

unscaled 10y ago
I assume this is exactly what SomeCallMeTim meant. Yes, explicitly awaitable futures/promises are slightly less ergonomic than coroutines, but this is a very reasonable trade-off. You give up: 1. Extra cognitive load every time you need to await. 2. You've got to be more careful when designing some generic higher-order functions (you need to decide which color their function arguments should be). In return you get: 1. Explicit concurrency. No function that you call unexpectedly suspends behind your back. In some cases this explicitness could be actually important (e.g. when using per-thread/process shared memory). 2. Futures/promises/tasks are first-class values. You could pass them along, save them for later, await on all or any of them and so forth. 3. Theoretically speaking, promises can be more memory efficient than coroutines since you don't need to allocate a full stack for each of them - just the state you need. "What Color Is Your Function" is misleading: there is no strictly superior solution for this tradeoff. Futures/promises encode concurrency as a monad and like every monad they are viral and once you start propagating them around, they can be a pain. But the alternative for not encoding some concern of your program as a monad is to let it pass through an implicit side-channel (I/O, error-handling, global state, built-in scheduler) that is harder to control and can't be treated as first-class object in the language. Go itself chose to go the explicit with error-handling. It doesn't even uses Monads, so it gets the worst of both worlds in my opinion, but the designers' main concern with exceptions which they managed to avert is still valid: with (unchecked) exceptions you never know which function you call is going to raise an error on you somewhere down the line. So yes, with Go you'll never have with unexpected throws, but you'll have to deal with unexpected goroutine suspension. In Go all of your functions will be happily colorless with regards to concurrency but will be colored with regards to error-handling. Again, both choices here are valid, but they are not so brain-dead simple or universal as this article is trying to put them.
- SomeCallMeTim 10y ago
  Very well put, thanks. Explicit promises have very nice, composable behaviors that I'm exploiting all the time, and that have no equivalent in the Lua coroutine world. And once you have propagated them around, I think the ergonomics are just fine; I've taken some pretty hairy callback logic that no one could fully understand without careful study and turned it into async/await code that pretty much anyone could follow. I think the implicitness of "this could suspend" does frequently cause serious problems; in Lua this shows up as the dreaded "Attempt to yield across a metamethod C call boundary" [1] bug. Talk about cognitive load! If you're calling C from Lua, and that C code then calls back into Lua, and the latter function yields, you get this error. So you have to be very careful about what is allowed to yield -- and since any function in Lua, at any point in the call stack, can yield, it's easy to accidentally shoot yourself in the foot. [1] http://stackoverflow.com/questions/8459459/lua-coroutine-error-tempt-to-yield-across-metamethod-c-call-boundary http://stackoverflow.com/questions/8459459/lua-coroutine-err...
- anonymoushn 10y ago
  > 2. Futures/promises/tasks are first-class values. You could pass them along, save them for later, await on all or any of them and so forth. Can you explain why this is unachievable other than by tagging your yield-capable call sites?
  - unscaled 10y ago
    You can acheive some composability (e.g. easily awaiting multiple events) by having first-class values for the coroutines/green-threads themselves. This is what greenlet/gevent does, and it's rather pleasant to use. Try doing something like gevent.joinall() on Go though. https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/ https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-a... Not fun. The other problem is that even with gevent, what you get is a first-class _coroutine_, not a first class class _continuation_. You have to actually spawn new coroutines (creating new stacks and going through sometimes unnecessary scheduler hoops) to get a first-class coroutine out of a possibly blocking function. Again, I'm not saying you can't do something like gevent.joinall(gevent.spawn(foo()), gevent.spawn(bar()), but: 1. Many languages don't support this (including Go and Lua, AFAIK). 2. It comes with its own tradeoffs again.

anonymoushn 10y ago

The async/await model and the coroutine model are equally capable of expressing "start these 3 tasks and wake me up when they're all done." I've written this before in Python using Greenlet.

SomeCallMeTim 10y ago
Lua coroutines don't have that ability out of the box. JavaScript promises do (Promise.all).
- deleted 10y ago
  [deleted]
- anonymoushn 10y ago
  This is an interesting characterization of the situation! I can save the 20 LOC involved in implementing this if I just give up the ability to write my own event loops and pepper my code with a bunch of additional "await"s every time I add an asynchronous operation to a function used by other functions.
  - unscaled 10y ago
    You can also implement your own event loops with promises, but it's a matter of tradeoffs again: You can't control the way functions written by someone else will schedule their internal continuations - you can only control the way you schedule your own continuation. I'm not sure if Javascript gives you a lot of control there, but C# allows you to define your own TaskScheduler and specify it on ContinueWith.