5 ms·

I'll admit I find coroutines difficult to grok. It seems to me that 'callback hell' is turning into 'coroutine hell'. The only plausible use-case I can see is e

I'll admit I find coroutines difficult to grok. It seems to me that 'callback hell' is turning into 'coroutine hell'. The only plausible use-case I can see is enabling functionality similar to that of Python's `yield`. Does threadpool::thread_loop() not have to check if the popped coroutine is suspended before attempting to resume it? Are they really more efficient than normal callbacks when doing async?

drenvuk 5y ago
You're not the only one. Coroutines are complicated as hell and have too much boiler plate BUT once you handle it for a general enough case you get javascript-esque async await syntax which is very, very nice. Take for instance, this code which relies on libuv for its event loop and co_await to retain its state during its execution: https://gist.github.com/Qix-/09532acd0f6c9a57c09bd9ce31b3023f https://gist.github.com/Qix-/09532acd0f6c9a57c09bd9ce31b3023... Lets say that you want to batch a bunch of database operations into one transaction. You could queue them up over the course of a few milliseconds, run the transaction, and then for each context that relied on different db operations simply return to each's previous point instead of having to call a handler. Granted, the handler is now inside of the `await_transform` needed to work with `co_await` but think of the possibilities. No weirdly separate callback function, no real need to make a class that encapsulates all of the operations for let's say a user's post request, and to top it all off, you can do this on a single thread. It's a tool for cleaner code but I'll be damned if it is really easy to understand. It's just so much stupid boiler plate and a strange way of putting it together.
- asdfasgasdgasdg 5y ago
  It still boggles the mind that they made it so hard to use this stuff in terms of boilerplate. Hopefully that will all be abstracted into a library that handles io, networking, multiprocessing and synchronization so most of us can just focus on writing the bits that do stuff. But I will never understand how the std maintainers could not manage to do this when every other language is doing async await support in their standard library Io utilities.
  - PaulDavisThe1st 5y ago
    One reason could be that C++ tends to be used more in performance-critical contexts, and frequently the abstractions present in "every other language" can get in the way of performance.
    - cobalt 5y ago
      also the memory management model in c++ is much more complex - most other languages just use garbage collection to deal with reference lifetimes
      - PaulDavisThe1st 5y ago
        garbage collection is one of the abstractions that can interfere with performance. it's out of the question for real time contexts, for example.
        pjmlp 5y ago
        Depends, https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/wp/products-services/jamaicavm/ https://www.aicas.com/wp/products-services/jamaicavm/ On the other hand there is real time where even malloc() is forbidden. https://en.m.wikipedia.org/wiki/MISRA_C https://en.m.wikipedia.org/wiki/MISRA_C
        hedora 5y ago
        The first one targets “sub-millisecond”, which is a eternity in current systems. According to the third link: > we observed a negative correlation between MISRA rule violations and observed faults So, MISRA isn’t really helping with real time at all. The second link claims pauses are capped at 10us, which (if true) is actually competing with careful use of malloc.
        pjmlp 5y ago
contravariant 5y ago
For what it's worth Python's yield pretty much is a coroutine, as it has a (very rarely used) feature that allows information from the caller to be passed back to the generator. The c++ implementation seems closer to lisps 'call with current continuation', though as far as I can tell all implementations achieve more or less the same thing (though thread safety might vary among the options). Actually continuation passing style (callbacks) are another way of doing the same thing, though they have the disadvantage that they require large structural changes to the code. It wouldn't surprise me if the callback hell can therefore also occur in all versions, though some might make it easier than others (python's implementation in particular makes it somewhat less likely by encouraging information to flow one way)
- danuker 5y ago
  > as it has a (very rarely used) feature that allows information from the caller to be passed back to the generator. The Twisted library encouraged heavy use of this before Python implemented async/await. https://twistedmatrix.com/documents/current/core/howto/defer-intro.html#inline-callbacks-using-yield https://twistedmatrix.com/documents/current/core/howto/defer...
MichaEiler 5y ago
There are a lot more possibilities. The reason why they are so complicated is simply to give the library developers all possible options. This features is afaik something targeted at them, so that boost, asio and others can implement powerful solutions without language restrictions. The only way to enqueue a coroutine is to call schedule() within a co_await statement/expression. In this process the coroutine is suspended. Therefore there should not be any coroutine within the queue, which we cannot immediately resume. I'm afraid I don't have any numbers available to compare coroutines with other approaches. But nevertheless in my opinion coroutines are benefitial because they keep their state (the stack frame, local variables) alive. If you use callbacks you would have to handle all these things yourself. Think about a generator for a sequence of numbers. You would have to store at least the counter variable manually. With a coroutine this happens automatically.

drenvuk 5y ago

You're not the only one. Coroutines are complicated as hell and have too much boiler plate BUT once you handle it for a general enough case you get javascript-esque async await syntax which is very, very nice. Take for instance, this code which relies on libuv for its event loop and co_await to retain its state during its execution: https://gist.github.com/Qix-/09532acd0f6c9a57c09bd9ce31b3023f https://gist.github.com/Qix-/09532acd0f6c9a57c09bd9ce31b3023... Lets say that you want to batch a bunch of database operations into one transaction. You could queue them up over the course of a few milliseconds, run the transaction, and then for each context that relied on different db operations simply return to each's previous point instead of having to call a handler. Granted, the handler is now inside of the `await_transform` needed to work with `co_await` but think of the possibilities. No weirdly separate callback function, no real need to make a class that encapsulates all of the operations for let's say a user's post request, and to top it all off, you can do this on a single thread. It's a tool for cleaner code but I'll be damned if it is really easy to understand. It's just so much stupid boiler plate and a strange way of putting it together.

asdfasgasdgasdg 5y ago
It still boggles the mind that they made it so hard to use this stuff in terms of boilerplate. Hopefully that will all be abstracted into a library that handles io, networking, multiprocessing and synchronization so most of us can just focus on writing the bits that do stuff. But I will never understand how the std maintainers could not manage to do this when every other language is doing async await support in their standard library Io utilities.
- PaulDavisThe1st 5y ago
  One reason could be that C++ tends to be used more in performance-critical contexts, and frequently the abstractions present in "every other language" can get in the way of performance.
  - cobalt 5y ago
    also the memory management model in c++ is much more complex - most other languages just use garbage collection to deal with reference lifetimes
    - PaulDavisThe1st 5y ago
      garbage collection is one of the abstractions that can interfere with performance. it's out of the question for real time contexts, for example.
      - pjmlp 5y ago
        Depends, https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/wp/products-services/jamaicavm/ https://www.aicas.com/wp/products-services/jamaicavm/ On the other hand there is real time where even malloc() is forbidden. https://en.m.wikipedia.org/wiki/MISRA_C https://en.m.wikipedia.org/wiki/MISRA_C
        hedora 5y ago
        The first one targets “sub-millisecond”, which is a eternity in current systems. According to the third link: > we observed a negative correlation between MISRA rule violations and observed faults So, MISRA isn’t really helping with real time at all. The second link claims pauses are capped at 10us, which (if true) is actually competing with careful use of malloc.
        pjmlp 5y ago

contravariant 5y ago

For what it's worth Python's yield pretty much is a coroutine, as it has a (very rarely used) feature that allows information from the caller to be passed back to the generator. The c++ implementation seems closer to lisps 'call with current continuation', though as far as I can tell all implementations achieve more or less the same thing (though thread safety might vary among the options). Actually continuation passing style (callbacks) are another way of doing the same thing, though they have the disadvantage that they require large structural changes to the code. It wouldn't surprise me if the callback hell can therefore also occur in all versions, though some might make it easier than others (python's implementation in particular makes it somewhat less likely by encouraging information to flow one way)

danuker 5y ago
> as it has a (very rarely used) feature that allows information from the caller to be passed back to the generator. The Twisted library encouraged heavy use of this before Python implemented async/await. https://twistedmatrix.com/documents/current/core/howto/defer-intro.html#inline-callbacks-using-yield https://twistedmatrix.com/documents/current/core/howto/defer...

MichaEiler 5y ago

There are a lot more possibilities. The reason why they are so complicated is simply to give the library developers all possible options. This features is afaik something targeted at them, so that boost, asio and others can implement powerful solutions without language restrictions. The only way to enqueue a coroutine is to call schedule() within a co_await statement/expression. In this process the coroutine is suspended. Therefore there should not be any coroutine within the queue, which we cannot immediately resume. I'm afraid I don't have any numbers available to compare coroutines with other approaches. But nevertheless in my opinion coroutines are benefitial because they keep their state (the stack frame, local variables) alive. If you use callbacks you would have to handle all these things yourself. Think about a generator for a sequence of numbers. You would have to store at least the counter variable manually. With a coroutine this happens automatically.