5 ms·

C++ coroutines do not spark joy (2021)

63 points by signa11 3y ago

npigrounet 3y ago
[dead]
dang 3y ago
Discussed at the time: C++ Coroutines Do Not Spark Joy - https://news.ycombinator.com/item?id=29064233 https://news.ycombinator.com/item?id=29064233 - Nov 2021 (250 comments)
7e 3y ago
I wouldn’t be surprised if the C++ coroutine push was for a perf. packet in some big tech company. Promo is hard. Meanwhile, the same optimizations one could use for efficient coroutines could just as well be applied to fibers, which are much nicer.
- bonzini 3y ago
  The semantics of fibers are difficult to define precisely. For example using thread local storage inside fibers can be problematic if a fiber sleeps in one thread and restarts in another, because the compiler is free to store something like &errno and use it across function calls. (And this is not theoretical—compilers will do this).
  - injidup 3y ago
    You see this exact problem in task based libraries like task based parallel libraries such as taskflow. It's easy to get burned using thread local variables if you are not careful because a task might yield to the scheduler which runs another task ( of the same type ) on the same thread and then your thread local is corrupted. You end up requiring more sophisticated object pools where you can check out and check in objects.
    - gpderetta 3y ago
      What bonzini is referring to is actually a more subtle issue: the compiler will CSE the (hidden) thread_local address calculation even across function calls. So even if you are careful and do not assume that thread_local state is preserved across function calls, your code can still be wrong as suddenly it will be accessing a thread_local owned by another thread. That might be as simple as dereferencing the wrong errno. It is very hard to workaround that in your code. The only practical solution is to never migrate coroutines to other threads.
  - gpderetta 3y ago
    Well, it would be nice if, optionally, GCC didn't do that. I believe that MSVC has a flag to prevent exactly this optimization.
- surajrmal 3y ago
  Stackless coroutines are useful in domains where stackful coroutines are simply not an option. Not every application runs on a server. There are tradeoffs associates with stackless coroutines, but I don't think it's questionable whether it was the right choice for a language like c++. If you want stackful coroutines, there are likely better suited languages for your domain that are not c++.
- pjmlp 3y ago
  They were initally proposed by Microsoft, based on the way C# coroutines work on top of async/await, and the asynchrounous runtime used in .NET Native and C++/CX for WinRT. Search for talks from Gor Nishanov.
metadat 3y ago
Wow, that API looks pretty awkward and unpleasant, with the co_yield, co_return, etc. Golang has spoiled me. Edit: @gloryjulio I'm sure it works fine, it's just so unergonomic to express compared to Golang. But this is nothing new with C++, it can do anything.. might not be pretty, but it can do it.
- gloryjulio 3y ago
  We only use co_await/co_return in the production code. It's basically like JavaScript promise and it's very easy to work with. Calling coroutine in non-coroutine code is also easy, just do blockingWait(co_routineFunc()); We almost never use multithreading code in the business layer now unless coroutine is actually causing performance issue. I haven't seen that happen yet
- jpc0 3y ago
  I mean goroutines are not exactly coroutines nor are they threads they are somewhere in between. And its pretty easy to use the same workflow as goroutines in C++ using std::thread and a concurrent queue implementation, tbb has a decent one and moodycamels is also very good. It's actual async/await that is a touch annoying but it's also fundamental building blocks, there are currently few abstractions ontop of it. I also believe strongly async/await is an antipattern. There are better models to handle concurrency well imho that don't use function colouring. Golangs goroutines and channels (effectively actors but that's a can of worms) are a great solution and probably what you should be reaching for instead of async/await.
  - surajrmal 3y ago
    Coroutines can be divided into two categories, stackless and stackful. goroutines are stackful coroutines.
    - jpc0 3y ago
      Except goroutines will absolutely spawn a thread which a coroutine by definition wont. Goroutines are parallel, coroutine are concurrent. Those are not the same thing. Coroutines co-operatively yeild to other coroutines. Parallel constructs run in parallel at the same time. Goroutines can run in parallel at the same time and thus are not a coroutine.
jayd16 3y ago
Weird that the yield statement is conflated with coroutines instead of iterators/generators. Even weirder to say its a "C# coroutine" feature when there's no coroutine in the C# language. It does look pretty similar to the yield keyword for C# iterators. Unity (and probably others) use yielding iterators as a somewhat hacky way to make coroutines, though. I wouldn't say its a good role model but it gets the job done. > Give me access to the generated struct. Allow me to put it on the stack of another function. Or as a member of another struct. Then I can store it on the heap if I want, but don’t force me. As for why they're allocated on the heap...seems like the stack would be a foot gun, no? To write a safe coroutine, you need to hope your caller doesn't blow out the stack between iterations or just defensively put everything on the heap yourself anyway. Enforcing safety seems like the right call but that's a matter of taste, I suppose. This does remind me of a recent C# change to the Task<T> promise type returned by async/await functions. Before they were only heap allocated because you need that to be able to properly await them multiple times. This is a major reason you can't use them in games and use things like Unity's Coroutines. As it turns out a lot of small heap allocs suck and stack allocation does come in handy for perf. Now we have stack allocated Task<T> types as well. It would be a nightmare if the unsafe was the default though... Why is life so complicated?
- LoganDark 3y ago
  > Weird that the yield statement is conflated with coroutines instead of iterators/generators. I think the conflict is between "yielding another value" and "yielding control back to the scheduler". Both of them use "yield" but they're different types of yield, and people use the same keyword for both, and either one can be implemented in terms of the other, and so on.
  - jayd16 3y ago
    They're not different though, right? Just two ways to think about what it means to return, no? How can you return a value without returning control to read that value? They're the same. Still, I don't think iterators and coroutines are necessarily the same. You can imagine a coroutine that doesn't return a value and only happens to share a thread or something.
bingo3131 3y ago
Regarding new/delete elision: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0981r0.html https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p09... You can also specify custom new/delete operations for your co-routine for control. I am not sure if you are allowed to delete them to guarantee elision either happens or the program fails to compile. Much like lambdas and ranged-based for loops, co-routines are pretty much defined as a code transformation into lower-level C++ rather than being black magic. Regarding the "inline" keyword being used to move a function's body into the caller: that is a compiler hint and not mandatory. The actual purpose of the "inline" function is to allow the implementation of a function to appear in multiple translation units without that causing linking issues (provided the implementation is identical). In terms of "I’m curious if anyone actually finds something useful to do with these.": the modern C++ Windows Runtime API is built upon async operations and co-routines.
- winstonprivacy 3y ago
  Yes, they are incredibly useful and performant in real time state machines. One of the best features of the language in certain situations.
- AshamedCaptain 3y ago
  > In terms of "I’m curious if anyone actually finds something useful to do with these.": the modern C++ Windows Runtime API is built upon async operations and co-routines. And this answers another question from the article: > in C++ I don’t know who [coroutines] are for, or who asked for this… And the answer is Microsoft. They already had an extension for coroutines in their compiler before the standard, the standard is not very different from their original implementation (albeit there are differences), and you can almost always see at least one @ms address in all the discussions.
  - gpderetta 3y ago
    My understanding is that the extension in MSVC was done specifically as a proof of concept for the standard proposal. In fact the authors also implemented it in clang as well. This is quite common. Especially for complex proposals, the committee wants field experience or at least proof that it is implementable before standardization.
fanf2 3y ago
See also Simon Tatham’s review https://www.chiark.greenend.org.uk/~sgtatham/quasiblog/coroutines-c++20/ https://www.chiark.greenend.org.uk/~sgtatham/quasiblog/corou...
- downut 3y ago
  Thank you for this pointer. I have been studying coroutines off and on for a while and my google/ddg foo is weak and did not find this. Or maybe google/ddg are getting weak. Sadness. I was using David Mazières' tutorial[1] but was still puzzled about the seemingly magical promise_type/coroutine_handle conversions that are nicely explained here. [1] https://www.scs.stanford.edu/~dm/blog/c++-coroutines.html https://www.scs.stanford.edu/~dm/blog/c++-coroutines.html
pavlov 3y ago
Coroutines are low-level building blocks for various asynchronous models. C++23 introduces std::generator which is roughly similar to Python and JavaScript generators, implemented on top of coroutines. If that model fits your application, you don’t need to mess with the coro API directly.
- jcelerier 3y ago
  Also plenty of other libs provide a neat API for c++20 coroutines: boost.asio, boost.cobalt, qcoro...
foota 3y ago
My understanding taken from someone smarter than me is that the issues with coroutines in C++ can be worked around for a good and efficient implementation.

63 points by signa11 3y ago

[dead]

dang 3y ago

Discussed at the time: C++ Coroutines Do Not Spark Joy - https://news.ycombinator.com/item?id=29064233 https://news.ycombinator.com/item?id=29064233 - Nov 2021 (250 comments)

7e 3y ago

I wouldn’t be surprised if the C++ coroutine push was for a perf. packet in some big tech company. Promo is hard. Meanwhile, the same optimizations one could use for efficient coroutines could just as well be applied to fibers, which are much nicer.

bonzini 3y ago
The semantics of fibers are difficult to define precisely. For example using thread local storage inside fibers can be problematic if a fiber sleeps in one thread and restarts in another, because the compiler is free to store something like &errno and use it across function calls. (And this is not theoretical—compilers will do this).
- injidup 3y ago
  You see this exact problem in task based libraries like task based parallel libraries such as taskflow. It's easy to get burned using thread local variables if you are not careful because a task might yield to the scheduler which runs another task ( of the same type ) on the same thread and then your thread local is corrupted. You end up requiring more sophisticated object pools where you can check out and check in objects.
  - gpderetta 3y ago
    What bonzini is referring to is actually a more subtle issue: the compiler will CSE the (hidden) thread_local address calculation even across function calls. So even if you are careful and do not assume that thread_local state is preserved across function calls, your code can still be wrong as suddenly it will be accessing a thread_local owned by another thread. That might be as simple as dereferencing the wrong errno. It is very hard to workaround that in your code. The only practical solution is to never migrate coroutines to other threads.
- gpderetta 3y ago
  Well, it would be nice if, optionally, GCC didn't do that. I believe that MSVC has a flag to prevent exactly this optimization.
surajrmal 3y ago
Stackless coroutines are useful in domains where stackful coroutines are simply not an option. Not every application runs on a server. There are tradeoffs associates with stackless coroutines, but I don't think it's questionable whether it was the right choice for a language like c++. If you want stackful coroutines, there are likely better suited languages for your domain that are not c++.
pjmlp 3y ago
They were initally proposed by Microsoft, based on the way C# coroutines work on top of async/await, and the asynchrounous runtime used in .NET Native and C++/CX for WinRT. Search for talks from Gor Nishanov.

metadat 3y ago

Wow, that API looks pretty awkward and unpleasant, with the co_yield, co_return, etc. Golang has spoiled me. Edit: @gloryjulio I'm sure it works fine, it's just so unergonomic to express compared to Golang. But this is nothing new with C++, it can do anything.. might not be pretty, but it can do it.

gloryjulio 3y ago
We only use co_await/co_return in the production code. It's basically like JavaScript promise and it's very easy to work with. Calling coroutine in non-coroutine code is also easy, just do blockingWait(co_routineFunc()); We almost never use multithreading code in the business layer now unless coroutine is actually causing performance issue. I haven't seen that happen yet
jpc0 3y ago
I mean goroutines are not exactly coroutines nor are they threads they are somewhere in between. And its pretty easy to use the same workflow as goroutines in C++ using std::thread and a concurrent queue implementation, tbb has a decent one and moodycamels is also very good. It's actual async/await that is a touch annoying but it's also fundamental building blocks, there are currently few abstractions ontop of it. I also believe strongly async/await is an antipattern. There are better models to handle concurrency well imho that don't use function colouring. Golangs goroutines and channels (effectively actors but that's a can of worms) are a great solution and probably what you should be reaching for instead of async/await.
- surajrmal 3y ago
  Coroutines can be divided into two categories, stackless and stackful. goroutines are stackful coroutines.
  - jpc0 3y ago
    Except goroutines will absolutely spawn a thread which a coroutine by definition wont. Goroutines are parallel, coroutine are concurrent. Those are not the same thing. Coroutines co-operatively yeild to other coroutines. Parallel constructs run in parallel at the same time. Goroutines can run in parallel at the same time and thus are not a coroutine.

jayd16 3y ago

Weird that the yield statement is conflated with coroutines instead of iterators/generators. Even weirder to say its a "C# coroutine" feature when there's no coroutine in the C# language. It does look pretty similar to the yield keyword for C# iterators. Unity (and probably others) use yielding iterators as a somewhat hacky way to make coroutines, though. I wouldn't say its a good role model but it gets the job done. > Give me access to the generated struct. Allow me to put it on the stack of another function. Or as a member of another struct. Then I can store it on the heap if I want, but don’t force me. As for why they're allocated on the heap...seems like the stack would be a foot gun, no? To write a safe coroutine, you need to hope your caller doesn't blow out the stack between iterations or just defensively put everything on the heap yourself anyway. Enforcing safety seems like the right call but that's a matter of taste, I suppose. This does remind me of a recent C# change to the Task<T> promise type returned by async/await functions. Before they were only heap allocated because you need that to be able to properly await them multiple times. This is a major reason you can't use them in games and use things like Unity's Coroutines. As it turns out a lot of small heap allocs suck and stack allocation does come in handy for perf. Now we have stack allocated Task<T> types as well. It would be a nightmare if the unsafe was the default though... Why is life so complicated?

LoganDark 3y ago
> Weird that the yield statement is conflated with coroutines instead of iterators/generators. I think the conflict is between "yielding another value" and "yielding control back to the scheduler". Both of them use "yield" but they're different types of yield, and people use the same keyword for both, and either one can be implemented in terms of the other, and so on.
- jayd16 3y ago
  They're not different though, right? Just two ways to think about what it means to return, no? How can you return a value without returning control to read that value? They're the same. Still, I don't think iterators and coroutines are necessarily the same. You can imagine a coroutine that doesn't return a value and only happens to share a thread or something.

bingo3131 3y ago

Regarding new/delete elision: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0981r0.html https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p09... You can also specify custom new/delete operations for your co-routine for control. I am not sure if you are allowed to delete them to guarantee elision either happens or the program fails to compile. Much like lambdas and ranged-based for loops, co-routines are pretty much defined as a code transformation into lower-level C++ rather than being black magic. Regarding the "inline" keyword being used to move a function's body into the caller: that is a compiler hint and not mandatory. The actual purpose of the "inline" function is to allow the implementation of a function to appear in multiple translation units without that causing linking issues (provided the implementation is identical). In terms of "I’m curious if anyone actually finds something useful to do with these.": the modern C++ Windows Runtime API is built upon async operations and co-routines.

winstonprivacy 3y ago
Yes, they are incredibly useful and performant in real time state machines. One of the best features of the language in certain situations.
AshamedCaptain 3y ago
> In terms of "I’m curious if anyone actually finds something useful to do with these.": the modern C++ Windows Runtime API is built upon async operations and co-routines. And this answers another question from the article: > in C++ I don’t know who [coroutines] are for, or who asked for this… And the answer is Microsoft. They already had an extension for coroutines in their compiler before the standard, the standard is not very different from their original implementation (albeit there are differences), and you can almost always see at least one @ms address in all the discussions.
- gpderetta 3y ago
  My understanding is that the extension in MSVC was done specifically as a proof of concept for the standard proposal. In fact the authors also implemented it in clang as well. This is quite common. Especially for complex proposals, the committee wants field experience or at least proof that it is implementable before standardization.

fanf2 3y ago

See also Simon Tatham’s review https://www.chiark.greenend.org.uk/~sgtatham/quasiblog/coroutines-c++20/ https://www.chiark.greenend.org.uk/~sgtatham/quasiblog/corou...

downut 3y ago
Thank you for this pointer. I have been studying coroutines off and on for a while and my google/ddg foo is weak and did not find this. Or maybe google/ddg are getting weak. Sadness. I was using David Mazières' tutorial[1] but was still puzzled about the seemingly magical promise_type/coroutine_handle conversions that are nicely explained here. [1] https://www.scs.stanford.edu/~dm/blog/c++-coroutines.html https://www.scs.stanford.edu/~dm/blog/c++-coroutines.html

pavlov 3y ago

Coroutines are low-level building blocks for various asynchronous models. C++23 introduces std::generator which is roughly similar to Python and JavaScript generators, implemented on top of coroutines. If that model fits your application, you don’t need to mess with the coro API directly.

jcelerier 3y ago
Also plenty of other libs provide a neat API for c++20 coroutines: boost.asio, boost.cobalt, qcoro...

foota 3y ago

My understanding taken from someone smarter than me is that the issues with coroutines in C++ can be worked around for a good and efficient implementation.