4 ms·

Since coroutines have been in the standard since C++20, I wish he would have explained better what are the advantages of using a Fiber over a Coroutine.

munch117 3y ago
Advantage: No function colouring. Just like with threads, you don't need to declare which functions might potentially yield. There's none of that noisy async/await syntax. Advantage: Fast switching. Switching between fibers is done by swapping in new values for the program counter and stack pointer. Compare with async, where yielding only takes you up one level of stack, so your code ends up walking a stack of coroutine invocations to get all the way back to the scheduler. Disadvantage: Expensive setup - each fiber needs its own pre-allocated stack. Disadvantage: Operating systems don't provide I/O systems readymade to work with fiber schedulers, so if you want to yield to a different fiber during blocking I/O, then you need to invent your own system to handle that. PS about naming: "Fiber" is the Microsoft Windows name for their implementation of full stack coroutines, but the proper platform-independent name for it is actually "coroutine". I've kept with the word "fiber" here to avoid misunderstandings. PS about C++: This is based on async like you find it in C# or Python, I don't really know about C++ coroutines.
- geertj 3y ago
  A disadvantage of the ‘no function coloring’ in fibers is that it makes lockless programming harder. A nested function call can switch from under you without your knowledge, making it hard to know where the preemption points are and whether to take locks when making updates to shared state. With function coloring you immediately see whether a function might switch as there’s a different calling syntax. I’ve programmed both fiber based systems and coroutines. I even created my own fiber libraries for Python (https://github.com/geertj/gruvi https://github.com/geertj/gruvi) and C++ (https://github.com/geertj/cgreenlet https://github.com/geertj/cgreenlet, mostly an experiment, and incorrectly named coroutines for C++ while it’s really fibers). In the Python version I experimented with some features to help you know whether a nested function might switch. In the end, for me and for the problem domains I worked in, the explicit async/await co-routine style wins over fibers. It gives you most of the performance benefits of user mode switching, all of the memory benefits, while keeping your code mostly lock free.
  - gpderetta 3y ago
    [I assume you are using lock free in a loose way, as lacking explicit mutual exclusion]. In practice in most real world applications in C++, absence of async calls is not enough to guard against re-entrancy. Shared state might still be mutated by callbacks, by recursively re-entering the event loop or by other threads. So you need to either document explicitly all the assumptions you are making and vet every single function you are calling in your implicit critical section, or you need some other mechanism that is going to look like a critical section anyway.
    - geertj 3y ago
      Yeah I was using the term lockless loosely here, not referring to atomics. I agree that using explicitly switched coroutines is not sufficient for mostly “lockless” programming (but I do think it’s necessary). I was coming from a thread per core perspective where you run one event loop per core and limit cross core communications (like in the Seastar framework). With this you should be able to perform most state manipulations without locks.
  - dfawcus 3y ago
    That is where an Actor or CSP style comes in useful, one then avoids the locks by having that state mutated in the context of one coroutine alone. Most problems can be recast in to that form, only a small subset are better handled by mutable shared state (and hence explicit locks. In those cases one should probably be modelling the updates as 'Compare and Swap' / 'Compare and Set' types of mutable operation, hence avoiding the situation where unexpected preemption can occur while holding a lock. From my perspective, having use both forms (manually and with language support), I generally find the coroutine (plus Actor/CSP) scheme to be easier to work with and reason about.
  - munch117 3y ago
    > A disadvantage of the ‘no function coloring’ in fibers is that it makes lockless programming harder. But if you treat fiber-style coroutines like threads, and use locks and message queues exactly the same as you would with threads (although differently implemented, of course), then coroutines are basically better-behaved, and nicely deterministic, threads. It's a wonderful programming model, with the one big drawback that you can't take advantage of multiple cores.
- pjmlp 3y ago
  C++ coroutines are based on C# async model, as they were originally proposed by Microsoft, they even use similar concepts in regards to compiler magic of Awaitable types with special methods being reckognised by the compiler, as means to create an awaitable type for any kind of data structure.
  - munch117 3y ago
    That's what I thought, thanks for confirming.
- indigoabstract 3y ago
  I see. Seems pretty similar conceptually to C++ coroutines. Fast switching between fibers is mainly what got me interested, of course. Thank you for the explanations.
- vacuity 3y ago
  Just a note on naming: fibers, goroutines, green threads, virtual threads and the like are stackful coroutines (miniature threads, essentially). Then there are stackless coroutines, which are state machine objects that have function coloring (async/await).
- surajrmal 3y ago
  Fun fact, what you call async is also widely referred to as coroutines. Stackless coroutines vs stackful coroutines seems like the most appropriate way to disambiguate the two.
  - munch117 3y ago
    Thanks, stackful/stackless is noted for next time. I still kinda like using "async" as a shorthand for stackless coroutines though.
    - gpderetta 3y ago
      Before async for I/O became popular, stackless coroutines were mainly used for generator abstractions (for example yield in python), so async can be reductive.
      - munch117 3y ago
        And that's exactly why I would say "async", because then everyone will know that I'm talking about something like async in Python/C#, and not something like Python generators. Technically, they're may well be almost the same thing, but practically, they're used very differently.
snaury 3y ago
Unfortunately in 2023 c++20 coroutines are just barely starting to become usable. Compilers have bugs, especially with symmetric transfer, e.g. msvc doesn't suspend until await_suspend returns (which causes races and double-frees), gcc stops using tail calls in some modes (which can cause stack overflow), I ended up emulating symmetric transfer because you can't rely on it, and emulating it is of course slower. Compilers spill local variables from await_suspend to coroutine frames, which again causes races, which necessitates workarounds like marking await_suspend noinline (in a non-portable way). Having noinline anything in the code path disables heap allocation elision, so all coroutines start to allocate from the heap. When comparing a function call to a coroutine call there's a vast difference in performance. And you need to use coroutines all the way down, which really adds up. Fibers have their own problems: you need to allocate stacks (and that may be expensive), you can't switch fibers between system threads (compilers cache thread local addresses, unfortunately with x86 linux tls often uses fs: segment addressing, so it may appear to work until it doesn't, and weirdly this problem existed in c++20 coroutines too until recent clang versions), widely used open source implementations (e.g. boost context / boost coroutine / boost fiber) are not even exception safe, because you need to save/restore exception globals when switching fibers, and nobody seems to care. :-/ One huge upside to fibers is that a function call is just a normal function call, and it's fast as a result. I wish c++ taken fibers more seriously and added steps for making them safe to use (portable way to handle global state, portable way to mark switching functions as invalidating tls addresses, etc.), we could have something similar to java virtual threads or go goroutines then.
- gpderetta 3y ago
  > exception safe, because you need to save/restore exception globals when switching fibers, Do you have more details about this issue? What exception globals? And for which ABI? You mean some sort of current pending exception when switching during unwinding? [I'm a huge fan of stackful coroutines and I wish were blessed by the standard]
  - snaury 3y ago
    C++ ABI has some per-thread globals: the number that is returned from std::uncaught_exceptions(), and the chain of currently caught exceptions. For example in llvm this is available with a cxa_get_globals call: https://github.com/llvm/llvm-project/blob/b05f1d93469fbd645118739798e51f21a9e410de/libcxxabi/src/cxa_exception.h#L161 https://github.com/llvm/llvm-project/blob/b05f1d93469fbd6451... These need to be saved/restored when switching fibers, otherwise fiber switches from catch clauses (and destructors!) are unsafe, throw without argument may rethrow incorrect exception, code that commits/rollbacks based on uncaught exceptions counter will not work correctly, etc. One example I know where this save/restore is implemented is the userver framework, but it seems to be unexpectedly rare in fiber implementations last time I looked.

munch117 3y ago

Advantage: No function colouring. Just like with threads, you don't need to declare which functions might potentially yield. There's none of that noisy async/await syntax. Advantage: Fast switching. Switching between fibers is done by swapping in new values for the program counter and stack pointer. Compare with async, where yielding only takes you up one level of stack, so your code ends up walking a stack of coroutine invocations to get all the way back to the scheduler. Disadvantage: Expensive setup - each fiber needs its own pre-allocated stack. Disadvantage: Operating systems don't provide I/O systems readymade to work with fiber schedulers, so if you want to yield to a different fiber during blocking I/O, then you need to invent your own system to handle that. PS about naming: "Fiber" is the Microsoft Windows name for their implementation of full stack coroutines, but the proper platform-independent name for it is actually "coroutine". I've kept with the word "fiber" here to avoid misunderstandings. PS about C++: This is based on async like you find it in C# or Python, I don't really know about C++ coroutines.

geertj 3y ago
A disadvantage of the ‘no function coloring’ in fibers is that it makes lockless programming harder. A nested function call can switch from under you without your knowledge, making it hard to know where the preemption points are and whether to take locks when making updates to shared state. With function coloring you immediately see whether a function might switch as there’s a different calling syntax. I’ve programmed both fiber based systems and coroutines. I even created my own fiber libraries for Python (https://github.com/geertj/gruvi https://github.com/geertj/gruvi) and C++ (https://github.com/geertj/cgreenlet https://github.com/geertj/cgreenlet, mostly an experiment, and incorrectly named coroutines for C++ while it’s really fibers). In the Python version I experimented with some features to help you know whether a nested function might switch. In the end, for me and for the problem domains I worked in, the explicit async/await co-routine style wins over fibers. It gives you most of the performance benefits of user mode switching, all of the memory benefits, while keeping your code mostly lock free.
- gpderetta 3y ago
  [I assume you are using lock free in a loose way, as lacking explicit mutual exclusion]. In practice in most real world applications in C++, absence of async calls is not enough to guard against re-entrancy. Shared state might still be mutated by callbacks, by recursively re-entering the event loop or by other threads. So you need to either document explicitly all the assumptions you are making and vet every single function you are calling in your implicit critical section, or you need some other mechanism that is going to look like a critical section anyway.
  - geertj 3y ago
    Yeah I was using the term lockless loosely here, not referring to atomics. I agree that using explicitly switched coroutines is not sufficient for mostly “lockless” programming (but I do think it’s necessary). I was coming from a thread per core perspective where you run one event loop per core and limit cross core communications (like in the Seastar framework). With this you should be able to perform most state manipulations without locks.
- dfawcus 3y ago
  That is where an Actor or CSP style comes in useful, one then avoids the locks by having that state mutated in the context of one coroutine alone. Most problems can be recast in to that form, only a small subset are better handled by mutable shared state (and hence explicit locks. In those cases one should probably be modelling the updates as 'Compare and Swap' / 'Compare and Set' types of mutable operation, hence avoiding the situation where unexpected preemption can occur while holding a lock. From my perspective, having use both forms (manually and with language support), I generally find the coroutine (plus Actor/CSP) scheme to be easier to work with and reason about.
- munch117 3y ago
  > A disadvantage of the ‘no function coloring’ in fibers is that it makes lockless programming harder. But if you treat fiber-style coroutines like threads, and use locks and message queues exactly the same as you would with threads (although differently implemented, of course), then coroutines are basically better-behaved, and nicely deterministic, threads. It's a wonderful programming model, with the one big drawback that you can't take advantage of multiple cores.
pjmlp 3y ago
C++ coroutines are based on C# async model, as they were originally proposed by Microsoft, they even use similar concepts in regards to compiler magic of Awaitable types with special methods being reckognised by the compiler, as means to create an awaitable type for any kind of data structure.
- munch117 3y ago
  That's what I thought, thanks for confirming.
indigoabstract 3y ago
I see. Seems pretty similar conceptually to C++ coroutines. Fast switching between fibers is mainly what got me interested, of course. Thank you for the explanations.
vacuity 3y ago
Just a note on naming: fibers, goroutines, green threads, virtual threads and the like are stackful coroutines (miniature threads, essentially). Then there are stackless coroutines, which are state machine objects that have function coloring (async/await).
surajrmal 3y ago
Fun fact, what you call async is also widely referred to as coroutines. Stackless coroutines vs stackful coroutines seems like the most appropriate way to disambiguate the two.
- munch117 3y ago
  Thanks, stackful/stackless is noted for next time. I still kinda like using "async" as a shorthand for stackless coroutines though.
  - gpderetta 3y ago
    Before async for I/O became popular, stackless coroutines were mainly used for generator abstractions (for example yield in python), so async can be reductive.
    - munch117 3y ago
      And that's exactly why I would say "async", because then everyone will know that I'm talking about something like async in Python/C#, and not something like Python generators. Technically, they're may well be almost the same thing, but practically, they're used very differently.

snaury 3y ago

Unfortunately in 2023 c++20 coroutines are just barely starting to become usable. Compilers have bugs, especially with symmetric transfer, e.g. msvc doesn't suspend until await_suspend returns (which causes races and double-frees), gcc stops using tail calls in some modes (which can cause stack overflow), I ended up emulating symmetric transfer because you can't rely on it, and emulating it is of course slower. Compilers spill local variables from await_suspend to coroutine frames, which again causes races, which necessitates workarounds like marking await_suspend noinline (in a non-portable way). Having noinline anything in the code path disables heap allocation elision, so all coroutines start to allocate from the heap. When comparing a function call to a coroutine call there's a vast difference in performance. And you need to use coroutines all the way down, which really adds up. Fibers have their own problems: you need to allocate stacks (and that may be expensive), you can't switch fibers between system threads (compilers cache thread local addresses, unfortunately with x86 linux tls often uses fs: segment addressing, so it may appear to work until it doesn't, and weirdly this problem existed in c++20 coroutines too until recent clang versions), widely used open source implementations (e.g. boost context / boost coroutine / boost fiber) are not even exception safe, because you need to save/restore exception globals when switching fibers, and nobody seems to care. :-/ One huge upside to fibers is that a function call is just a normal function call, and it's fast as a result. I wish c++ taken fibers more seriously and added steps for making them safe to use (portable way to handle global state, portable way to mark switching functions as invalidating tls addresses, etc.), we could have something similar to java virtual threads or go goroutines then.

gpderetta 3y ago
> exception safe, because you need to save/restore exception globals when switching fibers, Do you have more details about this issue? What exception globals? And for which ABI? You mean some sort of current pending exception when switching during unwinding? [I'm a huge fan of stackful coroutines and I wish were blessed by the standard]
- snaury 3y ago
  C++ ABI has some per-thread globals: the number that is returned from std::uncaught_exceptions(), and the chain of currently caught exceptions. For example in llvm this is available with a cxa_get_globals call: https://github.com/llvm/llvm-project/blob/b05f1d93469fbd645118739798e51f21a9e410de/libcxxabi/src/cxa_exception.h#L161 https://github.com/llvm/llvm-project/blob/b05f1d93469fbd6451... These need to be saved/restored when switching fibers, otherwise fiber switches from catch clauses (and destructors!) are unsafe, throw without argument may rethrow incorrect exception, code that commits/rollbacks based on uncaught exceptions counter will not work correctly, etc. One example I know where this save/restore is implemented is the userver framework, but it seems to be unexpectedly rare in fiber implementations last time I looked.