12 ms·

Fiber in C++: Understanding the Basics

135 points by ibobev 3y ago

Alifatisk 3y ago
Why is more and more languages embracing Fibers / Green Threads now? I’m curious
- gpderetta 3y ago
  pushback against the stackless coroutine model mostly (i.e. async/await). But people have been playing with stackful coroutines in C++ for decades.
- bmacho 3y ago
  No reasons, probably. No, really. If you flip a coin 10 times, 3-7 or 7-3 is a perfectly OK outcome, and our n (the more and more languages) is not even 10.
samsquire 3y ago
Thank you for this in-depth article. I am a less than a C++ beginner but I asked Stack Overflow how to run C++ coroutines in a thread pool. It seems coroutines in C++20 are unfinalised but don't quote me on that but I did get some sourcecode for older versions of the C++20 standard. I used Marce's Coll's excellent blog post about how to create coroutines in assembly by adjusting the RSP register. https://blog.dziban.net/posts/coroutines/ https://blog.dziban.net/posts/coroutines/ I extended Marce's code to run the coroutines in kernel threads: https://github.com/samsquire/assembly https://github.com/samsquire/assembly (see threadedcoroutines.S) I have been thinking of coroutines in terms of query compilation for database engines and the volcano query model and this article: https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html Tying together two pieces of code that call eachother in push or pull driven style is really powerful. Or if you're running multiple independent tasks that need their own state. This as I understand it is the original intent of object orientation that Alan Kay wanted and is represented by Erlang and partly Go. Specifically, I am thinking of compiler created coroutines where code can be interleaved at compile time rather than at runtime.
IceMichael 3y ago
Still looking for a lib that supports fibers... boost::asio is... well, boost, and therefore annoying. It also does not support it out of the box
- gpderetta 3y ago
  Boost asio is available as a standalone lib as well. What do you mean it doesn't support "it" out of the box?
  - wolletd 3y ago
    Asio has no coroutine implementation and C++20 Coros are stackless. So, to build fibers with asio, you need boost::coroutine for stackful coros.
- yellow_lead 3y ago
  Asio has strands which may be an alternative depending on the use case.
- dilyevsky 3y ago
  https://github.com/google/marl https://github.com/google/marl
hackyhacky 3y ago
I'm confused in how this article defines "fiber." At the beginning of the article, fibers are described as a lightweight userspace thread, similar to what is provided in the runtimes of Haskell and Go, and has been recently revived in Java as "virtual threads." However, later in the article (after a long and apparently irrelevant digression about stack management), they describe Windows API functions that are required for using fibers and seem to suggest that fibers are OS-level, and not application-managed. Am I missing something? [edit: typo]
- ojkelly 3y ago
  My general understanding is that a fiber is a lightweight thread, and conceptually can be implemented inside any program that wants to schedule its own fibers. They’re distinct from CPU threads. When I last dived into fibers I discovered windows was trying to make them a thing quite a while ago which is why we have things like [0]. Fibers being virtual threads, are recursive. In the same way you can run a VM inside a VM, you could build fibers on top of fibers (which sounds like a recipe for unpredictable performance). [0] https://learn.microsoft.com/en-us/windows/win32/procthread/using-fibers https://learn.microsoft.com/en-us/windows/win32/procthread/u...
- pvg 3y ago
  Windows has its own fibers facility which does some things for you but still leaves the scheduling to the application so it's still fibers. It's a little confusing in the article but they look at the Windows API as a kind of API template - what do you need to implement to have fibers. Then they describe their own implementation.
- j16sdiz 3y ago
  Fiber is a concept in Windows API for user-space thread. It was designed for MS SQL Server. It have a very confusing API and did not gain much use in 3rd party applications.
- nickelpro 3y ago
  The Windows fiber library is just that, a library. It's not an OS component. It's provided because stack switching in Windows is a somewhat fraught [1] and not officially documented procedure which requires modifying the TIB [2]. Doing it correctly isn't hard, and has been done many times in bullet-proof libs (notably boost::context), but MS would maybe rather you not. [1]: https://devblogs.microsoft.com/oldnewthing/20080215-00/?p=23443 https://devblogs.microsoft.com/oldnewthing/20080215-00/?p=23... [2]: https://en.wikipedia.org/wiki/Win32_Thread_Information_Block https://en.wikipedia.org/wiki/Win32_Thread_Information_Block
smallstepforman 3y ago
Again, academia and the tech press overlook to talk about Actors, and we live in a world with poorly implemented pretend Actor subsets, without any of the true Actor features (work stealing, message passing etc, and best of all, no contention since no shared resources, other than Actor references). Fibres, goroutines, promises, futures, job queues, etc, are poor substitutes for Actors.
- deleted 3y ago
  [deleted]
- grumpyprole 3y ago
  > Fibres, goroutines, promises, futures, job queues, etc, are poor substitutes for Actors. They are not necessarily substitutes, they are lower level primitives that allow more complex models, like Actors, to be implemented. All execution models have trade offs; and sometimes we might prefer an execution model without deadlock, livelock and starvation. So for a general purpose language, a complex model like Actors is ideally a library and not enforced as the only way of achieving concurrency.
- jeffreygoesto 3y ago
  No shared resources can be a performance problem, I can see why there are rusts and vales efforts to come to a slightly different tradeoff.
  - bmacho 3y ago
    Pony/Ponylang claims that they can do actors fast and safe (even safer than rust). There is a type system that allows one to share data between actors safely, or so they claim.
    - vacuity 3y ago
      I believe they use a GC, so they may be able to leverage the runtime to help out. If it's all type system-based, the compiler probably has to be stricter to ensure safety (Rice's theorem implies that compilers err on the side of rejecting some valid programs in order to reject all invalid programs). Maybe Pony could be a blend of Erlang and Rust?
endorphine 3y ago
> Fiber is quite a lightweight thread of execution. Like coroutine, fiber allows yielding at any point inside it. To some degree, we can regard fiber as a form of stackful coroutine [...] Can someone explain what the "stackful coroutine" term means (ideally with an example)? And how can one implement it?
- nickelpro 3y ago
  It's exactly what it says, it's a coroutine with its own stack. A fiber isn't to some degree a stackful coroutine, it is a stackful coroutine. A stackless coroutine does not have a its own stack, it uses the calling context's stack and therefore can only yield to the calling context from the top-level coroutine function. A stackful coroutine has its own stack, and so can yield to the calling context from anywhere. As far as how to implement, I always liked Malte Skarupke's blog post on the subject: https://probablydance.com/2013/02/20/handmade-coroutines-for-windows/ https://probablydance.com/2013/02/20/handmade-coroutines-for...
- nstbayless 3y ago
  I assume it's this: In Python, coroutines cannot yield from within another function call: if coroutine A calls function B, B cannot yield. In Lua, it's possible to yield a coroutine at any function depth: B can yield. To implement something like this in a compiled language, you just need multiple stacks instead of the usual 1. Most architectures, such as x86, have a stack pointer register; when yielding a coroutine, just change the sp register to point to some other stack -- when resuming, restore the sp. This is not a new concept. Pokémon on the gameboy did this for its UI fiber, for example.
  - geertj 3y ago
    > In Python, coroutines cannot yield from within another function call: if coroutine A calls function B, B cannot yield. I know you likely know this, but just for context. This is technically correct of course, B cannot yield, but not how you would use it in real life. If B needs to yield because it calls C which need to yield, then B needs to be a coroutine itself. This allows for a ‘nested yield’ where A, B, and C are all paused by yielding to their direct parent. Python made this explicit before async/await as you had to use “yield from” instead of just yield. With async/await that became “await”. This does lead to what’s typically mentioned as the biggest disadvantage of stackless coroutines which is that you get parallel sets of functions, one async and one sync. This mostly means that IO and networking libraries end up being either sync or async but not both.
indigoabstract 3y ago
Since coroutines have been in the standard since C++20, I wish he would have explained better what are the advantages of using a Fiber over a Coroutine.
- munch117 3y ago
  Advantage: No function colouring. Just like with threads, you don't need to declare which functions might potentially yield. There's none of that noisy async/await syntax. Advantage: Fast switching. Switching between fibers is done by swapping in new values for the program counter and stack pointer. Compare with async, where yielding only takes you up one level of stack, so your code ends up walking a stack of coroutine invocations to get all the way back to the scheduler. Disadvantage: Expensive setup - each fiber needs its own pre-allocated stack. Disadvantage: Operating systems don't provide I/O systems readymade to work with fiber schedulers, so if you want to yield to a different fiber during blocking I/O, then you need to invent your own system to handle that. PS about naming: "Fiber" is the Microsoft Windows name for their implementation of full stack coroutines, but the proper platform-independent name for it is actually "coroutine". I've kept with the word "fiber" here to avoid misunderstandings. PS about C++: This is based on async like you find it in C# or Python, I don't really know about C++ coroutines.
  - geertj 3y ago
    A disadvantage of the ‘no function coloring’ in fibers is that it makes lockless programming harder. A nested function call can switch from under you without your knowledge, making it hard to know where the preemption points are and whether to take locks when making updates to shared state. With function coloring you immediately see whether a function might switch as there’s a different calling syntax. I’ve programmed both fiber based systems and coroutines. I even created my own fiber libraries for Python (https://github.com/geertj/gruvi https://github.com/geertj/gruvi) and C++ (https://github.com/geertj/cgreenlet https://github.com/geertj/cgreenlet, mostly an experiment, and incorrectly named coroutines for C++ while it’s really fibers). In the Python version I experimented with some features to help you know whether a nested function might switch. In the end, for me and for the problem domains I worked in, the explicit async/await co-routine style wins over fibers. It gives you most of the performance benefits of user mode switching, all of the memory benefits, while keeping your code mostly lock free.
syncurrent 3y ago
Somewhat related to Fibres are the Trails of synchronous reactive programming languages. Both allow efficient logical concurrency based on cooperative scheduling. The nice thing with Trails is that the scheduling can be determined by the compiler by extracting dependencies from the synchronous reactive program thus increasing determinism of your app.
diffuse_l 3y ago
I believe that fibers are considered more trouble than they worth? [0] [0] https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=102989 https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10...
- csb6 3y ago
  The Gor Nishanov paper linked in that piece goes into a lot of detail of some of the downsides of fibers in an unmanaged language like C++. [0] http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
  - teleforce 3y ago
    Thanks for the link to the Gor's paper. Apparently the 2018 paper table only covered top 10 languages in the TIOBE index, I wished it can be updated and extended further to top 50 languages. Another popular compiled language with GC (by default) namely D language (TIOBE ranked 37 as of Sept 2023) does support fiber and it's implemented in its vibe.d web framework [1],[2]. Gor also mentioned about significant 160 ns overhead for stackful fiber in Go (goroutine) to interact with a C library i.e. cost of switching between Go's goroutines and C threads (as of 2018). It will be interesting to know the fiber overhead in D language since C compilation is now natively supported by D compiler [3]. [1] Fibers in Programming in D: https://ddili.org/ders/d.en/fibers.html https://ddili.org/ders/d.en/fibers.html [2] Fibers, what for: https://forum.dlang.org/thread/eowfajvnzqnncmfnjhnf@forum.dlang.org https://forum.dlang.org/thread/eowfajvnzqnncmfnjhnf@forum.dl... [3] Adding ANSI C11 C compiler to D so it can import and compile C files directly: https://news.ycombinator.com/item?id=27102584 https://news.ycombinator.com/item?id=27102584
PaulDavisThe1st 3y ago
Remind me again why no OS other than Solaris ever provided "scheduler activations" (i.e. the kernel calls back into a user-space scheduler whenever a kernel-level scheduling event occurs)
- kevingadd 3y ago
  Windows had a custom scheduler API (https://learn.microsoft.com/en-us/windows/win32/procthread/user-mode-scheduling https://learn.microsoft.com/en-us/windows/win32/procthread/u...) but they removed it, presumably because it wasn't used much
  - vacuity 3y ago
    As the Wikipedia page shows, NetBSD had an implementation but later removed it. There isn't really enough information to make a sound judgement on scheduler activations. Although if I had to guess, there might be too much overhead (maintenance of code and/or actual performance) for something that is rarely used. It might make more sense now that we're moving towards user-level concurrency everywhere (async/await, goroutines, virtual threads, etc.).
    - gpderetta 3y ago
      Basically yes. M:N just wasn't a good fit for the way applications used (and use) pthreads. For those that need pure CPU parallelism M:N is just useless overhead, while highly concurrent applications were either using one process per request for simplicity and robustness, or directly building on top of poll/epoll/kqueue for performance. So, at least at that time, M:N and scheduler activation was a solution in search of a problem. It might still see a resurgence on one way or another.
      - PaulDavisThe1st 3y ago
        But that's not what SA was about. Even with N:N architecture, if one kernel thread blocks (say, on I/O), the point is for the user-space scheduler to decide which thread should run next. The point was that the kernel scheduler cannot/should not understand the application thread scheduling requirements, and that when a kernel thread allocated to the task blocks, only user space can (correctly) decide what to do.
frozenport 3y ago
Some readers may have a question by now. What is the rationale behind the choices of the registers that need to be stored? Yet another article that goes into the gorey details of how fiber works but not how fibers are used. How do we write the "Applications own scheduler"?
OnlyMortal 3y ago
I’ve plenty of Classic Mac experience with cooperative threads. You’d basically yield to run another “thread”. Can someone tell me the difference from those to fibres?
ylow 3y ago
Some random historical notes coroutine stuff which are perhaps of interest to some: 1. You can implement "stack-free coroutines" in C with some really entertaining macros https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html 2. Does anyone remembers Cilk? Its still the fastest stackful coroutine models I know of. Having implemented and used both before (for different purposes), I like both, but with a mild preference for the stackful version. Largely because you don't have function coloring problems, and you can actually get a proper stack trace and is so significantly easier to debug when things go wrong. It is also good to differentiate coroutines from async (they seem to be very interleaved these days). Coroutines are a mechanic to achieve mutual recursion / generators. And that is fine for expressing certain algorithms and systems in a much cleaner fashion. Note that parallelism is not necessarily implied by "coroutine". Async is a mechanic to "doing something else" while waiting for IO. Fibers or state machines are both different solutions to this problem. Certain coroutine implementations can help with this, but I think it is massively overused. Rust due to the function coloring problem, ends up requiring almost everything to be marked "async". And some golang code I have read seems to overuse goroutines unecessarily. I think use of async should be narrow and minimized, and localized to only the places where it makes sense.
- gpderetta 3y ago
  > Does anyone remembers Cilk? Yes! It's a shame it never gained traction and Intel abandoned it, eventually getting dropped from icc and gcc.

135 points by ibobev 3y ago

Alifatisk 3y ago

Why is more and more languages embracing Fibers / Green Threads now? I’m curious

gpderetta 3y ago
pushback against the stackless coroutine model mostly (i.e. async/await). But people have been playing with stackful coroutines in C++ for decades.
bmacho 3y ago
No reasons, probably. No, really. If you flip a coin 10 times, 3-7 or 7-3 is a perfectly OK outcome, and our n (the more and more languages) is not even 10.

samsquire 3y ago

Thank you for this in-depth article. I am a less than a C++ beginner but I asked Stack Overflow how to run C++ coroutines in a thread pool. It seems coroutines in C++20 are unfinalised but don't quote me on that but I did get some sourcecode for older versions of the C++20 standard. I used Marce's Coll's excellent blog post about how to create coroutines in assembly by adjusting the RSP register. https://blog.dziban.net/posts/coroutines/ https://blog.dziban.net/posts/coroutines/ I extended Marce's code to run the coroutines in kernel threads: https://github.com/samsquire/assembly https://github.com/samsquire/assembly (see threadedcoroutines.S) I have been thinking of coroutines in terms of query compilation for database engines and the volcano query model and this article: https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html Tying together two pieces of code that call eachother in push or pull driven style is really powerful. Or if you're running multiple independent tasks that need their own state. This as I understand it is the original intent of object orientation that Alan Kay wanted and is represented by Erlang and partly Go. Specifically, I am thinking of compiler created coroutines where code can be interleaved at compile time rather than at runtime.

IceMichael 3y ago

Still looking for a lib that supports fibers... boost::asio is... well, boost, and therefore annoying. It also does not support it out of the box

gpderetta 3y ago
Boost asio is available as a standalone lib as well. What do you mean it doesn't support "it" out of the box?
- wolletd 3y ago
  Asio has no coroutine implementation and C++20 Coros are stackless. So, to build fibers with asio, you need boost::coroutine for stackful coros.
yellow_lead 3y ago
Asio has strands which may be an alternative depending on the use case.
dilyevsky 3y ago
https://github.com/google/marl https://github.com/google/marl

hackyhacky 3y ago

I'm confused in how this article defines "fiber." At the beginning of the article, fibers are described as a lightweight userspace thread, similar to what is provided in the runtimes of Haskell and Go, and has been recently revived in Java as "virtual threads." However, later in the article (after a long and apparently irrelevant digression about stack management), they describe Windows API functions that are required for using fibers and seem to suggest that fibers are OS-level, and not application-managed. Am I missing something? [edit: typo]

ojkelly 3y ago
My general understanding is that a fiber is a lightweight thread, and conceptually can be implemented inside any program that wants to schedule its own fibers. They’re distinct from CPU threads. When I last dived into fibers I discovered windows was trying to make them a thing quite a while ago which is why we have things like [0]. Fibers being virtual threads, are recursive. In the same way you can run a VM inside a VM, you could build fibers on top of fibers (which sounds like a recipe for unpredictable performance). [0] https://learn.microsoft.com/en-us/windows/win32/procthread/using-fibers https://learn.microsoft.com/en-us/windows/win32/procthread/u...
pvg 3y ago
Windows has its own fibers facility which does some things for you but still leaves the scheduling to the application so it's still fibers. It's a little confusing in the article but they look at the Windows API as a kind of API template - what do you need to implement to have fibers. Then they describe their own implementation.
j16sdiz 3y ago
Fiber is a concept in Windows API for user-space thread. It was designed for MS SQL Server. It have a very confusing API and did not gain much use in 3rd party applications.
nickelpro 3y ago
The Windows fiber library is just that, a library. It's not an OS component. It's provided because stack switching in Windows is a somewhat fraught [1] and not officially documented procedure which requires modifying the TIB [2]. Doing it correctly isn't hard, and has been done many times in bullet-proof libs (notably boost::context), but MS would maybe rather you not. [1]: https://devblogs.microsoft.com/oldnewthing/20080215-00/?p=23443 https://devblogs.microsoft.com/oldnewthing/20080215-00/?p=23... [2]: https://en.wikipedia.org/wiki/Win32_Thread_Information_Block https://en.wikipedia.org/wiki/Win32_Thread_Information_Block

smallstepforman 3y ago

Again, academia and the tech press overlook to talk about Actors, and we live in a world with poorly implemented pretend Actor subsets, without any of the true Actor features (work stealing, message passing etc, and best of all, no contention since no shared resources, other than Actor references). Fibres, goroutines, promises, futures, job queues, etc, are poor substitutes for Actors.

deleted 3y ago
[deleted]
grumpyprole 3y ago
> Fibres, goroutines, promises, futures, job queues, etc, are poor substitutes for Actors. They are not necessarily substitutes, they are lower level primitives that allow more complex models, like Actors, to be implemented. All execution models have trade offs; and sometimes we might prefer an execution model without deadlock, livelock and starvation. So for a general purpose language, a complex model like Actors is ideally a library and not enforced as the only way of achieving concurrency.
jeffreygoesto 3y ago
No shared resources can be a performance problem, I can see why there are rusts and vales efforts to come to a slightly different tradeoff.
- bmacho 3y ago
  Pony/Ponylang claims that they can do actors fast and safe (even safer than rust). There is a type system that allows one to share data between actors safely, or so they claim.
  - vacuity 3y ago
    I believe they use a GC, so they may be able to leverage the runtime to help out. If it's all type system-based, the compiler probably has to be stricter to ensure safety (Rice's theorem implies that compilers err on the side of rejecting some valid programs in order to reject all invalid programs). Maybe Pony could be a blend of Erlang and Rust?

endorphine 3y ago

> Fiber is quite a lightweight thread of execution. Like coroutine, fiber allows yielding at any point inside it. To some degree, we can regard fiber as a form of stackful coroutine [...] Can someone explain what the "stackful coroutine" term means (ideally with an example)? And how can one implement it?

nickelpro 3y ago
It's exactly what it says, it's a coroutine with its own stack. A fiber isn't to some degree a stackful coroutine, it is a stackful coroutine. A stackless coroutine does not have a its own stack, it uses the calling context's stack and therefore can only yield to the calling context from the top-level coroutine function. A stackful coroutine has its own stack, and so can yield to the calling context from anywhere. As far as how to implement, I always liked Malte Skarupke's blog post on the subject: https://probablydance.com/2013/02/20/handmade-coroutines-for-windows/ https://probablydance.com/2013/02/20/handmade-coroutines-for...
nstbayless 3y ago
I assume it's this: In Python, coroutines cannot yield from within another function call: if coroutine A calls function B, B cannot yield. In Lua, it's possible to yield a coroutine at any function depth: B can yield. To implement something like this in a compiled language, you just need multiple stacks instead of the usual 1. Most architectures, such as x86, have a stack pointer register; when yielding a coroutine, just change the sp register to point to some other stack -- when resuming, restore the sp. This is not a new concept. Pokémon on the gameboy did this for its UI fiber, for example.
- geertj 3y ago
  > In Python, coroutines cannot yield from within another function call: if coroutine A calls function B, B cannot yield. I know you likely know this, but just for context. This is technically correct of course, B cannot yield, but not how you would use it in real life. If B needs to yield because it calls C which need to yield, then B needs to be a coroutine itself. This allows for a ‘nested yield’ where A, B, and C are all paused by yielding to their direct parent. Python made this explicit before async/await as you had to use “yield from” instead of just yield. With async/await that became “await”. This does lead to what’s typically mentioned as the biggest disadvantage of stackless coroutines which is that you get parallel sets of functions, one async and one sync. This mostly means that IO and networking libraries end up being either sync or async but not both.

indigoabstract 3y ago

Since coroutines have been in the standard since C++20, I wish he would have explained better what are the advantages of using a Fiber over a Coroutine.

munch117 3y ago
Advantage: No function colouring. Just like with threads, you don't need to declare which functions might potentially yield. There's none of that noisy async/await syntax. Advantage: Fast switching. Switching between fibers is done by swapping in new values for the program counter and stack pointer. Compare with async, where yielding only takes you up one level of stack, so your code ends up walking a stack of coroutine invocations to get all the way back to the scheduler. Disadvantage: Expensive setup - each fiber needs its own pre-allocated stack. Disadvantage: Operating systems don't provide I/O systems readymade to work with fiber schedulers, so if you want to yield to a different fiber during blocking I/O, then you need to invent your own system to handle that. PS about naming: "Fiber" is the Microsoft Windows name for their implementation of full stack coroutines, but the proper platform-independent name for it is actually "coroutine". I've kept with the word "fiber" here to avoid misunderstandings. PS about C++: This is based on async like you find it in C# or Python, I don't really know about C++ coroutines.
- geertj 3y ago
  A disadvantage of the ‘no function coloring’ in fibers is that it makes lockless programming harder. A nested function call can switch from under you without your knowledge, making it hard to know where the preemption points are and whether to take locks when making updates to shared state. With function coloring you immediately see whether a function might switch as there’s a different calling syntax. I’ve programmed both fiber based systems and coroutines. I even created my own fiber libraries for Python (https://github.com/geertj/gruvi https://github.com/geertj/gruvi) and C++ (https://github.com/geertj/cgreenlet https://github.com/geertj/cgreenlet, mostly an experiment, and incorrectly named coroutines for C++ while it’s really fibers). In the Python version I experimented with some features to help you know whether a nested function might switch. In the end, for me and for the problem domains I worked in, the explicit async/await co-routine style wins over fibers. It gives you most of the performance benefits of user mode switching, all of the memory benefits, while keeping your code mostly lock free.

syncurrent 3y ago

Somewhat related to Fibres are the Trails of synchronous reactive programming languages. Both allow efficient logical concurrency based on cooperative scheduling. The nice thing with Trails is that the scheduling can be determined by the compiler by extracting dependencies from the synchronous reactive program thus increasing determinism of your app.

diffuse_l 3y ago

I believe that fibers are considered more trouble than they worth? [0] [0] https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=102989 https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10...

csb6 3y ago
The Gor Nishanov paper linked in that piece goes into a lot of detail of some of the downsides of fibers in an unmanaged language like C++. [0] http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
- teleforce 3y ago
  Thanks for the link to the Gor's paper. Apparently the 2018 paper table only covered top 10 languages in the TIOBE index, I wished it can be updated and extended further to top 50 languages. Another popular compiled language with GC (by default) namely D language (TIOBE ranked 37 as of Sept 2023) does support fiber and it's implemented in its vibe.d web framework [1],[2]. Gor also mentioned about significant 160 ns overhead for stackful fiber in Go (goroutine) to interact with a C library i.e. cost of switching between Go's goroutines and C threads (as of 2018). It will be interesting to know the fiber overhead in D language since C compilation is now natively supported by D compiler [3]. [1] Fibers in Programming in D: https://ddili.org/ders/d.en/fibers.html https://ddili.org/ders/d.en/fibers.html [2] Fibers, what for: https://forum.dlang.org/thread/eowfajvnzqnncmfnjhnf@forum.dlang.org https://forum.dlang.org/thread/eowfajvnzqnncmfnjhnf@forum.dl... [3] Adding ANSI C11 C compiler to D so it can import and compile C files directly: https://news.ycombinator.com/item?id=27102584 https://news.ycombinator.com/item?id=27102584

PaulDavisThe1st 3y ago

Remind me again why no OS other than Solaris ever provided "scheduler activations" (i.e. the kernel calls back into a user-space scheduler whenever a kernel-level scheduling event occurs)

kevingadd 3y ago
Windows had a custom scheduler API (https://learn.microsoft.com/en-us/windows/win32/procthread/user-mode-scheduling https://learn.microsoft.com/en-us/windows/win32/procthread/u...) but they removed it, presumably because it wasn't used much
- vacuity 3y ago
  As the Wikipedia page shows, NetBSD had an implementation but later removed it. There isn't really enough information to make a sound judgement on scheduler activations. Although if I had to guess, there might be too much overhead (maintenance of code and/or actual performance) for something that is rarely used. It might make more sense now that we're moving towards user-level concurrency everywhere (async/await, goroutines, virtual threads, etc.).
  - gpderetta 3y ago
    Basically yes. M:N just wasn't a good fit for the way applications used (and use) pthreads. For those that need pure CPU parallelism M:N is just useless overhead, while highly concurrent applications were either using one process per request for simplicity and robustness, or directly building on top of poll/epoll/kqueue for performance. So, at least at that time, M:N and scheduler activation was a solution in search of a problem. It might still see a resurgence on one way or another.
    - PaulDavisThe1st 3y ago
      But that's not what SA was about. Even with N:N architecture, if one kernel thread blocks (say, on I/O), the point is for the user-space scheduler to decide which thread should run next. The point was that the kernel scheduler cannot/should not understand the application thread scheduling requirements, and that when a kernel thread allocated to the task blocks, only user space can (correctly) decide what to do.

frozenport 3y ago

Some readers may have a question by now. What is the rationale behind the choices of the registers that need to be stored? Yet another article that goes into the gorey details of how fiber works but not how fibers are used. How do we write the "Applications own scheduler"?

OnlyMortal 3y ago

I’ve plenty of Classic Mac experience with cooperative threads. You’d basically yield to run another “thread”. Can someone tell me the difference from those to fibres?

ylow 3y ago

Some random historical notes coroutine stuff which are perhaps of interest to some: 1. You can implement "stack-free coroutines" in C with some really entertaining macros https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html 2. Does anyone remembers Cilk? Its still the fastest stackful coroutine models I know of. Having implemented and used both before (for different purposes), I like both, but with a mild preference for the stackful version. Largely because you don't have function coloring problems, and you can actually get a proper stack trace and is so significantly easier to debug when things go wrong. It is also good to differentiate coroutines from async (they seem to be very interleaved these days). Coroutines are a mechanic to achieve mutual recursion / generators. And that is fine for expressing certain algorithms and systems in a much cleaner fashion. Note that parallelism is not necessarily implied by "coroutine". Async is a mechanic to "doing something else" while waiting for IO. Fibers or state machines are both different solutions to this problem. Certain coroutine implementations can help with this, but I think it is massively overused. Rust due to the function coloring problem, ends up requiring almost everything to be marked "async". And some golang code I have read seems to overuse goroutines unecessarily. I think use of async should be narrow and minimized, and localized to only the places where it makes sense.

gpderetta 3y ago
> Does anyone remembers Cilk? Yes! It's a shame it never gained traction and Intel abandoned it, eventually getting dropped from icc and gcc.