5 ms·

Fiber[1] is another term for coroutines. They are essentially an implementation of cooperative multithreading. [1] https://en.wikipedia.org/wiki/Fiber_%28compu

by Aeolos 10y ago

Fiber[1] is another term for coroutines. They are essentially an implementation of cooperative multithreading. [1] https://en.wikipedia.org/wiki/Fiber_%28computer_science%29 https://en.wikipedia.org/wiki/Fiber_%28computer_science%29

warmwaffles 10y ago
Any simple C libraries that demonstrate this? I'm rather curious.
- MarkSweep 10y ago
  See libtask: https://swtch.com/libtask/ https://swtch.com/libtask/
  - aidenn0 10y ago
    And mongrel2, which I hack on occasionally uses libtask to good effect for this. It's architecture is much simpler than this presentation though because mongrel2 is one thread per process, so inside the process there is only one fiber executing at any given time.
- corysama 10y ago
  There's no official support for fibers in Posix. There are lots of implementations anyway, but it's hard to get completely right. Long ago, the Win32 team got tired of people rolling their own subtley incorrect/incomplete fiber implementations for Windows and added official support to put an end to that support issue https://msdn.microsoft.com/en-us/library/windows/desktop/ms682661.aspx https://msdn.microsoft.com/en-us/library/windows/desktop/ms6...
  - btrask 10y ago
    Actually, there is: https://en.wikipedia.org/wiki/Setcontext https://en.wikipedia.org/wiki/Setcontext A library I personally recommend, which supports several platforms and back-ends: http://byuu.org/library/libco/ http://byuu.org/library/libco/
    - deleted 10y ago
      [deleted]
    - sillysaurus3 10y ago
      setcontext isn't Posix, so corysama is the best kind of correct. But setcontext exists on most flavors of Linux, FreeBSD, and OS X, so you can basically assume it's supported.
      - gpderetta 10y ago
        setcontext was posix until POSIX.2008.
        deleted 10y ago
        [deleted]
    - david-given 10y ago
      A few years back I built an app which used coroutines extensively (<plug> greylisting SMTP proxy, now abandoned because I don't run my own mailserver any more, http://cowlark.com/spey/ http://cowlark.com/spey/ </plug>). I originally used getcontext/setcontext. Superficially it was really easy, but turned into a goddamned nightmare. The problem is that pthreads gets on really badly with user-allocated stacks; some pthreads implementations find the current thread ID by looking at a magic value on the current thread's stack, and of course with a user-allocated stack this wasn't there. But why was I using pthreads alongside coroutines, you ask? Well, I wasn't. But I was linking to a library which was referring to pthreads symbols, even though it wasn't calling them; which made the linker automatically pull in the thread-safe glibc, so malloc/free ended up wrapping in mutexes... The net effect was bizarre and infrequent crashes that only showed up on machines which weren't mine and that I didn't have access to. I spent months debugging that. Eventually I switched to simulating coroutines using pthreads with a big lock so that only one thread could run at a time. (Which actually turned out to be better, because now I could do some operations properly in parallel.) Beware of setcontext! Bewaaaaaare!
    - byuu 10y ago
      Just a fair warning, setcontext is horrendously slow. It seems to be doing kernel transitions or something. It's about 600 times slower than longjmp with jmpbuf stack pointer hacking, which is itself already about twice as slow as an ideal assembly version.
  - shinymark 10y ago
    Our team just got hit by this. We are using fibers in a custom C++ game engine without issue on Windows, OSX, and some console platforms. Recently we began the work to bring the code base to iOS. Guess what? No fiber support. Doh!
    - corysama 10y ago
      Looks [1] like http://byuu.org/library/libco/ http://byuu.org/library/libco/ (mentioned below) might work on iOS. I would definitely be interested in fiber libs that are confirmed to work on iOS and/or Android if anyone knows any. libco looks like it might fit the bill. But, it's not completely clear and I have not tried it. [1] http://libretro.com/forums/archive/index.php?t-785.html http://libretro.com/forums/archive/index.php?t-785.html
      - clevernickname 10y ago
        libco relies on being able to map WX pages for a rather silly reason[1], so it won't run on iOS without some work. [1] The author didn't like requiring assemblers to build his projects, so instead the coroutine switch machine code is embedded in the binary as a byte array, which is then copied to a page with WX permissions when the library is initialized. https://github.com/creationix/libco/blob/master/amd64.c https://github.com/creationix/libco/blob/master/amd64.c
        byuu 10y ago
        Why do you feel [1] is a silly reason? You can build my software right now in one step: "install TDM-GCC"; anything additional complicates things and reduces portability. There's also versions that don't require assembler code at all (setjmp/longjmp version), and you can also make an ios.c version that has inline assembler since that target will only ever have a single compiler anyway.
        clevernickname 10y ago
        Ironic in light of your comments on the OpenBSD forced W^X thing, but I happen to agree that strict W^X should be used everywhere possible. It may not matter for higan (though it would be funny to see someone exploit it), but it could be a deal breaker for more security-conscious projects that might want to use libco (say, for a cothreaded server). As for assemblers, I just don't see how requiring gas or maybe NASM/YASM for x86 would complicate the build process so much. Heck, gas is a dependency of gcc. The only problem I see is for Visual Studio users, which could be solved with using NASM/YASM for Windows (which is not really a big deal when the co_swap_function has to be different due to the calling conventions anyway), or a separate MASM implementation for Visual Studio (giving you duplicate implementations with VS Windows and gcc Windows on x86 and x86_64 respectively). I know you're consistent with your views on code clarity and that this is a hobby for you etc, but libco seems like a tiny, static enough library that it would make sense to accept a little ugliness in order to Do The Right Thing™. (PS: Thank you for higan, I've learned so much from it over the years.)
- notduncansmith 10y ago
  Libmill could be worth a look. https://github.com/sustrik/libmill https://github.com/sustrik/libmill
deleted 10y ago
[deleted]

by Aeolos 10y ago

warmwaffles 10y ago

Any simple C libraries that demonstrate this? I'm rather curious.

MarkSweep 10y ago
See libtask: https://swtch.com/libtask/ https://swtch.com/libtask/
- aidenn0 10y ago
  And mongrel2, which I hack on occasionally uses libtask to good effect for this. It's architecture is much simpler than this presentation though because mongrel2 is one thread per process, so inside the process there is only one fiber executing at any given time.
corysama 10y ago
There's no official support for fibers in Posix. There are lots of implementations anyway, but it's hard to get completely right. Long ago, the Win32 team got tired of people rolling their own subtley incorrect/incomplete fiber implementations for Windows and added official support to put an end to that support issue https://msdn.microsoft.com/en-us/library/windows/desktop/ms682661.aspx https://msdn.microsoft.com/en-us/library/windows/desktop/ms6...
- btrask 10y ago
  Actually, there is: https://en.wikipedia.org/wiki/Setcontext https://en.wikipedia.org/wiki/Setcontext A library I personally recommend, which supports several platforms and back-ends: http://byuu.org/library/libco/ http://byuu.org/library/libco/
  - deleted 10y ago
    [deleted]
  - sillysaurus3 10y ago
    setcontext isn't Posix, so corysama is the best kind of correct. But setcontext exists on most flavors of Linux, FreeBSD, and OS X, so you can basically assume it's supported.
    - gpderetta 10y ago
      setcontext was posix until POSIX.2008.
      - deleted 10y ago
        [deleted]
  - david-given 10y ago
    A few years back I built an app which used coroutines extensively (<plug> greylisting SMTP proxy, now abandoned because I don't run my own mailserver any more, http://cowlark.com/spey/ http://cowlark.com/spey/ </plug>). I originally used getcontext/setcontext. Superficially it was really easy, but turned into a goddamned nightmare. The problem is that pthreads gets on really badly with user-allocated stacks; some pthreads implementations find the current thread ID by looking at a magic value on the current thread's stack, and of course with a user-allocated stack this wasn't there. But why was I using pthreads alongside coroutines, you ask? Well, I wasn't. But I was linking to a library which was referring to pthreads symbols, even though it wasn't calling them; which made the linker automatically pull in the thread-safe glibc, so malloc/free ended up wrapping in mutexes... The net effect was bizarre and infrequent crashes that only showed up on machines which weren't mine and that I didn't have access to. I spent months debugging that. Eventually I switched to simulating coroutines using pthreads with a big lock so that only one thread could run at a time. (Which actually turned out to be better, because now I could do some operations properly in parallel.) Beware of setcontext! Bewaaaaaare!
  - byuu 10y ago
    Just a fair warning, setcontext is horrendously slow. It seems to be doing kernel transitions or something. It's about 600 times slower than longjmp with jmpbuf stack pointer hacking, which is itself already about twice as slow as an ideal assembly version.
- shinymark 10y ago
  Our team just got hit by this. We are using fibers in a custom C++ game engine without issue on Windows, OSX, and some console platforms. Recently we began the work to bring the code base to iOS. Guess what? No fiber support. Doh!
  - corysama 10y ago
    Looks [1] like http://byuu.org/library/libco/ http://byuu.org/library/libco/ (mentioned below) might work on iOS. I would definitely be interested in fiber libs that are confirmed to work on iOS and/or Android if anyone knows any. libco looks like it might fit the bill. But, it's not completely clear and I have not tried it. [1] http://libretro.com/forums/archive/index.php?t-785.html http://libretro.com/forums/archive/index.php?t-785.html
    - clevernickname 10y ago
      libco relies on being able to map WX pages for a rather silly reason[1], so it won't run on iOS without some work. [1] The author didn't like requiring assemblers to build his projects, so instead the coroutine switch machine code is embedded in the binary as a byte array, which is then copied to a page with WX permissions when the library is initialized. https://github.com/creationix/libco/blob/master/amd64.c https://github.com/creationix/libco/blob/master/amd64.c
      - byuu 10y ago
        Why do you feel [1] is a silly reason? You can build my software right now in one step: "install TDM-GCC"; anything additional complicates things and reduces portability. There's also versions that don't require assembler code at all (setjmp/longjmp version), and you can also make an ios.c version that has inline assembler since that target will only ever have a single compiler anyway.
        clevernickname 10y ago
        Ironic in light of your comments on the OpenBSD forced W^X thing, but I happen to agree that strict W^X should be used everywhere possible. It may not matter for higan (though it would be funny to see someone exploit it), but it could be a deal breaker for more security-conscious projects that might want to use libco (say, for a cothreaded server). As for assemblers, I just don't see how requiring gas or maybe NASM/YASM for x86 would complicate the build process so much. Heck, gas is a dependency of gcc. The only problem I see is for Visual Studio users, which could be solved with using NASM/YASM for Windows (which is not really a big deal when the co_swap_function has to be different due to the calling conventions anyway), or a separate MASM implementation for Visual Studio (giving you duplicate implementations with VS Windows and gcc Windows on x86 and x86_64 respectively). I know you're consistent with your views on code clarity and that this is a hobby for you etc, but libco seems like a tiny, static enough library that it would make sense to accept a little ugliness in order to Do The Right Thing™. (PS: Thank you for higan, I've learned so much from it over the years.)
notduncansmith 10y ago
Libmill could be worth a look. https://github.com/sustrik/libmill https://github.com/sustrik/libmill

deleted 10y ago

[deleted]