4 ms·

The main problems I've seen with cooperative threading (aka fibers) are threefold: 1. You need to decide how much stack space to allocate to a fiber beforehand

The main problems I've seen with cooperative threading (aka fibers) are threefold: 1. You need to decide how much stack space to allocate to a fiber beforehand. If you go over this limit, you will segfault. 2. Thread local storage doesnt exist anymore since you're potentially swapping between different kernel threads (and different fibers are using the same thread and will share storage). 3. Certain types of locks can now fail because the lock isnt transferred to the new thread upon being woken up. Other locks run into rentrancy problems here. Does anyone know of good solutions to these problems? Is there a library that takes all these things into account?

loeg 6y ago
(2) and (3) are sort of symptoms of trying to use the C runtime in what is definitely not a native C threading environment. (1), Golang does something like this by probing on every function call and having movable stacks. Historically they used a linked list of stack pages, but gave that up in 1.4 because the performance was bad. The same does not work in C because C compilers do not expect pointers to the stack to become invalid while a stack frame is live.
Reelin 6y ago
Regarding thread local storage - isn't that typically implemented using the segment registers (on x86 at least)? So you would need to vary those between fibers (instead of just between kernel threads) and your scheduler would have to explicitly save and restore them. I would guess that's not possible to do in portable C though.
Blackthorn 6y ago
I think there's a fourth problem that's even more dire: what do you do about blocking? A blocking call stalls the whole system. It doesn't in a non-cooperative threading environment. Various languages have "solved" this by not allowing you to use blocking calls and sometimes add syntactic sugar like async/await, but this sucks. The point of having threads is to use them like threads. At least Go gets this right.
- kccqzy 6y ago
  > Various languages have "solved" this by not allowing you to use blocking calls That is the right solution. Do you seriously think in Go when you make an apparently blocking system call, an actual blocking system call is made? No, you are calling the runtime provided version that doesn't actually block. Of course in C/C++ that needs more discipline to do.
  - Blackthorn 6y ago
    > Do you seriously think in Go when you make an apparently blocking system call, an actual blocking system call is made? Who cares? From the perspective of the programmer, it's a blocking call.
    - kccqzy 6y ago
      Then how is a syntactic construct like "await blocking_call()" inferior? I fail to see your point. The await keyword even signals to you the fact that this call may be blocking the progress of the current fiber/coroutine. And back to the original example, it's totally possible for a production implementation to handhold the user like this by, e.g. supplying a modified libc so that all the blocking system calls are linked against the runtime's implementation. That would basically achieve what you liked in Go.
      - Blackthorn 6y ago
        > Then how is a syntactic construct like "await blocking_call()" inferior? It's inferior because you have to actually specify it whenever you make such a call, rather than having the runtime or library be smart and figure it out. > And back to the original example, it's totally possible for a production implementation to handhold the user like this by, e.g. supplying a modified libc so that all the blocking system calls are linked against the runtime's implementation. I know. I didn't say it was intractable. Just like problems 1-3, they all have potential solutions -- they just involve hard work.
        6y ago
kccqzy 6y ago
1. Easy on a 64-bit system. Just allocate dozens of megabytes of virtual address space with the memory being not readable nor writable. Mark a small proportion as readable and writable. In response to segfaults, grow the stack by marking even more memory as readable and writable. My toy implementation does this. It's very easy to do in about 20 lines of code. 2. You will need to reimplement thread local storage yourself. A production-quality implementation should handle this for you. Thread local storage just means having a special section of memory as a read-only template, and then duplicate them into fresh pages upon the creation of the fiber, mark them as writable, call constructors. And then remember to save the segment registers when switching. Tedious but doable. 3. You need your own locks. This is apparent even in high-level languages like Python. Notice how Python asyncio provides its own mutexes and semaphores? A production quality implementation will handle this for you, but you'll need to use them instead of the OS-provided ones.

loeg 6y ago

(2) and (3) are sort of symptoms of trying to use the C runtime in what is definitely not a native C threading environment. (1), Golang does something like this by probing on every function call and having movable stacks. Historically they used a linked list of stack pages, but gave that up in 1.4 because the performance was bad. The same does not work in C because C compilers do not expect pointers to the stack to become invalid while a stack frame is live.

Reelin 6y ago

Regarding thread local storage - isn't that typically implemented using the segment registers (on x86 at least)? So you would need to vary those between fibers (instead of just between kernel threads) and your scheduler would have to explicitly save and restore them. I would guess that's not possible to do in portable C though.

Blackthorn 6y ago

I think there's a fourth problem that's even more dire: what do you do about blocking? A blocking call stalls the whole system. It doesn't in a non-cooperative threading environment. Various languages have "solved" this by not allowing you to use blocking calls and sometimes add syntactic sugar like async/await, but this sucks. The point of having threads is to use them like threads. At least Go gets this right.

kccqzy 6y ago
> Various languages have "solved" this by not allowing you to use blocking calls That is the right solution. Do you seriously think in Go when you make an apparently blocking system call, an actual blocking system call is made? No, you are calling the runtime provided version that doesn't actually block. Of course in C/C++ that needs more discipline to do.
- Blackthorn 6y ago
  > Do you seriously think in Go when you make an apparently blocking system call, an actual blocking system call is made? Who cares? From the perspective of the programmer, it's a blocking call.
  - kccqzy 6y ago
    Then how is a syntactic construct like "await blocking_call()" inferior? I fail to see your point. The await keyword even signals to you the fact that this call may be blocking the progress of the current fiber/coroutine. And back to the original example, it's totally possible for a production implementation to handhold the user like this by, e.g. supplying a modified libc so that all the blocking system calls are linked against the runtime's implementation. That would basically achieve what you liked in Go.
    - Blackthorn 6y ago
      > Then how is a syntactic construct like "await blocking_call()" inferior? It's inferior because you have to actually specify it whenever you make such a call, rather than having the runtime or library be smart and figure it out. > And back to the original example, it's totally possible for a production implementation to handhold the user like this by, e.g. supplying a modified libc so that all the blocking system calls are linked against the runtime's implementation. I know. I didn't say it was intractable. Just like problems 1-3, they all have potential solutions -- they just involve hard work.
      - 6y ago

kccqzy 6y ago

1. Easy on a 64-bit system. Just allocate dozens of megabytes of virtual address space with the memory being not readable nor writable. Mark a small proportion as readable and writable. In response to segfaults, grow the stack by marking even more memory as readable and writable. My toy implementation does this. It's very easy to do in about 20 lines of code. 2. You will need to reimplement thread local storage yourself. A production-quality implementation should handle this for you. Thread local storage just means having a special section of memory as a read-only template, and then duplicate them into fresh pages upon the creation of the fiber, mark them as writable, call constructors. And then remember to save the segment registers when switching. Tedious but doable. 3. You need your own locks. This is apparent even in high-level languages like Python. Notice how Python asyncio provides its own mutexes and semaphores? A production quality implementation will handle this for you, but you'll need to use them instead of the OS-provided ones.