3 ms·

Can someone explain this a bit more? Is this closer to say Goroutines for user space, with a light scheduler but handled by the kernel?

ActorNightly 6y ago
You basically have kernel threads, in which you can have more than one user space threads. The advantage is that you can context switch between threads without having to syscall, which speeds up things substantially.
- wahern 6y ago
  > The advantage is that you can context switch between threads without having to syscall, which speeds up things substantially. There's still a syscall, but it doesn't enter the kernel scheduler or other machinery, which is where most of the time is spent. Linux syscall overhead (switching into ring 0) was and remains (apparently[1]) miniscule, which is why Linux ended up with 1:1 thread scheduling. But everything else--what those syscalls do--has grown in complexity. The original calculus for rejecting userspace scheduler activations and M:N threading for 1:1 threading no longer holds. What's old is new again. [1] Would be interesting to know if Google's kernels employ all the new Spectre mitigations.
  - anitil 6y ago
    I'm only part way through the proposal, but perhaps this would be a candidate for being placed in the vDSO so there's no context switch at all.
    - wahern 6y ago
      A vDSO can't transfer execution to another thread. Theoretically a vDSO could manipulate a thread's scheduling context to move it up or down a queue, but it couldn't force immediate execution of another thread, couldn't suspend the current thread, and definitely couldn't do both those things simultaneously, which is what the patch does.
pcwalton 6y ago
The idea is to get the benefits of things like goroutines in a 1:1 framework. The main inherent advantage of userspace threads like goroutines is that you can context switch between them without going through the OS scheduler. Although the OS scheduler is quite efficient, sometimes you know that you want to immediately run some other thread, perhaps because you're passing a synchronous message to it. The reason why you would want this instead of the traditional implementation of goroutines is that it's compatible with existing code. When you buy into an M:N framework you lose compatibility with all existing libraries, because they could use things like blocking I/O, thread-local storage, or signals. The traditional solution to this that e.g. Go uses is to have some OS threads that you proxy all such calls from your userland M:N threads to. This works, but it's slow, and if you forget to use that mechanism you can end up starving your OS threads or worse. With the switchto patches here, you can get the best of both worlds: fast and precise control over thread scheduling from userland and compatibility with existing libraries. Ideally, it'd enable interoperability among different threading implementations: you could imagine that calling C++ or Rust functions from Go goroutines could become as fast as calling Go functions (though you would still have to deal with the small stack problem).
- CraftThatBlock 6y ago
  Thanks, this was really helpful!

ActorNightly 6y ago

You basically have kernel threads, in which you can have more than one user space threads. The advantage is that you can context switch between threads without having to syscall, which speeds up things substantially.

wahern 6y ago
> The advantage is that you can context switch between threads without having to syscall, which speeds up things substantially. There's still a syscall, but it doesn't enter the kernel scheduler or other machinery, which is where most of the time is spent. Linux syscall overhead (switching into ring 0) was and remains (apparently[1]) miniscule, which is why Linux ended up with 1:1 thread scheduling. But everything else--what those syscalls do--has grown in complexity. The original calculus for rejecting userspace scheduler activations and M:N threading for 1:1 threading no longer holds. What's old is new again. [1] Would be interesting to know if Google's kernels employ all the new Spectre mitigations.
- anitil 6y ago
  I'm only part way through the proposal, but perhaps this would be a candidate for being placed in the vDSO so there's no context switch at all.
  - wahern 6y ago
    A vDSO can't transfer execution to another thread. Theoretically a vDSO could manipulate a thread's scheduling context to move it up or down a queue, but it couldn't force immediate execution of another thread, couldn't suspend the current thread, and definitely couldn't do both those things simultaneously, which is what the patch does.

pcwalton 6y ago

The idea is to get the benefits of things like goroutines in a 1:1 framework. The main inherent advantage of userspace threads like goroutines is that you can context switch between them without going through the OS scheduler. Although the OS scheduler is quite efficient, sometimes you know that you want to immediately run some other thread, perhaps because you're passing a synchronous message to it. The reason why you would want this instead of the traditional implementation of goroutines is that it's compatible with existing code. When you buy into an M:N framework you lose compatibility with all existing libraries, because they could use things like blocking I/O, thread-local storage, or signals. The traditional solution to this that e.g. Go uses is to have some OS threads that you proxy all such calls from your userland M:N threads to. This works, but it's slow, and if you forget to use that mechanism you can end up starving your OS threads or worse. With the switchto patches here, you can get the best of both worlds: fast and precise control over thread scheduling from userland and compatibility with existing libraries. Ideally, it'd enable interoperability among different threading implementations: you could imagine that calling C++ or Rust functions from Go goroutines could become as fast as calling Go functions (though you would still have to deal with the small stack problem).

CraftThatBlock 6y ago
Thanks, this was really helpful!