5 ms·
Google proposes user threads for Linux
- doener 6y agoVia https://www.golem.de/news/kernel-google-will-user-threads-fuer-linux-2007-149885.html https://www.golem.de/news/kernel-google-will-user-threads-fu...
- ITisINwhomo 6y agoGerman is gibberish; but whatever; hope the patch or idea is a good thing. <3 Thanks for reporting dude.
- pcwalton 6y agoSweet! I've been asking for this for a while. This seems to me the best way to do M:N threading: with true kernel support.
- CraftThatBlock 6y agoCan someone explain this a bit more? Is this closer to say Goroutines for user space, with a light scheduler but handled by the kernel?
- ActorNightly 6y agoYou basically have kernel threads, in which you can have more than one user space threads. The advantage is that you can context switch between threads without having to syscall, which speeds up things substantially.
- wahern 6y ago> The advantage is that you can context switch between threads without having to syscall, which speeds up things substantially. There's still a syscall, but it doesn't enter the kernel scheduler or other machinery, which is where most of the time is spent. Linux syscall overhead (switching into ring 0) was and remains (apparently[1]) miniscule, which is why Linux ended up with 1:1 thread scheduling. But everything else--what those syscalls do--has grown in complexity. The original calculus for rejecting userspace scheduler activations and M:N threading for 1:1 threading no longer holds. What's old is new again. [1] Would be interesting to know if Google's kernels employ all the new Spectre mitigations.
- anitil 6y agoI'm only part way through the proposal, but perhaps this would be a candidate for being placed in the vDSO so there's no context switch at all.
- wahern 6y agoA vDSO can't transfer execution to another thread. Theoretically a vDSO could manipulate a thread's scheduling context to move it up or down a queue, but it couldn't force immediate execution of another thread, couldn't suspend the current thread, and definitely couldn't do both those things simultaneously, which is what the patch does.
- pcwalton 6y agoThe idea is to get the benefits of things like goroutines in a 1:1 framework. The main inherent advantage of userspace threads like goroutines is that you can context switch between them without going through the OS scheduler. Although the OS scheduler is quite efficient, sometimes you know that you want to immediately run some other thread, perhaps because you're passing a synchronous message to it. The reason why you would want this instead of the traditional implementation of goroutines is that it's compatible with existing code. When you buy into an M:N framework you lose compatibility with all existing libraries, because they could use things like blocking I/O, thread-local storage, or signals. The traditional solution to this that e.g. Go uses is to have some OS threads that you proxy all such calls from your userland M:N threads to. This works, but it's slow, and if you forget to use that mechanism you can end up starving your OS threads or worse. With the switchto patches here, you can get the best of both worlds: fast and precise control over thread scheduling from userland and compatibility with existing libraries. Ideally, it'd enable interoperability among different threading implementations: you could imagine that calling C++ or Rust functions from Go goroutines could become as fast as calling Go functions (though you would still have to deal with the small stack problem).
- CraftThatBlock 6y agoThanks, this was really helpful!
- telotortium 6y agoWow, Google is finally open-sourcing this - it was first presented publicly in 2013.
- justaguy88 6y ago"The PDF and video can go pound sand; you get to fully explain things here."
- crmrc114 6y agohttps://lore.kernel.org/lkml/20200723112757.GN5523@worktop.programming.kicks-ass.net/ https://lore.kernel.org/lkml/20200723112757.GN5523@worktop.p... Yeah, that reply made me laugh. Never change Kernel mailing list, never change.
- The_rationalist 6y agoWhat pro and cons does it have versus project Loom?
- m0zg 6y agoThis sounds like Google Fibers, and it'll be pretty groundbreaking if this happens. It's basically an elegant, and extremely scalable way to do N:M threading which obviates the need for async in a lot of cases, and all the debugging complications that result from that. Your code looks "linear" and feels "linear". It's easy to debug and understand. Yet you get to have essentially as many threads as you like within each process, and you have effective, useful patterns to manage those threads, and reduce the opportunity to introduce threading bugs. That'd be one of the most welcome additions to the kernel in years IMO. And the change is fairly minimal.
- wilsonthewhale 6y agoI recall M:N threading being attempted in the past (in some BSD systems, at least), with mixed results ultimately resulting in a move to 1:1 threading for most of them. What's different about this attempt that puts it over the old? An honest question
- darthrupert 6y agoOh cool. I've been wondering to myself why the complexities of async cannot be abstracted into threads instead in an efficient way. I hope this pans out.