4 ms·

The go runtime moves stacks between threads? Oof that’s horrible, any pointers to the logic behind it? I’m curious the rationale.

duelingjello 7y ago
I think to keep Go code directly callable from C, they have to follow the platform's C calling conventions which means the same stack layout. So for cooperative concurrency on a single thread to work, each Goroutine needs its very own stack. On Intel, that means saving stack pointers RSP and RBP (16 bytes) for each. Also, each will need memory allocated for its stack for the stack pointers to point to... another 8-16 bytes (pointer and length).
- echlebek 7y ago
  The gc compiler, used by the vast majority of Go developers, does not use the C calling convention. https://golang.org/doc/faq#Do_Go_programs_link_with_Cpp_programs https://golang.org/doc/faq#Do_Go_programs_link_with_Cpp_prog...
pcwalton 7y ago
The valid reasons for M:N threading are reducing syscalls on goroutine spawning, and avoiding the overhead of the kernel scheduler on context switch.
gok 7y ago
Like much of Go, it was likely done because it made it easier to recycle Plan 9 code.
- enneff 7y ago
  Not sure where you got this idea. The only significant part of the Go codebase that was inherited from Plan 9 was the C compilers, used to build the original Go compiler that was written (from scratch) in C. I think perhaps a hash table implementation was also brought over from P9. That stuff is all long gone now, though. The idea that Go's scheduler design is somehow inherited from Plan 9 is ridiculous.
benaadams 7y ago
There are two main ways to do async/concurrency where you release the thread to do other work while you are waiting. 1. stackless (async/await) where the operation becomes an inspectable object that you can choose what to do with (awaiting being suspend for completion) as taken by C#, C++, Python, JS, PHP, Swift and Rust 2. "with stack" where you pretend its not async; but this means when something else uses the thread you need to get the suspended operation's stuff off the thread; usually by not using the thread's stack at all and having it in the heap and just jumping into and out of these "off-thread" stacks; as used by Go and being looked at for Java (as Project Loom) Interesting paper on it http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136... > While fibers may have looked like an attractive approach to write scalable concurrent code in the 90s, the experience of using fibers, the advances in operating systems, hardware and compiler technology (stackless coroutines), made them no longer a recommended facility. Disadvantage of stackless is the extra boilerplate (e.g. async/await everywhere); though it also gives more control as the consumer of the operations (e.g. fanout and wait for many; or continue not waiting for the result at all) Advantage of the "with stack" approach is it looks the same as non async code as its all hidden (goroutines aside); which is why Java is no doubt looking at doing it as there is a large body of code that would need to be rewritten so "hiding it" is easier to avoid that. C# had/has teething issues when async/await was introduced as it kept the initial thread blocking methods; and added the async and they don't mix very well, you need really to go one way or the other when developing. Javascript leapt at async/await as it was all async anyway, but callback based which makes for horrible code to follow; so it made everything much cleaner.

duelingjello 7y ago

I think to keep Go code directly callable from C, they have to follow the platform's C calling conventions which means the same stack layout. So for cooperative concurrency on a single thread to work, each Goroutine needs its very own stack. On Intel, that means saving stack pointers RSP and RBP (16 bytes) for each. Also, each will need memory allocated for its stack for the stack pointers to point to... another 8-16 bytes (pointer and length).

echlebek 7y ago
The gc compiler, used by the vast majority of Go developers, does not use the C calling convention. https://golang.org/doc/faq#Do_Go_programs_link_with_Cpp_programs https://golang.org/doc/faq#Do_Go_programs_link_with_Cpp_prog...

pcwalton 7y ago

The valid reasons for M:N threading are reducing syscalls on goroutine spawning, and avoiding the overhead of the kernel scheduler on context switch.

gok 7y ago

Like much of Go, it was likely done because it made it easier to recycle Plan 9 code.

enneff 7y ago
Not sure where you got this idea. The only significant part of the Go codebase that was inherited from Plan 9 was the C compilers, used to build the original Go compiler that was written (from scratch) in C. I think perhaps a hash table implementation was also brought over from P9. That stuff is all long gone now, though. The idea that Go's scheduler design is somehow inherited from Plan 9 is ridiculous.

benaadams 7y ago

There are two main ways to do async/concurrency where you release the thread to do other work while you are waiting. 1. stackless (async/await) where the operation becomes an inspectable object that you can choose what to do with (awaiting being suspend for completion) as taken by C#, C++, Python, JS, PHP, Swift and Rust 2. "with stack" where you pretend its not async; but this means when something else uses the thread you need to get the suspended operation's stuff off the thread; usually by not using the thread's stack at all and having it in the heap and just jumping into and out of these "off-thread" stacks; as used by Go and being looked at for Java (as Project Loom) Interesting paper on it http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136... > While fibers may have looked like an attractive approach to write scalable concurrent code in the 90s, the experience of using fibers, the advances in operating systems, hardware and compiler technology (stackless coroutines), made them no longer a recommended facility. Disadvantage of stackless is the extra boilerplate (e.g. async/await everywhere); though it also gives more control as the consumer of the operations (e.g. fanout and wait for many; or continue not waiting for the result at all) Advantage of the "with stack" approach is it looks the same as non async code as its all hidden (goroutines aside); which is why Java is no doubt looking at doing it as there is a large body of code that would need to be rewritten so "hiding it" is easier to avoid that. C# had/has teething issues when async/await was introduced as it kept the initial thread blocking methods; and added the async and they don't mix very well, you need really to go one way or the other when developing. Javascript leapt at async/await as it was all async anyway, but callback based which makes for horrible code to follow; so it made everything much cleaner.