3 ms·

>> dynamic stack allocation of coroutines, which can probably be optimized away anyway This seems interesting. Do you have any pointers to places/papers I can

>> dynamic stack allocation of coroutines, which can probably be optimized away anyway This seems interesting. Do you have any pointers to places/papers I can look more into this? I'm also curious, since the stacks have to be rather small when you are running several thousands of coroutines (like Go), how often people get into issues of running out of stack because of some big stack allocation somewhere and stuff like that.

zackmorris 7y ago
I haven't studied it deeply, but a breadcrumb would be that cooperative threads (green threads) are equivalent to coroutines. Ok it looks like current techniques are stackless runtimes and compiling coroutines to stackless continuations: https://en.wikipedia.org/wiki/Stackless_Python https://en.wikipedia.org/wiki/Stackless_Python https://stackless.readthedocs.io/en/v3.6.4-slp/library/stackless/stackless.html https://stackless.readthedocs.io/en/v3.6.4-slp/library/stack... http://jessenoller.com/blog/2009/02/23/stackless-you-got-your-coroutines-in-my-subroutines http://jessenoller.com/blog/2009/02/23/stackless-you-got-you... https://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1346&context=enece_facpub https://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?r... https://pdfs.semanticscholar.org/b9aa/49e4b7a00e6c9f0d8c18ba9240c1223bc098.pdf https://pdfs.semanticscholar.org/b9aa/49e4b7a00e6c9f0d8c18ba... https://www.osnews.com/story/9822/protothreads-extremely-lightweight-stackless-threads-in-c/ https://www.osnews.com/story/9822/protothreads-extremely-lig... This looks like a rare gem, although I just started reading it: https://cs.indiana.edu/~dfried/dfried/mex.pdf https://cs.indiana.edu/~dfried/dfried/mex.pdf I grew up with the cooperative multithreading of classic Mac OS and was really shocked when I first saw Javascript back in the 90s and it had no notion of it (because it didn't have generators). That sent us down the callback hell evolutionary dead end, through promises/futures and finally to async/await where we are now. That could have been largely avoided if we had listened to programming language experts!
- truncate 7y ago
  Oh I think I misundetstood you (or you me), and I didn't really articulate my question well. I'm well aware of how stackless coroutines are implemented. My main concern was whenever you do that, you lose the ability to look at backtrace when something goes wrong. So some implementations keep a separate stack for each coroutine (like Go), and switch SP whenever context is switched. That way you don't have bunch of random function calls in your stack trace. The problem though is that these individual stack for each couroutine has to be pretty small in general (since you would spawn hundreds of thousands of these). Go solved this temporarily using fragmented stack, and later ditched that in favor of copying the whole stack over. And I thought, you were talking about optimizations around this whole "stackfull" coroutines thing so that I can have my backtrace.

zackmorris 7y ago

I haven't studied it deeply, but a breadcrumb would be that cooperative threads (green threads) are equivalent to coroutines. Ok it looks like current techniques are stackless runtimes and compiling coroutines to stackless continuations: https://en.wikipedia.org/wiki/Stackless_Python https://en.wikipedia.org/wiki/Stackless_Python https://stackless.readthedocs.io/en/v3.6.4-slp/library/stackless/stackless.html https://stackless.readthedocs.io/en/v3.6.4-slp/library/stack... http://jessenoller.com/blog/2009/02/23/stackless-you-got-your-coroutines-in-my-subroutines http://jessenoller.com/blog/2009/02/23/stackless-you-got-you... https://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1346&context=enece_facpub https://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?r... https://pdfs.semanticscholar.org/b9aa/49e4b7a00e6c9f0d8c18ba9240c1223bc098.pdf https://pdfs.semanticscholar.org/b9aa/49e4b7a00e6c9f0d8c18ba... https://www.osnews.com/story/9822/protothreads-extremely-lightweight-stackless-threads-in-c/ https://www.osnews.com/story/9822/protothreads-extremely-lig... This looks like a rare gem, although I just started reading it: https://cs.indiana.edu/~dfried/dfried/mex.pdf https://cs.indiana.edu/~dfried/dfried/mex.pdf I grew up with the cooperative multithreading of classic Mac OS and was really shocked when I first saw Javascript back in the 90s and it had no notion of it (because it didn't have generators). That sent us down the callback hell evolutionary dead end, through promises/futures and finally to async/await where we are now. That could have been largely avoided if we had listened to programming language experts!

truncate 7y ago
Oh I think I misundetstood you (or you me), and I didn't really articulate my question well. I'm well aware of how stackless coroutines are implemented. My main concern was whenever you do that, you lose the ability to look at backtrace when something goes wrong. So some implementations keep a separate stack for each coroutine (like Go), and switch SP whenever context is switched. That way you don't have bunch of random function calls in your stack trace. The problem though is that these individual stack for each couroutine has to be pretty small in general (since you would spawn hundreds of thousands of these). Go solved this temporarily using fragmented stack, and later ditched that in favor of copying the whole stack over. And I thought, you were talking about optimizations around this whole "stackfull" coroutines thing so that I can have my backtrace.