5 ms·

Seems like this would be more like tokio for Rust rather than Go's goroutines.

steveklabnik 9y ago
Yes, and we're exploring a similar thing in Rust right now, though we call them "generators." A key question right now is, can we have an implementation where the heap allocations that occur here don't have to? It's not 100% clear.
- tmandry 9y ago
  The post says that if a compiler can prove "that the lifetime of the coroutine is indeed strictly nested within the lifetime of the caller", it would be able to avoid heap allocations. It makes it sound like an optional optimization, though. I think Rust clearly has an advantage here: since lifetimes are well-defined concept in the language itself, it should also be well-defined whether a generator can be allocated on the stack. Of course, I'm probably missing some hairy edge cases here. Another aspect I found interesting was the ability to define custom code which runs every time a certain coroutine is suspended, or returns. In my reading, this allows for the coroutine to e.g. manage its own membership in a resume queue. Tokio tackles this problem differently, but is this more powerful than Rust coroutines? Is there a mechanism by which a generic type could be used to wrap a generator and provide this functionality in a similar way? Asking partly as a point of conversation, and partly as someone who isn't fully steeped in Rust's type system yet.
  - steveklabnik 9y ago
    Yes, the issue here is what happens on moves? https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-referential-structs/ https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-re... and the two follow up posts are the latest thinking on this topic, and by latest, I mean "posted in the last few days". This is cutting-edge stuff! The intention is for this stuff to support Tokio; that is, generators make asyc/await work, which lets you write futures code more easily, to be run with Tokio.
    - tmandry 9y ago
      Interesting. The fourth post from today seems very promising. It gives me increasing confidence that Rust has chosen its code abstractions very well, to know that we can provably solve this problem without heap allocations. The lifetime binding tricks feel reminiscent of template metaprogramming-fu in C++, though the parts that have to be reasoned about past the standard library seem smaller and cleaner. In Rust, it feels like we are leveraging our cleverness to accomplish a higher order of expression than what’s possible in other imperative languages today. In 20 years, maybe we’ll be talking about a promising new language concept that reminds of how exciting Rust lifetimes were back in 2018 :)
- GolDDranks 9y ago
  It's interesting that people refer the same thing with "generators", "semicoroutines" or "stackless coroutines". (A resumable computation that doesn't have a growable stack) On the other hand, "stackful coroutines" and "coroutines" without a qualifier often refers things with dynamically growing stacks. It seems to me that the C++ Coroutines refer to the former, which likely complicates the terminology further.
  - pjmlp 9y ago
    Because coroutines are the building blocks of generators. Being stackless is an implementation detail, even if a relevant one.
  - tpush 9y ago
    They are all specific restrictions on and/or implementations details of delimited continuations anyway :).
- mark_edward 9y ago
  As I understand, generators are a subset of coroutines. Generators can only yield to their caller, but coroutines can yield to other coroutines. coroutines are more general. they can be implemented on top of generators with some fancy stuff i don't remember involving returning tokens to some kind of scheduler/dispatcher in the caller

Yes, and we're exploring a similar thing in Rust right now, though we call them "generators." A key question right now is, can we have an implementation where the heap allocations that occur here don't have to? It's not 100% clear.

tmandry 9y ago
The post says that if a compiler can prove "that the lifetime of the coroutine is indeed strictly nested within the lifetime of the caller", it would be able to avoid heap allocations. It makes it sound like an optional optimization, though. I think Rust clearly has an advantage here: since lifetimes are well-defined concept in the language itself, it should also be well-defined whether a generator can be allocated on the stack. Of course, I'm probably missing some hairy edge cases here. Another aspect I found interesting was the ability to define custom code which runs every time a certain coroutine is suspended, or returns. In my reading, this allows for the coroutine to e.g. manage its own membership in a resume queue. Tokio tackles this problem differently, but is this more powerful than Rust coroutines? Is there a mechanism by which a generic type could be used to wrap a generator and provide this functionality in a similar way? Asking partly as a point of conversation, and partly as someone who isn't fully steeped in Rust's type system yet.
- steveklabnik 9y ago
  Yes, the issue here is what happens on moves? https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-referential-structs/ https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-re... and the two follow up posts are the latest thinking on this topic, and by latest, I mean "posted in the last few days". This is cutting-edge stuff! The intention is for this stuff to support Tokio; that is, generators make asyc/await work, which lets you write futures code more easily, to be run with Tokio.
  - tmandry 9y ago
    Interesting. The fourth post from today seems very promising. It gives me increasing confidence that Rust has chosen its code abstractions very well, to know that we can provably solve this problem without heap allocations. The lifetime binding tricks feel reminiscent of template metaprogramming-fu in C++, though the parts that have to be reasoned about past the standard library seem smaller and cleaner. In Rust, it feels like we are leveraging our cleverness to accomplish a higher order of expression than what’s possible in other imperative languages today. In 20 years, maybe we’ll be talking about a promising new language concept that reminds of how exciting Rust lifetimes were back in 2018 :)
GolDDranks 9y ago
It's interesting that people refer the same thing with "generators", "semicoroutines" or "stackless coroutines". (A resumable computation that doesn't have a growable stack) On the other hand, "stackful coroutines" and "coroutines" without a qualifier often refers things with dynamically growing stacks. It seems to me that the C++ Coroutines refer to the former, which likely complicates the terminology further.
- pjmlp 9y ago
  Because coroutines are the building blocks of generators. Being stackless is an implementation detail, even if a relevant one.
- tpush 9y ago
  They are all specific restrictions on and/or implementations details of delimited continuations anyway :).
mark_edward 9y ago
As I understand, generators are a subset of coroutines. Generators can only yield to their caller, but coroutines can yield to other coroutines. coroutines are more general. they can be implemented on top of generators with some fancy stuff i don't remember involving returning tokens to some kind of scheduler/dispatcher in the caller