6 ms·

It looks like a lot of people are missing the point here. Yes a coroutine library would be a worse/more cumbersome way to do concurrency than the go keyword.

It looks like a lot of people are missing the point here. Yes a coroutine library would be a worse/more cumbersome way to do concurrency than the go keyword. The use case motivating all the complexity is function iterators, where `range` can be used on functions of type `func() (T, bool)`. That has been discussed in the Go community for a long time, and the semantics would be intuitive/obvious to most Go programmers. This post addresses the next thing: Assuming function iterators are added to the language, how do I write one of these iterators that I can use in a for loop? It starts by noticing that it is often very easy to write push iterators, and builds up to a push-to-pull adapter. It also includes a general purpose mechanism for coroutines, which the adapter is built on. If all of this goes in, I think it will be bad practice to use coroutines for things other than iteration, just like it's bad practice to use channels/goroutines in places where a mutex would do.

deleted 3y ago
[deleted]
djha-skin 3y ago
Still a kitchen sink move, though, isn't it? Like, no careful thinking and good 80/20 solution this time. Just "huh, we'd need coroutines to do this `right` so let's just do that" When they added generics, they really, really thought long and hard, and came up with a compromise that was brilliant and innovative in the balance it struck. I would have hoped to see something like that here, like "we're adding this one feature to goroutines to have control in specific situations" feels like something that would be better than "we're going full Rust on this problem and just adding it."
- morelisp 3y ago
  This has been under discussion for a long time. https://github.com/golang/go/issues/43557 https://github.com/golang/go/issues/43557 https://github.com/golang/go/discussions/56413 https://github.com/golang/go/discussions/56413 I'm not sure Russ's personal blog is any kind of official statement "this is what we're doing" yet?
  - chrsig 3y ago
    > I'm not sure Russ's personal blog is any kind of official statement "this is what we're doing" yet? I've found it's generally a very strong indicator. But to your point, this isn't a new issue, there's been a good amount of discussion around it over the years.
    - earthboundkid 3y ago
      In the old discussion, I specifically complained about how it would add back door coroutines to Go via iterators, and now Russ has a post about formalizing the concept of a coroutine, while also keeping the optimization of coroutines as a runtime implementation detail. I feel vindicated. :-) https://github.com/golang/go/discussions/56413#discussioncomment-4052481 https://github.com/golang/go/discussions/56413#discussioncom...
rcme 3y ago
What is wrong with: for { next := getNext() ... } What is the advantage of writing this as: for next := range getNext { ... }
- bouk 3y ago
  The first doesn't do control flow
  - rcme 3y ago
    Depends how getNext is defined.
    - alphazard 3y ago
      The only way to do control flow in getNext, consistent with how you partially defined it, is to panic. That means there is some important code wrapping the whole loop body that you left out.
    - zeroxfe 3y ago
      That doesn't make sense. `getNext()` has no ability to break out of the loop.
- alphazard 3y ago
  In practice the difference would be closer to: getNext := iterableThing.Iterator() for { next, ok := getNext() if !ok { break } ... } vs. for next := range iterableThing.Iterator() { ... } One advantage is that it's slightly shorter, which matters for very common patterns--people complain about `err != nil` after all. Another advantage is there isn't another variable for everyone to name differently. Another advantage is that everyone can't do the control flow slightly differently either.
  - tmpz22 3y ago
    > complain about `err != nil` after all Not to nitpick this specifically but as a generic reminder not all complaints are worthy of shifting the trajectory of a massively popular programming language. Balancing "worthy" and "unworthy" changes is really hard both in the community and discussions like this one. I don't envy the teams that have to do it.
eklitzke 3y ago
I think it's also worth mentioning that for certain specific use cases coroutines are much more efficient than a full goroutine, because switching to/from a coroutine doesn't require context switching or rescheduling anything. If you have two cooperating tasks that are logically synchronous anyway (e.g. an iterator) it's much more efficient to just run everything on the same CPU because the kernel doesn't have to reschedule anything or power down/wake up CPU cores, and the data is in the right CPU caches, so you'll get better cache latency and hit rates. With goroutines this may happen anyway, but it's not guaranteed and at the minimum you have the overhead of going through the Go runtime goroutine scheduler which is fast, but not as fast as just executing a different code context in the same goroutine. Coroutines offer more predictable scheduling behavior because you know that task A is switching directly to task B, whereas otherwise the goroutine scheduler could switch to another available task that wants to run. The last section of the blog post goes into this, where Russ shows that an optimized runtime implementation of coroutines is 10x faster than emulating them with goroutines. Google has internal kernel patches that implement cooperating multithreading this way (internally they're called fibers), and the patches exist for precisely this reason: better latency and more predictable scheduling behavior. Paul Turner gave a presentation about this at LPC ~10 years ago that explains more about the motivation for the patches and why they improve efficiency: https://www.youtube.com/watch?v=KXuZi9aeGTw https://www.youtube.com/watch?v=KXuZi9aeGTw
- unscaled 3y ago
  It's not just about performance, but also safety and ergonomics. Since true coroutines[1] offer predictable scheduling, their behavior with regards to data races and deadlocks is also more predictable. If programmers try to manually implement iterators, generators and interleaved state machines with their own goroutines and channels, it's not just performance that suffers - there is too much room for error. [1] I'm using the qualifier "true" here, since many modern languages (such as Python, Kotlin) use the term "coroutines" for something that is more like Go's Goroutines than Lua's coroutines. Unlike Go, they are not preemptible, but they are (at least by default) implicitly resumed when necessary by some scheduler, and they may execute on different kernel threads and switch contexts.
AYBABTME 3y ago
I think coroutines in Go will make it possible to use Go as a host for clean definition of discrete element simulations. Without it, yielding to an actor is clunky.
- deleted 3y ago
  [deleted]
dirtsoc 3y ago
Why not just range (or use an switch) over a channel and have a go routine running that pushes values into the channel? I still don’t see the need for coroutines.
- saturn_vk 3y ago
  This is too slow to really be useful on its own

deleted 3y ago

[deleted]

djha-skin 3y ago

Still a kitchen sink move, though, isn't it? Like, no careful thinking and good 80/20 solution this time. Just "huh, we'd need coroutines to do this `right` so let's just do that" When they added generics, they really, really thought long and hard, and came up with a compromise that was brilliant and innovative in the balance it struck. I would have hoped to see something like that here, like "we're adding this one feature to goroutines to have control in specific situations" feels like something that would be better than "we're going full Rust on this problem and just adding it."

morelisp 3y ago
This has been under discussion for a long time. https://github.com/golang/go/issues/43557 https://github.com/golang/go/issues/43557 https://github.com/golang/go/discussions/56413 https://github.com/golang/go/discussions/56413 I'm not sure Russ's personal blog is any kind of official statement "this is what we're doing" yet?
- chrsig 3y ago
  > I'm not sure Russ's personal blog is any kind of official statement "this is what we're doing" yet? I've found it's generally a very strong indicator. But to your point, this isn't a new issue, there's been a good amount of discussion around it over the years.
  - earthboundkid 3y ago
    In the old discussion, I specifically complained about how it would add back door coroutines to Go via iterators, and now Russ has a post about formalizing the concept of a coroutine, while also keeping the optimization of coroutines as a runtime implementation detail. I feel vindicated. :-) https://github.com/golang/go/discussions/56413#discussioncomment-4052481 https://github.com/golang/go/discussions/56413#discussioncom...

rcme 3y ago

What is wrong with: for { next := getNext() ... } What is the advantage of writing this as: for next := range getNext { ... }

bouk 3y ago
The first doesn't do control flow
- rcme 3y ago
  Depends how getNext is defined.
  - alphazard 3y ago
    The only way to do control flow in getNext, consistent with how you partially defined it, is to panic. That means there is some important code wrapping the whole loop body that you left out.
  - zeroxfe 3y ago
    That doesn't make sense. `getNext()` has no ability to break out of the loop.
alphazard 3y ago
In practice the difference would be closer to: getNext := iterableThing.Iterator() for { next, ok := getNext() if !ok { break } ... } vs. for next := range iterableThing.Iterator() { ... } One advantage is that it's slightly shorter, which matters for very common patterns--people complain about `err != nil` after all. Another advantage is there isn't another variable for everyone to name differently. Another advantage is that everyone can't do the control flow slightly differently either.
- tmpz22 3y ago
  > complain about `err != nil` after all Not to nitpick this specifically but as a generic reminder not all complaints are worthy of shifting the trajectory of a massively popular programming language. Balancing "worthy" and "unworthy" changes is really hard both in the community and discussions like this one. I don't envy the teams that have to do it.

eklitzke 3y ago

I think it's also worth mentioning that for certain specific use cases coroutines are much more efficient than a full goroutine, because switching to/from a coroutine doesn't require context switching or rescheduling anything. If you have two cooperating tasks that are logically synchronous anyway (e.g. an iterator) it's much more efficient to just run everything on the same CPU because the kernel doesn't have to reschedule anything or power down/wake up CPU cores, and the data is in the right CPU caches, so you'll get better cache latency and hit rates. With goroutines this may happen anyway, but it's not guaranteed and at the minimum you have the overhead of going through the Go runtime goroutine scheduler which is fast, but not as fast as just executing a different code context in the same goroutine. Coroutines offer more predictable scheduling behavior because you know that task A is switching directly to task B, whereas otherwise the goroutine scheduler could switch to another available task that wants to run. The last section of the blog post goes into this, where Russ shows that an optimized runtime implementation of coroutines is 10x faster than emulating them with goroutines. Google has internal kernel patches that implement cooperating multithreading this way (internally they're called fibers), and the patches exist for precisely this reason: better latency and more predictable scheduling behavior. Paul Turner gave a presentation about this at LPC ~10 years ago that explains more about the motivation for the patches and why they improve efficiency: https://www.youtube.com/watch?v=KXuZi9aeGTw https://www.youtube.com/watch?v=KXuZi9aeGTw

unscaled 3y ago
It's not just about performance, but also safety and ergonomics. Since true coroutines[1] offer predictable scheduling, their behavior with regards to data races and deadlocks is also more predictable. If programmers try to manually implement iterators, generators and interleaved state machines with their own goroutines and channels, it's not just performance that suffers - there is too much room for error. [1] I'm using the qualifier "true" here, since many modern languages (such as Python, Kotlin) use the term "coroutines" for something that is more like Go's Goroutines than Lua's coroutines. Unlike Go, they are not preemptible, but they are (at least by default) implicitly resumed when necessary by some scheduler, and they may execute on different kernel threads and switch contexts.

AYBABTME 3y ago

I think coroutines in Go will make it possible to use Go as a host for clean definition of discrete element simulations. Without it, yielding to an actor is clunky.

deleted 3y ago
[deleted]

dirtsoc 3y ago

Why not just range (or use an switch) over a channel and have a go routine running that pushes values into the channel? I still don’t see the need for coroutines.

saturn_vk 3y ago
This is too slow to really be useful on its own