10 ms·

Interesting article, which gives some good idea how to structure concurrent programs with less shooting at your feet :). A bit long winded until it comes to the

by _ph_ 4y ago

Interesting article, which gives some good idea how to structure concurrent programs with less shooting at your feet :). A bit long winded until it comes to the actual core concept being presented: spawn concurrent routines within a block which doesn't exit until the last routine has exited. This is a good concept an can make a lot of code clearer. It prevents some data races, as you can reason that after the block no concurrent operations are still ongoing, but of course it could make the block lock up by itself, if one of the routines doesn't finish. This is certainly an interesting concept I might try out myself more specifically. It is also important to know, especially as the "go" statement of the language Go is put into the headline, that this very language already supports nursery-like structures, just doens't have the syntax sugar the python extension of the author has. It is called a wait group. See for an example here: https://gobyexample.com/waitgroups https://gobyexample.com/waitgroups So, except for the syntactic sugar around it, the nurseries compare to creating a wait group in a block, spawn goroutines which each call Done() on the wait group on exit and at the end of the block call Wait() to errect the same boundary, nurseries do. Of course you can also pass the waitgroup object to any goroutine created inside goroutines. This is a very common pattern in Go, but indeed, it probably should be presented more clearly and up front in tutorials about goroutines. So for that, I will keep the article around, it shows the concept nicely - perhaps I might do a pure "go" version of it, which then shows the go implementation of nurseries. Might be nice to add to the original article, that not only the presented python library is a solution, but also that there is a native go way of achieving that, as the article uses "go" as the negative example :p

conradludgate 4y ago
I think you missed the point of the article. Its suggesting that goroutine style concurrency should be completely replaced. Much like if/for etc has replaced goto, structured concurrency can replace goroutines. Note: goto can be used like an if or a for loop. You've just made the argument that goroutines can be used like 'nurseries'. You've essentially argued in favour of goto from the articles perspective I want to note again that this article is not actually about go. It is the same in most languages with concurrency primitives. The key takeaway is that having to manually manage waitgroups is room to make mistakes or to introduce spaghetti concurrency, while you might be used to it at this point, it doesn't make it the best system for the future of concurrency
- _ph_ 4y ago
  No, I didn't miss the point of the article. I just feel, it is important to note that the proposed style of concurency is available in go and point out to how you would implement it, if you cannot use this Python library. I would agree, it would be nice to have a similar syntactic construct in go to enforce this pattern, though it is actually possible to basically implement that with higher order functions. And yes, I think goto has a place in any language, if you are aware of how you should and how you shouldn't use it. In most cases the typical constructs of structured programming are preferrable, but not in all.
  - kubanczyk 4y ago
    > No, I didn't miss the point of the article. I just feel, it is important to note that the proposed style of concurency is available in go Apart from the TFA covering a particular python lib, I'd like to point out how the Go philosophy worked before. Given: Situation 1: "the proposed style of concurency is available in go" so let's have a free ride and we can have linters and human reviewers trying to catch all the concurrency bugs faster than contributors are inserting them. /s Situation 2: the proposed style of concurrency is the only one available in the language. Approach 2 would make sense.
    - _ph_ 4y ago
      You are missing a very importand third point: Situation 3: we have understood the value of the proposed style of concurrency and think it is the right one for most situations, consequentely an according API has been created and it is strongly recommended for all usages of concurrency that fall into the pattern. This doesn't require the removal of the primitives used to build this API from the language. No linters needed, just a bit of common sense not to use the low-level API unnneeded.
      - cb321 4y ago
        In support of your point, this largely tracks way the "goto statement" shakeout has largely gone down in a number of programming languages over the decades - "available but discouraged" as in "Please do not write simple loops with if-and-goto." Access to source inline|linked assembly is also often available. That is perhaps less of a hazard since people probably have a less easy time "convincing themselves they understand it."
freiherr 4y ago
This was also my immediate thought, wait groups and a channel for any error would work just like the nursery. However I still see the advantage for this new approach as a provider for sane defaults. Right now you need to structure the concurrent routines yourself and are free to mess up. Nurseries should make it the other way around. I'm not sold on that but may give it a chance.
wbl 4y ago
Waitgroups are not the same thing as they cannot be dynamically sized in the same way nurseries can.
- swagonomixxx 4y ago
  Not sure what you mean by this. You can dynamically add to a wait group using the Add() method.
  - wbl 4y ago
    So I slightly misremembered the subtlety and it is more forgiving in that Add can be called while waiting if a thread is still executing in it.
drwiggly 4y ago
Its not the same. Anything in the nursery can cause cancellation of all async work. Cancellation is safe to unwind in all workers by scope exit rules. Completion is just a type of cancellation. Spawn 10 workers to lookup Dog in 10 different dictionaries, the first one to get the answer wins. This is hard to do with out language/runtime cancellation support. Note below is a practical lib to get close to this https://blog.labix.org/2011/10/09/death-of-goroutines-under-control https://blog.labix.org/2011/10/09/death-of-goroutines-under-... .. https://github.com/go-tomb/tomb/tree/v2 https://github.com/go-tomb/tomb/tree/v2
- swagonomixxx 4y ago
  You can easily cancel worker goroutines in Go using contexts and their associated cancel functions (listen on ctx.Done() in a select statement).
- duped 4y ago
  > This is hard to do with out language/runtime cancellation support. It is trivial if the workers can communicate using a shared flag that represents "Dog is found."
d0mine 4y ago
At best, waitgroups are to the structured concurrency as gotos are to loops/ifs(structured programming)--you can implement the latter in terms of the former but the structured part is better (even if/because it is less powerful) as the history shows.

by _ph_ 4y ago

conradludgate 4y ago

I think you missed the point of the article. Its suggesting that goroutine style concurrency should be completely replaced. Much like if/for etc has replaced goto, structured concurrency can replace goroutines. Note: goto can be used like an if or a for loop. You've just made the argument that goroutines can be used like 'nurseries'. You've essentially argued in favour of goto from the articles perspective I want to note again that this article is not actually about go. It is the same in most languages with concurrency primitives. The key takeaway is that having to manually manage waitgroups is room to make mistakes or to introduce spaghetti concurrency, while you might be used to it at this point, it doesn't make it the best system for the future of concurrency

_ph_ 4y ago
No, I didn't miss the point of the article. I just feel, it is important to note that the proposed style of concurency is available in go and point out to how you would implement it, if you cannot use this Python library. I would agree, it would be nice to have a similar syntactic construct in go to enforce this pattern, though it is actually possible to basically implement that with higher order functions. And yes, I think goto has a place in any language, if you are aware of how you should and how you shouldn't use it. In most cases the typical constructs of structured programming are preferrable, but not in all.
- kubanczyk 4y ago
  > No, I didn't miss the point of the article. I just feel, it is important to note that the proposed style of concurency is available in go Apart from the TFA covering a particular python lib, I'd like to point out how the Go philosophy worked before. Given: Situation 1: "the proposed style of concurency is available in go" so let's have a free ride and we can have linters and human reviewers trying to catch all the concurrency bugs faster than contributors are inserting them. /s Situation 2: the proposed style of concurrency is the only one available in the language. Approach 2 would make sense.
  - _ph_ 4y ago
    You are missing a very importand third point: Situation 3: we have understood the value of the proposed style of concurrency and think it is the right one for most situations, consequentely an according API has been created and it is strongly recommended for all usages of concurrency that fall into the pattern. This doesn't require the removal of the primitives used to build this API from the language. No linters needed, just a bit of common sense not to use the low-level API unnneeded.
    - cb321 4y ago
      In support of your point, this largely tracks way the "goto statement" shakeout has largely gone down in a number of programming languages over the decades - "available but discouraged" as in "Please do not write simple loops with if-and-goto." Access to source inline|linked assembly is also often available. That is perhaps less of a hazard since people probably have a less easy time "convincing themselves they understand it."

freiherr 4y ago

This was also my immediate thought, wait groups and a channel for any error would work just like the nursery. However I still see the advantage for this new approach as a provider for sane defaults. Right now you need to structure the concurrent routines yourself and are free to mess up. Nurseries should make it the other way around. I'm not sold on that but may give it a chance.

wbl 4y ago

Waitgroups are not the same thing as they cannot be dynamically sized in the same way nurseries can.

swagonomixxx 4y ago
Not sure what you mean by this. You can dynamically add to a wait group using the Add() method.
- wbl 4y ago
  So I slightly misremembered the subtlety and it is more forgiving in that Add can be called while waiting if a thread is still executing in it.

drwiggly 4y ago

Its not the same. Anything in the nursery can cause cancellation of all async work. Cancellation is safe to unwind in all workers by scope exit rules. Completion is just a type of cancellation. Spawn 10 workers to lookup Dog in 10 different dictionaries, the first one to get the answer wins. This is hard to do with out language/runtime cancellation support. Note below is a practical lib to get close to this https://blog.labix.org/2011/10/09/death-of-goroutines-under-control https://blog.labix.org/2011/10/09/death-of-goroutines-under-... .. https://github.com/go-tomb/tomb/tree/v2 https://github.com/go-tomb/tomb/tree/v2

swagonomixxx 4y ago
You can easily cancel worker goroutines in Go using contexts and their associated cancel functions (listen on ctx.Done() in a select statement).
duped 4y ago
> This is hard to do with out language/runtime cancellation support. It is trivial if the workers can communicate using a shared flag that represents "Dog is found."

d0mine 4y ago

At best, waitgroups are to the structured concurrency as gotos are to loops/ifs(structured programming)--you can implement the latter in terms of the former but the structured part is better (even if/because it is less powerful) as the history shows.