7 ms·
What. This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to te
by arghwhat 9y ago
What.
This article proposes a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc.
It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel).
The construct is a decently usable, already applied tool to handle a set of problems, but the article takes the issue way out of proportions and overhypes the proprosed solution. The "with" example for benefits to not having a "go" statement seem rather bogus, especially seeing that such RAII constructs do not exist in Go (no destructors, remember?).
Trying to claim that "go" is as terrible as the original "goto" is ignorance of the original problems. Bad use of goto can be a nightmare to track (as the author tried to illustrate), but goroutines do not jump around, they branch from the main goroutine, following normal control flow from there. They are easy to follow, and the language is designed so that you can throw around with them and forget them without them causing you problems.
Also, this article is comparing a list of concurrency constructs and one parallism construct (pthread_create—threading.Thread doesn't count as parallism due to GIL) to callbacks, something which have nothing to do with concurrency at all. Very odd.
- scott_s 9y agoI thought the article's comparison of go routines to goto was fair, in the context made: when calling a function in a language that allows go-routine like things (including pthread_create), one can't know if background tasks will be spawned. This is similar in the free-form time of unrestricted goto that one could not know if control flow really would return to the point lexically after the function call. His proposed abstraction does allow one to look after a nursery block and know whether or not background tasks are still running.
- arghwhat 9y agoKnowing whether a background task has spawned is very different from not being able to follow the control flow (goto potentially jumping to an entirely different function body). Now, while Go is designed mostly for you to not care about goroutines, there are some corner cases where one must know if a resource is used by anything, such as the chase of when you wish to close a file handle. However, I'd argue that this is not related to go's concurrency model and the presence of background tasks at all. It's related to object lifetimes. This should be made clear at API surfaces. A language solution to this would be Rust's lifetimes, not a new concurrency model.
- marcus_holmes 9y agoBut this concept does also solve that problem. Maybe more simply. Rust is renowned for being a total headache with borrowing and lifetimes. Nurseries might be a simpler solution for this. As you say, a corner case. But a common one...
- lilyball 9y agoNurseries and lifetimes are orthogonal concepts. In fact, Rust already has a "nursery", except it's generally called a scoped threadpool. They use lifetimes to ensure the threads can reference stuff on the stack without copying it. So a "nursery" is basically just a scoped threadpool with a 'static lifetime.
- steveklabnik 9y agoThere is some interesting stuff about this in the /r/programming thread https://www.reddit.com/r/programming/comments/8es8x3/notes_on_structured_concurrency_or_go_statement/dxxucgi/ https://www.reddit.com/r/programming/comments/8es8x3/notes_o...
- stouset 9y agoI don’t think there’s any reason a nursery has to have a static lifetime.
- eridius 9y agoThe description of the nursery is you can pass it around wherever you like. More generally, this concept seems to have been designed in a language without lifetimes. You obviously can have a scoped threadpool without a 'static lifetime, but if you want to pass it around to arbitrary locations then you do.
- twic 9y agoI think the whole point of the nursery's design is that you can create and destroy them during the life of your program, and that destroying them forms a barrier where the owner of the nursery waits for its children to finish. Having a nursery with a static lifetime in Rust would therefore be pointless, as it would live until the end of the program.
- wilun 9y agoOne can also not "known" of all the other side effect that will happen. Unless they are documented informally or encoded formally.
- speleding 9y agoYou realise that makes you sound exactly like an old Fortran programmer refusing to give up goto? A for loop is just a wrapped goto! ;-) I think the author proposes a useful way to structure multi-threading. The comparison to goto isn't perfect and he needs to play fast and loose with some terms to keep the analogy working but he makes a good point. I'm not convinced yet that the nursery pattern should be the only allowed way to start a thread but saying "you should use it unless you have a good reason not to" is a good provocation to get the discussion going.
- platinumrad 9y ago"I think x is like goto." "I disagree; I don't think x is like goto." "Now that we've agreed that x is like goto, not wanting to give up x makes you like a big dinosauric dummy~"
- kybernetikos 9y ago> 'A for loop is just a wrapped goto!' corresponds fairly closely with > a "nursery", which is just a wrapped sync.WaitGroup/pthread_join/futures::future::join_all/a reactor that waits for all tasks to terminate/etc. The fact that the 'nursery' can and has been implemented with more 'primitive' constructs is actually a point of similarity with goto/structured programming not a point of difference. The GP didn't seem to spot this, despite making it their first complaint, so it was good that the post you're responding to did. The post you're responding to is not calling the GP a dummy because of a disagreement, it's pointing out the apparent criticism the GP levelled was exactly the same criticism levelled against structured programming, and ultimately goto lost.
- arghwhat 9y agoAs someone whose work includes hardware driver programming, I'd rather not give up goto. It would make a lot of error handling extremely cumbersome, and much less readable. :( However, I find "go" and "goto" to not intersect at all. I've written this many times in other comments on this thread, so I'd rather not type it out again, but the TL;DR: is that "goto" can make understanding a function when read difficult, while "go" is clear when read. No function is understood if never read, and spawning "background tasks" is a core part of asynchronous programming (and thus not an unusual side-effect). The "nursery pattern" is a decent construct that I have used quite often whenever I felt a need, but it doesn't appear to really solve any issues mentioned in the post. I also elaborated on this quite a few times already, so TL;DR: the only real problem of goroutines is things like references to potentially closed objects, but any method may end up storing an internal reference used at a later call, making the issue not related to concurrency.
- zamalek 9y agoAgree with everything. I think the misunderstanding here is that goroutines are not classical threads (which would share many problems with `goto`) - they exist somewhere between coroutines and actors (because of channels+select). Actors are an established and mature solution to many headaches surrounding concurrency and parallelism. Goroutines share more in common with method calls (which are a form of branching) and even with single-threaded scenarios, methods can do surprising things - especially if you have shared global state. Additionally, if you control how data is shared (such as message passing - channels) you shouldn't have to be concerned about what the other thread is doing - so long as it reacts to messages that you send to it.
- tomtheelder 9y agoI don't think that's a misunderstanding at all. The argument is all about control flow and programmer understanding. The actual underlying mechanisms aren't what he is arguing against, instead he dislikes the potential for programmer confusion when code is being executed that wasn't expected. That is 100% possible with goroutines. Method calls aren't really branching. The code path is still totally linear. You could basically copy paste the code in the method definition in place of its call and get the same outcome (not literally but you know what I mean). For goroutines this is not the case. I think your points about state ownership are right on, but that's kind of the author's whole point. Right now, there are a lot of things you have to be very conscious of to write good code that executes cleanly using goroutines/threads/etc. That is very much the same as writing good code with gotos. It's 100%, unequivocally possible to write good code using gotos (every control flow structure dijkstra proposed can be represented with them), it just requires a lot of added thought, and the potential for mistakes is much higher. The author is not proposing any functionality that doesn't already exist, just a new control pattern to reduce the chance of creating problems.
- arghwhat 9y ago> That is 100% possible with goroutines. It really isn't. The big difference is that "goto" can lead to code you read being grossly misunderstood due to complicated flows, potentially even absurd things like jumping to a different function. (And as you mention, good code can be written with goto's—it can make particularly error handlers much easier to read in C.) This is not the case at all with "go", which is extremely clear as to what it does and how the flow will go. However, as a caller, you do not know the flow of the function you call unless you read it. As with any type of asynchronous programming, a function might have scheduled something for later execution: It might be a future, a promise, a timeout/interval, network receive callbacks, or a goroutine. That is, strictly speaking, none of your concern as long as the function lives up to its contract. If not, all bets are off regardless. > ... it just requires a lot of added thought, and the potential for mistakes is much higher. I get your point, but I must disagree that asynchronous programming (which is what goroutines is merely an implementation of) increase the complexity of writing good code, nor increase the likelyhood of state ownership mismanagement in any measurable fashion. > The author is not proposing any functionality that doesn't already exist, just a new control pattern to reduce the chance of creating problems. I don't really find that the author suggests anything new at all. Rather, the author takes an existing blob of code and claims a new benefit from it. EDIT: erroneous "in which case" replaced with "if not", as initially intended.
- tomtheelder 9y agoIt really is a whole wrapped up group of existing constructs, but that doesn't mean there isn't merit in it. The author mentions how this was central to Dijkstra's proposal for structured programming: "And now that Dijkstra understood the problem, he was able to solve it. Here's his revolutionary proposal: we should stop thinking of if/loops/function calls as shorthands for goto, but rather as fundamental primitives in their own rights – and we should remove goto entirely from our languages." > The "with" example for benefits to not having a "go" statement seem rather bogus, especially seeing that such RAII constructs do not exist in Go (no destructors, remember?). I don't really understand what you mean by this. You would certainly need a different mechanism in Go than in Python, but I fail to see how that is a criticism of the argument he is making. He has built one implementation in one language, clearly it would look different in other languages. > Trying to claim that "go" is as terrible as the original "goto" is ignorance of the original problems. Bad use of goto can be a nightmare to track (as the author tried to illustrate), but goroutines do not jump around, they branch from the main goroutine, following normal control flow from there. They are easy to follow, and the language is designed so that you can throw around with them and forget them without them causing you problems. Why exactly is branching better? With "go" statements, you can easily end up with goroutines that you aren't even aware of floating around doing things that you aren't aware of. His example of calls in external libraries is probably the best illustration. Currently, you could call what you think is a simple function from a library, and end up with a whole host of goroutines you didn't expect floating around, doing things and using resources. Articles about writing good libraries [0] have to make points about how you need to take care of this stuff. Gotos create problems when you end up surprised about what code is being executed, making tracing execution paths difficult. Branching creates a different set of problems: most crucially that you can have code being executed and not even realize it's happening. The issues are different, but his analogy is really spot on IMO. Both of these things can be solved with code reading and debugging, but the author's whole point is that with an improved abstraction you no longer have to worry about that sort of thing. > Also, this article is comparing a list of concurrency constructs and one parallism construct (pthread_create—threading.Thread doesn't count as parallism due to GIL) to callbacks, something which have nothing to do with concurrency at all. Very odd. These are all related concepts. They are about doing different things "at the same time," just with different definitions of what the phrase means. The problems the author are describing are considerably worse in a scenario where actual processes are being spawned, but the nursery pattern is relevant in all of them. It's about control flow, not capabilities. Even using threading with a GIL can create execution paths that you aren't really aware of until you actually look through the code or debug.
- wybiral 9y agoMost articles riding on the "x considered harmful" meme are clickbait without much content.
- stouset 9y agoNothing could describe this post (or anything on the author’s site in general) less accurately than your comment.
- wybiral 9y agoHow so?
- scott_s 9y agoBecause the author engages directly with the core point Dijkstra made, and proposes a new abstraction which parallels with that core point. The author does so in a clear and thoughtful way which cannot be dismissed out-of-hand.
- arghwhat 9y agoAnd yet misses the point entirely, presenting non-original work that only acts as a band-aid on a symptom, rather than dealing with the cause (which is not related to concurrency constructs at all).
- justinpombrio 9y ago> The "with" example for benefits to not having a "go" statement seem rather bogus, especially seeing that such RAII constructs do not exist in Go (no destructors, remember?). You've greatly underestimated how general this problem is. First, Python's `with` statement has nothing to do with destructors. From the PEPM for `with`: with VAR = EXPR: BLOCK which roughly translates into this: VAR = EXPR VAR.__enter__() try: BLOCK finally: VAR.__exit__() `exit` is a method, not a destructor. As a result, Go could easily gain a `with`-like construct. Moreso, the issue applies even if there's nothing like `with` in the language. If I'm reading a function definition, and that definition uses the "acquire, try, finally, release" pattern that is the desugaring of `with`, then it sure would be nice to know that nothing from BLOCK is running after the `finally` statement has run. `go` breaks that assumption.
- arghwhat 9y agoI mentioned RAII because the text around "with" mentions its use with RAII. Go does not have exceptions (panic is not meant as "normal" flow control), and therefore has no use for a "with" construct. "defer" is used for a somewhat similar purpose. Go does not have destructors, and therefore has no possible implementation of RAII. However, none of this applies to goroutines. A goroutine only gives errors if the author decides that such is necessary. If so, it will likely be through an error channel. There is no unexpected code paths through such readout, rendering "with" and RAII useless. So again, for a post that was very focused on complaining directly about the "go" keyword, I was expecting something applicable to Go.
- ellyagg 9y agoCan I just mention that anyone who begins their comment with What. ...and: [whatever] is just a [whatever] ...comes across as a bit of an asshole? Why not just make your point instead of investing a lot in making sure everyone knows that you're smart and the other person is dumb? Maybe Knuth was just as rude to Dijkstra, who knows?
- arghwhat 9y agoOf course you can. I felt that such "aggressive" countering was justified considering the equivalently grand claims of the post ("EXTREMELY_POPULAR_CONSTRUCT considered harmful").
- fixermark 9y ago> It then uses an exception-like model for error propagation to "solve" error handling (which is fairly easy to handle with a channel). If none of your code ever actually waits on the error channel when you spawn a goroutine, what happens? Is there any circumstance where that behavior is preferable to guaranteeing something must wait on the error channel?
- arghwhat 9y agoGo's error paradigm is through return values that can be ignored, so an error channel would mimic "non-async" error handling in this case: Checking an error is is always up to the developer. The discussion about error paradigms (exceptions vs. globals vs. error values, forced checking vs. free choice, etc.) is quite a big one, and arguably an entirely different topic.
- cryptonector 9y agoCallbacks have plenty to do with concurrency: first you compile everything into CPS (where callbacks are continuations) and then a trivial event loop nets you cooperative multitasking. When you don't have language support for this but you need concurrency, you just write something resembling CPS by hand and call it "callback hell". > (pthread_create—threading.Thread doesn't count as parallism due to GIL) Eh? Why would you have a GIL? Maybe in Go, but so what, just don't use such a language/run-time.