4 ms·

Many of the ideas inside the library are also explained in the Structured Concurrency blog post on the authors homepage: http://250bpm.com/blog:71 http://250bpm

Many of the ideas inside the library are also explained in the Structured Concurrency blog post on the authors homepage: http://250bpm.com/blog:71 http://250bpm.com/blog:71 I really sympathize with a lot of them, e.g. scoping the lifetime of all child processes (goroutines, ...) to the lifetime of their parent and having good cancellation/timeout/close mechanisms for everything. I'm trying to apply these patterns in Go and it can solve a lot of scenarios in a more obvious and less spaghetti fashion. Libdill seems to bringt a view more primitives to the table which for example make cancellation and waiting for child processes to finish easier. But the basic mechanism can be also applied in Go (e.g. via context.Context and a WaitGroup/Channel for waiting on the childprocess to finish) and therefore most likely also to libmill.

the_duke 10y ago
Go does not allow external cancelling of goroutines though, which I have wrestled with a few times. The workaround is, of course, to have a "done" bool channel which is checked by the routine, but that's not always applicable or convenient.
- Matthias247 10y ago
  Yes, that's what I meant with context.Context (which encapsulates a cancellation channel) - but you can of course also use a plain channel. The difference is that Go is more explicit in this (you need a bigger amount of boilerplate code to setup cancellation and to wait for child processes to finish) and that you need to select on channels for reacting to cancellation signals instead of only relying on return values. All in all I think you can achieve the goal with both approaches. Go favors to provide more basic building blocks whereas libdill seems to be a little bit more opinionated how your concurrency should be structured.
- jerf 10y ago
  First, since people generally assume replies are disagreement, yes, goroutines can not be externally cancelled and I don't expect that to ever change. However, you can often mostly simulate what this library does in Go fairly effectively, since the goroutines end up stereotypically used. For instance, if your goroutine's purpose in life is to read from a socket, you can close the socket from another goroutine and that will result in all reads immediately failing. If you close the resources the goroutine expects, you get similar results. You do have to structure your code for this, though; if the goroutine that you're trying to close both opens a socket and listens for it, then another goroutine trying to close that socket may fail if it tries before the first one even has the socket open. One solution I've sometimes used is to lock around a struct with a bool and a socket reference; the goroutine opens the socket, takes the lock, closes if the bool is true and updates the socket reference if it isn't to point to the socket, then drops the lock. (One lock is generally not a great penalty next to the costs of acquiring the socket.) If an external goroutine tries to close the resource, it takes the lock, closes the socket reference if available, and sets the "closed" bool if it isn't. It sounds complicated in English but isn't that much code. It's a cute trick in C to turn all blocking calls into ECANCELLED results (no sarcasm, it is a cute trick), but generally goroutines should be small enough that they have an obvious and small set of "resources" that can be closed, where one such resource is definitely the dominant case. (And Go is all about considering the dominant case and not inserting tons of language features for the longer-tail cases.) A "done" channel is a good solution if your goroutine already has a "select" statement in it. I've found that correct use of the select statement actually requires more such "structural" channels than is sometimes advertised; in particular, whenever you write code where you want the receiver to close a channel for some reason, the correct solution under current Go constraints is actually to add a channel that goes back to the sender, which the sender must be modified to check, and close that. I find rather more of my channels end up being "struct{}" than I expected at first.
  - rumcajz 10y ago
    To me, the trick with closing the resource sounds like a hack. Imagine you wanted to generalise it: We have a goroutine that may use some resources: sockets, files, keyboard, screen etc. To shut it down you cut it off from all those resources and hope that it will die. It sounds wrong somehow.
    - jerf 10y ago
      I'd agree it was a hack if it were ill-defined, but I don't think it is. I don't think Go introduced that trick. In many cases you have to write your code to be able to deal with the resource being unavailable at any moment anyhow, especially networks, where Go is on its home turf. (As I've said more than once, the farther you are from writing something that looks like a network server, the less happy you will be with Go.)
rumcajz 10y ago
Yes, the library design is based on that article. However, there are couple of simplifications that were done in the meantime. Most importantly, the article goes into great pains to define a single API that deals with both soft and hard cancellation of coroutines. The library takes advantage of the fact that soft cancellation is always application-specific and provides a single API to do hard cancellation (hclose(h)). User is free to implement the soft cancellation themselves.

the_duke 10y ago

Go does not allow external cancelling of goroutines though, which I have wrestled with a few times. The workaround is, of course, to have a "done" bool channel which is checked by the routine, but that's not always applicable or convenient.

Matthias247 10y ago
Yes, that's what I meant with context.Context (which encapsulates a cancellation channel) - but you can of course also use a plain channel. The difference is that Go is more explicit in this (you need a bigger amount of boilerplate code to setup cancellation and to wait for child processes to finish) and that you need to select on channels for reacting to cancellation signals instead of only relying on return values. All in all I think you can achieve the goal with both approaches. Go favors to provide more basic building blocks whereas libdill seems to be a little bit more opinionated how your concurrency should be structured.
jerf 10y ago
First, since people generally assume replies are disagreement, yes, goroutines can not be externally cancelled and I don't expect that to ever change. However, you can often mostly simulate what this library does in Go fairly effectively, since the goroutines end up stereotypically used. For instance, if your goroutine's purpose in life is to read from a socket, you can close the socket from another goroutine and that will result in all reads immediately failing. If you close the resources the goroutine expects, you get similar results. You do have to structure your code for this, though; if the goroutine that you're trying to close both opens a socket and listens for it, then another goroutine trying to close that socket may fail if it tries before the first one even has the socket open. One solution I've sometimes used is to lock around a struct with a bool and a socket reference; the goroutine opens the socket, takes the lock, closes if the bool is true and updates the socket reference if it isn't to point to the socket, then drops the lock. (One lock is generally not a great penalty next to the costs of acquiring the socket.) If an external goroutine tries to close the resource, it takes the lock, closes the socket reference if available, and sets the "closed" bool if it isn't. It sounds complicated in English but isn't that much code. It's a cute trick in C to turn all blocking calls into ECANCELLED results (no sarcasm, it is a cute trick), but generally goroutines should be small enough that they have an obvious and small set of "resources" that can be closed, where one such resource is definitely the dominant case. (And Go is all about considering the dominant case and not inserting tons of language features for the longer-tail cases.) A "done" channel is a good solution if your goroutine already has a "select" statement in it. I've found that correct use of the select statement actually requires more such "structural" channels than is sometimes advertised; in particular, whenever you write code where you want the receiver to close a channel for some reason, the correct solution under current Go constraints is actually to add a channel that goes back to the sender, which the sender must be modified to check, and close that. I find rather more of my channels end up being "struct{}" than I expected at first.
- rumcajz 10y ago
  To me, the trick with closing the resource sounds like a hack. Imagine you wanted to generalise it: We have a goroutine that may use some resources: sockets, files, keyboard, screen etc. To shut it down you cut it off from all those resources and hope that it will die. It sounds wrong somehow.
  - jerf 10y ago
    I'd agree it was a hack if it were ill-defined, but I don't think it is. I don't think Go introduced that trick. In many cases you have to write your code to be able to deal with the resource being unavailable at any moment anyhow, especially networks, where Go is on its home turf. (As I've said more than once, the farther you are from writing something that looks like a network server, the less happy you will be with Go.)

rumcajz 10y ago

Yes, the library design is based on that article. However, there are couple of simplifications that were done in the meantime. Most importantly, the article goes into great pains to define a single API that deals with both soft and hard cancellation of coroutines. The library takes advantage of the fact that soft cancellation is always application-specific and provides a single API to do hard cancellation (hclose(h)). User is free to implement the soft cancellation themselves.