3 ms·
I find that combining context and errgroup, with due care, lets us approximate Structured Concurrency to great benefit. I just think more care is due than most
by fl0ki 3y ago
I find that combining context and errgroup, with due care, lets us approximate Structured Concurrency to great benefit. I just think more care is due than most people give.
When a function returns, we should be able to trust that any goroutines it created have already terminated and will not have other side effects. This is important because Go doesn't enforce read/write thread-safety any other way, so we need it to be clear from the code when those reads/writes may happen.
It's also important because side effects from those lingering routines could have other consequences, e.g. the retry for an IO operation could overlap with a past attempt, violating invariants in ways that would be really hard to reproduce and debug.
This sounds really simple, why wouldn't you do it that way? Idiomatic use of errgroup encourages you to do it that way, but not everyone does it that way, and sadly not every project even uses errgroup in the first place. It's very common to see a routine observe cancellation and return immediately, even if it created its own goroutines which it can't guarantee have aborted yet.
Aside, it's also sadly extremely common to see people reinvent errgroup badly with "error channels" that at best don't join the other routines and at worst deadlock them because they block on sending errors that nobody is receiving any more.
That's why if you do this, you basically ban the `go` keyword and strictly use errgroup, even for routines which can't return errors. (WaitGroup can do this too, but it's harder to use right, because the Add/Done count have to add up exactly and there's no enforcement that they do).
If you do this right, then it shouldn't matter whether a function creates goroutines to help with its work, such as timeout channels or parallel processing or what have you. What matters is that the function still acts like a synchronous one from the outside.
The worst I've seen is when people know they have to use errgroup, but they create one large errgroup in main and pass it around as a mutable argument to everything to add more tasks to it. They don't understand that when it's used correctly, it also nests and encapsulates entirely, so it's never the argument to or return from a function.
Of course it gets more complicated if an object has long-running goroutines that outlive any particular function. Then you need to call more functions just to create those wait points. For example, it's not enough to cancel a database cursor as a context, you should still block on closing it, otherwise its own routines can still be running when you go back and start another operation. Again, sadly all too common.
Caveat 1: errgroup only returns the first error, which for many routines is just "cancelled". That's not useful, and it takes the place of what could have been a real error. I suppress errors like that, so that actual cleanup errors, if any, are the ones surfaced.
Caveat 2: errgroup doesn't trap panics, if you want the panic to be surfaced as a neatly packaged error, you have to install your own handler. Every project I have has its own simple version of this, and I've seen many other projects come to the same conclusion.