4 ms·
I’ll say that using a worker pool is, IMO, a bit tricker than it sounds. The whole point of making goroutine creation cheap in the first place is that you can j
by klodolph 5y ago
I’ll say that using a worker pool is, IMO, a bit tricker than it sounds. The whole point of making goroutine creation cheap in the first place is that you can just spawn a ton of them, and then use blocking calls anywhere you want. If you have a pool of goroutines, then you open yourself to the condition that all the goroutines in the pool block, waiting for code to execute that can’t schedule because there are no available goroutines.
Any time you want to limit concurrency (e.g. by using a pool), my experience is that you have to think hard about the correct way to do that. This despite the fact that the underlying idea of a worker pool seems very innocent and easy to use. Read a few papers on the topic and you’ll see a variety of approaches. For example, if work blocks, do you give up the slot to other work? No easy answer!
- gnfargbl 5y agoI don't think using a worker pool is so bad, and I usually use one in preference to spawning lots of goroutines. I tend to follow this pattern https://play.golang.org/p/KtCPYaDiVFl https://play.golang.org/p/KtCPYaDiVFl, in which synchronization between the workers and the master relies on the language's built-in features and does not use explicit mutexes. > For example, if work blocks, do you give up the slot to other work? No easy answer! That's true in languages where threads are expensive, but goroutines are so cheap that you can probably avoid the problem simply by running a very large pool of workers.