4 ms·

Am I the only one who finds Go concurrency model to be extremely bug prone and difficult to work with? Writing an efficient pipeline of goroutines connected wit

by kubb 5y ago

Am I the only one who finds Go concurrency model to be extremely bug prone and difficult to work with? Writing an efficient pipeline of goroutines connected with channels that doesn't deadlock is a verbose and subtle mess with a whole bunch of code duplication. For a lot of the things that I want to achieve, functional abstractions like parallel map and reduce would be much more understandable, and easier to use. That whole talk about philosophy distracts from the actual tools that Go gives you to manage concurrency which in my opinion can be improved a lot.

Thaxll 5y ago
"Writing an efficient pipeline of goroutines connected with channels that doesn't deadlock is a verbose and subtle mess with a whole bunch of code duplication." It takes 30 lines of code and it's very easy.
- 1ris 5y ago
  This has to be depended on the pipeline you want to have.
  - Thaxll 5y ago
    It depends but saying it's very complicated and it's easy to deadlock is pure bs. Where is the complexity in Go to have a pool of workers looking for jobs on a channel exactly?
    - ackfoobar 5y ago
      > Where is the complexity in Go to have a pool of workers looking for jobs on a channel exactly? This is just one kind of pipeline. If this is all you have done, you may have the impression that concurrency (in Go) is easy. BTW, this is worth reading: Understanding Real-World Concurrency Bugs in Go https://songlh.github.io/paper/go-study.pdf https://songlh.github.io/paper/go-study.pdf
      - Thaxll 5y ago
        I know that paper and it's indeed a good one but I still don't agree with OP that "Go concurrency model to be extremely bug prone".
        BobbyJo 5y ago
        I agree. Pointing out the difficulty of concurrency as a general premise doesn't invalidate Go's model. For the use cases it was designed for (asynchronous services) it's model is probably the best. For use cases it wasn't designed for, it probably sucks.
    - kubb 5y ago
      > pool of workers looking for jobs on a channel It really feels like the design was driven by this one example only.
- IshKebab 5y ago
  Yeah until you need to cleanly shut everything down and avoid deadlocks or unbounded memory use. It could definitely do with some improvements especially around closing channels and telling goroutines to stop.
bjt 5y ago
I think different people read "Go concurrency model" to mean different things. If we're talking about green threads (goroutines), I'm a huge fan. Having had a lot of success using Gevent in the Python world, it was great coming over to the Go world where that pattern was baked into the language. If we're talking about channels, I'm really not a fan. Not nearly as easy to use as the equivalent Queue library over in Python, despite being baked into the language. My experience aligns with this post: https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-should-feel-bad/ https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-s... Almost every time someone's suggested a Go channel, I've found it simpler to use a mutex or waitgroup.
- kubb 5y ago
  I'm referring to the "share memory by communicating" idea mentioned in the article. I don't think channels are bad in general, but the way they work in Go leaves a lot to be desired.
bjackman 5y ago
I think that the "share by communicating" thing has led some people to think that these goroutine labyrinths are somehow The Way with Go - that's certainly how I felt when I started out with it. I absolutely agree that once things get really pipeliney it can be incredibly hard to debug and there are almost always lockups, and it can start to feel like select-oriented programming. I've found that the more cautious I've become with channels and goroutines the better luck I've had with Go. When reviewing code these days go into high alert if I see a channel crossing a function boundary or a naked (i.e. without an errgroup or WaitGroup) go statement.

by kubb 5y ago

Thaxll 5y ago

"Writing an efficient pipeline of goroutines connected with channels that doesn't deadlock is a verbose and subtle mess with a whole bunch of code duplication." It takes 30 lines of code and it's very easy.

1ris 5y ago
This has to be depended on the pipeline you want to have.
- Thaxll 5y ago
  It depends but saying it's very complicated and it's easy to deadlock is pure bs. Where is the complexity in Go to have a pool of workers looking for jobs on a channel exactly?
  - ackfoobar 5y ago
    > Where is the complexity in Go to have a pool of workers looking for jobs on a channel exactly? This is just one kind of pipeline. If this is all you have done, you may have the impression that concurrency (in Go) is easy. BTW, this is worth reading: Understanding Real-World Concurrency Bugs in Go https://songlh.github.io/paper/go-study.pdf https://songlh.github.io/paper/go-study.pdf
    - Thaxll 5y ago
      I know that paper and it's indeed a good one but I still don't agree with OP that "Go concurrency model to be extremely bug prone".
      - BobbyJo 5y ago
        I agree. Pointing out the difficulty of concurrency as a general premise doesn't invalidate Go's model. For the use cases it was designed for (asynchronous services) it's model is probably the best. For use cases it wasn't designed for, it probably sucks.
  - kubb 5y ago
    > pool of workers looking for jobs on a channel It really feels like the design was driven by this one example only.
IshKebab 5y ago
Yeah until you need to cleanly shut everything down and avoid deadlocks or unbounded memory use. It could definitely do with some improvements especially around closing channels and telling goroutines to stop.

bjt 5y ago

I think different people read "Go concurrency model" to mean different things. If we're talking about green threads (goroutines), I'm a huge fan. Having had a lot of success using Gevent in the Python world, it was great coming over to the Go world where that pattern was baked into the language. If we're talking about channels, I'm really not a fan. Not nearly as easy to use as the equivalent Queue library over in Python, despite being baked into the language. My experience aligns with this post: https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-should-feel-bad/ https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-s... Almost every time someone's suggested a Go channel, I've found it simpler to use a mutex or waitgroup.

kubb 5y ago
I'm referring to the "share memory by communicating" idea mentioned in the article. I don't think channels are bad in general, but the way they work in Go leaves a lot to be desired.

bjackman 5y ago

I think that the "share by communicating" thing has led some people to think that these goroutine labyrinths are somehow The Way with Go - that's certainly how I felt when I started out with it. I absolutely agree that once things get really pipeliney it can be incredibly hard to debug and there are almost always lockups, and it can start to feel like select-oriented programming. I've found that the more cautious I've become with channels and goroutines the better luck I've had with Go. When reviewing code these days go into high alert if I see a channel crossing a function boundary or a naked (i.e. without an errgroup or WaitGroup) go statement.