18 ms·

Problems with Go channels (2016)

160 points by mpweiher 2y ago

t8sr 2y ago
When I did my 20% on Go at Google, about 10 years ago, we already had a semi-formal rule that channels must not appear in exported function signatures. It turns out that using CSP in any large, complex codebase is asking for trouble, and that this is true even about projects where members of the core Go team did the CSP. If you take enough steps back and really think about it, the only synchronization primitive that exists is a futex (and maybe atomics). Everything else is an abstraction of some kind. If you're really determined, you can build anything out of anything. That doesn't mean it's always a good idea. Looking back, I'd say channels are far superior to condition variables as a synchronized cross-thread communication mechanism - when I use them these days, it's mostly for that. Locks (mutexes) are really performant and easy to understand and generally better for mutual exclusion. (It's in the name!)
- throwaway150 2y ago
  What is "20% on Go"? What is it 20% of?
  - ramon156 2y ago
    I assume this means "20% of my work on go" aka 1 out of 5 work days working on golang
  - NiloCK 2y ago
    Google historically allowed employees to self-direct 20% of their working time (onto any google project I think).
  - darkr 2y ago
    At least historically, google engineers had 20% of their time to spend on projects not related to their core role
    - kyrra 2y ago
      This still exists today. For example, I am on the payments team but I have a 20% project working on protobuf. I had to get formal approval from my management chain and someone on the protobuf team. And it is tracked as part of my performance reviews. They just want to make sure I'm not building something useless that nobody wants and that I'm not just wasting the company's time.
      - vrosas 2y ago
        I see why they do this, but man it almost feels like asking your boss for approval on where you go on vacation. Do people get dinged if their 20% time project doesn't pan out, or they lose interest later on?
        NBJack 2y ago
        Previously it could be anything you wanted. These days, you need formal approval. Google has changed a bit.
        kyrra 2y ago
        It has nothing to do with success. It's entirely for making sure some one besides the person doing the 20% agrees with the idea behind the project.
        hedora 2y ago
        Lol. They’d be better off giving people the option to work 4 days if they also signed over right of first refusal for hobby projects.
      - rollcat 2y ago
        I never worked at Google (or any other large corp for that matter), but this sounds like the exact opposite of an environment that spawned GMail. As you think back even to the very early days of computing, you'll find individuals or small teams like Grace Hopper, the Unix gang, PARC, etc that managed to change history by "building something useless". Granted, throughout history that happened less than 1% of the time, but it will never happen if you never try. Maybe Google no longer has any space for innovation.
- dfawcus 2y ago
  How large do you deem to be large in this context? I had success in using a CSP style, with channels in many function signatures in a ~25k line codebase. It had ~15 major types of process, probably about 30 fixed instances overall in a fixed graph, plus a dynamic sub-graph of around 5 processes per 'requested action'. So those sub-graph elements were the only parts which had to deal with tear-down, and clean up. There were then additionally some minor types of 'process' (i.e. goroutines) within many of those major types, but they were easier to reason about as they only communicated with that major element. Multiple requested actions could be present, so there could be multiple sets of those 5 process groups connected, but they had a maximum lifetime of a few minutes. I only ended up using explicit mutexes in two of the major types of process. Where they happened to make most sense, and hence reduced system complexity. There were about 45 instances of the 'go' keyword. (Updated numbers, as I'd initially misremembered/miscounted the number of major processes)
  - hedora 2y ago
    How many developers did that scale to? Code bases that I’ve seen that are written in that style are completely illegible. Once the structure of the 30 node graph falls out of the last developer’s head, it’s basically game over. To debug stuff by reading the code, each message ends up having 30 potential destinations. If a request involves N sequential calls, the control flow can be as bad as 30^N paths. Reading the bodies of the methods that are invoked generally doesn’t tell you which of those paths are wired up. In some real world code I have seen, a complicated thing wires up the control flow, so recovering the graph from the source code is equivalent to the halting problem. None of these problems apply to async/await because the compiler can statically figure out what’s being invoked, and IDE’s are generally as good at figuring that out as the compiler.
    - dfawcus 2y ago
      That was two main developers, one doing most of the code and design, the other a largely closed subset of 3 or 4 nodes. Plus three other developers co-opted for implementing some of the nodes. [1] The problem space itself could have probably grown to twice the number of lines of code, but there wouldn't have needed to be any more developers. Possibly only the original two. The others were only added for meeting deadlines. As to the graph, it was fixed, but not a full mesh. A set of pipelines, with no power of N issue, as the collection of places things could talk to was fixed. A simple diagram represented the major message flow between those 30 nodes. Testing of each node was able to be performed in isolation, so UT of each node covered most of the behaviour. The bugs were three deadlocks, one between two major nodes, one with one major node. The logging around the trigger for the deadlock allowed the cause to be determined and fixed. The bugs arose due to time constraints having prevented an analysis of the message flows to detect the loops/locks. So for most messages, there were a limited number of destinations, mostly two, for some 5. For a given "request", the flow of messages to the end of the fixed graph would be passing through 3 major nodes. That then spawned the creation of the dynamic graph, with it having two major flows. One a control flow through another 3, the other a data flow through a different 3. Within that dynamic graph there was a richer flow of messages, but the external flow from it simply had the two major paths. Yes, reading the bodies of the methods does not inform as to the flows. One either had to read the "main" routine which built the graph, or better refer to the graph diagram and message flows in the design document. Essentially a similar problem to dealing with "microservices", or plugable call-backs, where the structure can not easily be determined from the code alone. This is where design documentation is necessary. However I found it easier to comprehend and work with / debug due to each node being a prodable "black box", plus having the graph of connections and message flows. [1] Of those, only the first had any exerience with CSP or Go. The CSP expereince being with a library for C, the Go experience some minimal use a year earlier. The other developers were all new to CSP and Go. The first two developers were "senior" / "experienced".
- ChrisSD 2y ago
  I think the two basic synchronisation primitives are atomics and thread parking. Atomics allow you to share data between two or more concurrently running threads whereas parking allows you to control which threads are running concurrently. Whatever low-level primitives the OS provides (such as futexes) is more an implementation detail. I would tentatively make the claim that channels (in the abstract) are at heart an interface rather than a type of synchronisation per se. They can be implemented using Mutexes, pure atomics (if each message is a single integer) or any number of different ways. Of course, any specific implementation of a channel will have trade-offs. Some more so than others.
  - im3w1l 2y ago
    To me message passing is like it's own thing. It's the most natural way of thinking about information flow in a system consisting of physically separated parts.
  - LtWorf 2y ago
    What you think is not very relevant if it doesn't match how CPUs work.
    - ChrisSD 2y ago
      huh?
      - hedora 2y ago
        I think they mean that message channels are an expensive and performance unstable abstraction. You could address the concern by choosing a CPU architecture that included infinite capacity FIFOS that connected its cores into arbitrary runtime directed graphs. Of course, that architecture doesn’t exist. If it did, dispatching an instruction would have infinite tail latency and unbounded power consumption.
- i_don_t_know 2y ago
  > When I did my 20% on Go at Google, about 10 years ago, we already had a semi-formal rule that channels must not appear in exported function signatures. That sounds reasonable. From what little Erlang/Elixir code I’ve seen, the sending and receiving of messages is also hidden as an implementation detail in modules. The public interface did not expose concurrency or synchronization to callers. You might use them under the hood to implement your functionality, but it’s of no concern to callers, and you’re free to change the implementation without impacting callers.
  - throwawaymaths 2y ago
    AND because they're usually hidden as implementation detail, a consumer of your module can create simple mocks of your module (or you can provide one)
- catern 2y ago
  >If you take enough steps back and really think about it, the only synchronization primitive that exists is a futex (and maybe atomics). Everything else is an abstraction of some kind. You're going to be surprised when you learn that futexes are an abstraction too, ultimately relying on this thing called "cache coherence". And you'll be really surprised when you learn how cache coherence is implemented.
ricardobeat 2y ago
Strange to go all this length without mentioning the approaches that solve the problem in that first example: 1. send a close message on the channel that stops the goroutine 2. use a Context instance - `ctx.Done()` returns a channel you can select on Both are quite easy to grasp and implement.
- jtolds 2y ago
  Hi! No, I think you've misunderstood the assignment. The example posits that you have a "game" running, which should end when the last player leaves. While only using channels as a synchronization primitive (a la CSP), at what point do you decide the last player has left, and where and when do you call close on the channel?
  - blablabla123 2y ago
    I don't think channels should be used for everything. In some cases I think it's possible to end up with very lean code. But yes, if you have a stop channel for the other stop channel it probably means you should build your code around other mechanisms. Since CSP is mentioned, how much would this apply to most applications anyway? If I write a small server program, I probably won't want to write it on paper first. With one possible exception I never heard of anyone writing programs based on CSP (calculations?)
    - franticgecko3 2y ago
      > Since CSP is mentioned, how much would this apply to most applications anyway? If I write a small server program, I probably won't want to write it on paper first. With one possible exception I never heard of anyone writing programs based on CSP (calculations?) CSP is really in the realm of formal methods. No you wouldn't formulate your server program as CSP, but if you were writing software for a medical device, perhaps. This is the FDR4 model checker for CSP, it's a functional programming language that implements CSP semantics and may be used to assert (by exhaustion, IIRC) the correctness of your CSP model. https://cocotec.io/fdr/ https://cocotec.io/fdr/ I believe I'm in the minority of Go developers that have studied CSP, I fell into Go by accident and only took a CSP course at university because it was interesting, however I do give credit to studying CSP for my successes with Go.
regularfry 2y ago
This was 2016. Is it all still true? I know things will be backwards compatible, but I haven't kept track of what else has made it into the toolbox since then.
- mort96 2y ago
  Channels haven't really changed since then, unless there was some significant evolution between 2016 and ~2018 that I don't know about. 2025 Go code that uses channels looks very similar to 2018 Go code that uses channels.
  - regularfry 2y ago
    I'm also wondering about the internals though. There are a couple of places that GC and the hypothetical sufficiently-smart-compiler are called out in the article where you could think there might be improvements possible without breaking existing code.
- sapiogram 2y ago
  Absolutely nothing has changed at the language level, and for using channels and the `go` keyword directly, there isn't really tooling to help either. Most experienced Golang practitioners have reached the same conclusions as this blog post: Just don't use channels, even for problems that look simple. I used Go professionally for two years, and it's by far the worst thing about the language. The number of footguns is astounding.
  - pdimitar 2y ago
    Okay, but what do you use then?
- athoscouto 2y ago
  Yes. See update 2 FTA for a 2019 study on go concurrency bugs. Most go devs that I know consider using higher level synchronization mechanisms the right way to go (pun intended). sync.WaitGroup and errgroup are two common used options.
- deleted 2y ago
anarki8 2y ago
I find myself using channels in async Rust more than any other sync primitives. No more deadlock headaches. Easy to combine multiple channels in one state-keeping loop using combinators. And the dead goroutines problem described in the article doesn't exist in Rust.
- ninkendo 2y ago
  Same. It’s a pattern I’m reaching for a lot, whenever I have multiple logical things that need to run concurrently. Generally: - A struct that represents the mutable state I’m wrapping - A start(self) method which moves self to a tokio task running a loop reading from an mpsc::Receiver<Command> channel, and returns a Handle object which is cloneable and contains the mpsc::Sender end - The handle can be used to send commands/requests (including one shot channels for replies) - When the last handle is dropped, the mpsc channel is dropped and the loop ends It basically lets me think of each logical concurrent service as being like a tcp server that accepts requests. They can call each other by holding instances of the Handle type and awaiting calls (this can still deadlock if there’s a call cycle and the handling code isn’t put on a background task… in practice I’ve never made this mistake though) Some day I’ll maybe start using an actor framework (like Axum/etc) which formalizes this a bit more, but for now just making these types manually is simple enough.
- tuetuopay 2y ago
  This article has an eerie feeling now that async rust is production grade and widely used. I do use a lot the basic pattern of `loop { select! { ... } }` that manages its own state. And compared to the article, there's no dead coroutine, and no shared state managed by the coroutine: seeing the `NewGame` function return a `*Game` to the managed struct, this is an invitation for dumb bugs. This would be downright impossible in Rust, and coerces you in an actual CSP pattern where the interaction with the shared state is only through channels. Add a channel for exit, another for bookeeping, and you're golden. I often have a feeling that a lot of the complaints are self-inflicted Go problems. The author briefly touches on them with the special snowflakes that are the stdlib's types. Yes, genericity is one point where channels are different, but the syntax is another one. Why on earth is a `chan <- elem` syntax necessary over `chan.Send(elem)`? This would make non-blocking versions trivial to expose and discover for users (hello Rust's `.try_send()` methods). Oh and related to the first example of "exiting when all players left", we also see the lack of proper API for go channels: you can't query if there still are producers for the channel because gc and pointers and shared channel objetc itself and yadda. Meanwhile in rust, producers are reference-counted and the channel automatically closed when there are no more producers. The native Go channels can't do that (granted, they could, with a wrapper and dedicated sender and receiver types).
noor_z 2y ago
It's very possible I'm just bad at Go but it seems to me that the result of trying to adhere to CSP in my own Go projects is the increasing use of dedicated lifecycle management channels like `shutdownChan`. Time will tell how burdensome this pattern proves to be but it's definitely not trivial to maintain now.
- sapiogram 2y ago
  You're not bad at Go, literally everyone I know who has tried to do this has concluded it's a bad idea. Just stop using channels, there's a nice language hidden underneath the CSP cruft.
  - vrosas 2y ago
    I've found the smartest go engineer in the room is usually the one NOT using channels.
    - pdimitar 2y ago
      Okay, but what are the viable alternatives?
- fireflash38 2y ago
  Is using a server context a bad idea? Though tbh using it for the cancelation is a shutdown channel in disguise hah.
fpoling 2y ago
I once read a book from 1982 that was arguing about CSP implementation in Ada that it lead to proliferation of threads (called tasks in Ada) and code complexity when mutex -based solutions were simpler. Go implementations of CSP somewhat mitigated the problems raised in the book by supporting buffered channels, but even with that with CSP one end up with unnecessary tasks which also brings the problem of their lifetime management as the article mentioned.
- sapiogram 2y ago
  Unfortunately, Go also made their channels worse by having their nil semantics be complete lunacy. See the "channel API is inconsistent and just cray-cray" section in the article.
  - chuckadams 2y ago
    The fact that Go requires a zero value for everything, including nil for objects like it was Java and the 1990's all over again, is one of the reasons I'm not particularly inclined to use Go for anything. Even PHP is non-nullable-by-default nowadays.
    - NBJack 2y ago
      Wait until you try refactoring in Go. Java has plenty of warts, but the explicit inheritance in it and other languages makes working across large codebases much simpler when you need to restructure something. Structural typing is surprisingly messy in practice.
      - chuckadams 2y ago
        I love structural typing, use it all day long in TypeScript, and wish every language had it. I've never run into issues with refactoring, though I suppose renaming a field in a type when the consumers are only matching by a literal shape might be less than 100% reliable. It looks like Go does support anonymous interfaces, but I'm not aware of how much they're actually used in real-world code (whereas in TS inlined object shapes are fairly common in trivial cases).
anacrolix 2y ago
I've been using Go since 2011. One year less than the author. Channels are bad. No prioritization. No combining with other synchronisation primitives without extra goroutines. In Go, no way to select on a variable number of channels (without more goroutines). The poor type system doesn't let you improve abstractions. Basically anywhere I see a channel in most people's code particular in the public interface, I know it's going to be buggy. And I've seen so many bugs. Lots of abandoned projects are because they started with channels and never dug themselves out. The lure to use channels is too strong for new users. The nil and various strange shapes of channel methods aren't really a problem they're just hard for newbs. Channels in Go should really only be used for signalling, and only if you intend to use a select. They can also act as reducers, fan out in certain cases. Very often in those cases you have a very specific buffer size, and you're still only using them to avoid adding extra goroutines and reverting to pure signalling.
- politician 2y ago
  One nit: reflect.Select supports a dynamic set of channels. Very few programs need it though, so a rough API isn’t a bad trade-off. In my entire experience with Go, I’ve needed it once, and it worked perfectly.
  - lanstin 2y ago
    I almost always only use Channels as the data path between fixed sized pools of workers. At each point I can control if blocking or not, and my code uses all the (allocated) CPUs pretty evenly. Channels are excellent for this data flow design use case. I have a little pain when I do a cli as the work appears during the run and it’s tricky to guarantee you exit when all the work is done and not before. Usually Ihave a sleep one second, wait for wait group, sleep one more second at the end of the CLI main. If my work doesn’t take minutes or hours to run, I generally don’t use Go.
- jessekv 2y ago
  How about channel channels? https://github.com/twpayne/go-pubsub/blob/master/pubsub.go#L17 https://github.com/twpayne/go-pubsub/blob/master/pubsub.go#L...
zerr 2y ago
Is the tl;dr: instead of channels, just use mutexes (and a shared state) explicitly with goroutines?
- guilhas 2y ago
  Use the one that fits your problem https://go.dev/wiki/MutexOrChannel https://go.dev/wiki/MutexOrChannel But in any case you will end up using a wrapper on either
  - zerr 2y ago
    So akin to Erlang processes.
politician 2y ago
My rule of thumb is that the goroutine that writes to a channel is responsible for closing it. In this case, a deferred call to close the channel in HandlePlayer is sufficient. Still, this example has other issues (naked range over a channel?!) potentially contributing to the author’s confusion. However, this post was also written almost a decade ago, so perhaps it’s a result of being new to the language? If I cared to look, I’d probably be able to find the corresponding HN thread from that year full of arguments about this, hah.
- sateesh 2y ago
  > In this case, a deferred call to close the channel in HandlePlayer is sufficient It is not clear from the example, but I presume there would multiple players, i.e there will calls of the form: g.HandlePlayer(p1) g.HandlePlayer(p2) .. in such a case one player closing the channel would affect rest of the producers too.
- sapiogram 2y ago
  > My rule of thumb is that the goroutine that writes to a channel is responsible for closing it. In this case, a deferred call to close the channel in HandlePlayer is sufficient. This isn't your rule of thumb, it's the only practical way to do it. The problems arise when you have multiple goroutines writing to a channel, which is the case here. > Still, this example has other issues (naked range over a channel?!) potentially contributing to the author’s confusion. You sound way more confused than the author. I think you've misunderstood what the (admittedly very abstract) example is supposed to be doing.
franticgecko3 2y ago
I'd like to refute the 'channels are slow' part of this article. If you run a microbenchmark which seems like what has been done, then channels look slow. If you try the contention with thousands of goroutines on a high core count machine, there is a significant inflection point where channels start outperforming sync.Mutex The reason is that sync.Mutex, if left to wait long enough will enter a slow code path and if memory serves, will call out to a kernel futex. The channel will not do this because the mutex that a channel is built with is exists in the go runtime - that's the special sauce the author is complaining doesn't exist but didn't try hard enough to seek it out. Anecdotally, we have ~2m lines of Go and use channels extensively in a message passing style. We do not use channels to increment a shared number, because that's ridiculous and the author is disingenuous in their contrived example. No serious Go shop is using a channel for that.
- chuckadams 2y ago
  > We do not use channels to increment a shared number, because that's ridiculous and the author is disingenuous in their contrived example. No serious Go shop is using a channel for that. Talk about knocking down strawmen: it's a stand-in for shared state, and understanding that should be a minimum bar for serious discussion.
  - franticgecko3 2y ago
    And implying I don't understand toy examples and responding with this is apparently above the bar for serious discussion.
    - deleted 2y ago
      [deleted]
  - pdimitar 2y ago
    Dude. Criticizing a contrived example that is never used in production code is a very fair game. You didn't need to go all "but actually!" here.
- mrkeen 2y ago
DeathArrow 2y ago
I don't know Go, but can't the situation be improved somehow? Either make channels better or remove them altogether?
- sapiogram 2y ago
  As a language user? Just stop using them entirely. It's what most people do eventually.
  - deleted 2y ago
    [deleted]
- ReflectedImage 2y ago
  Well the problem is best stated as "I don't know how to use channels and I don't intend to learn". So perhaps some kinda of developer education is the solution?
thomashabets2 2y ago
Unlike the author, I would actually say that Go is bad. This article illustrates my frustration with Go very well, on a meta level. Go's design consistently at every turn chose the simplest (one might say "dumbest", but I don't mean it entirely derogatory) way to do something. It was the simplest most obvious choice made by a very competent engineer. But it was entirely made in isolation, not by a language design expert. Go designs did not actually go out and research language design. It just went with the gut feel of the designers. But that's just it, those rules are there for a reason. It's like the rules of airplane design: Every single rule was written in blood. You toss those rules out (or don't even research them) at your own, and your user's, peril. Go's design reminds me of Brexit, and the famous "The people of this country have had enough of experts". And like with Brexit, it's easy to give a lame catch phrase, which seems convincing and makes people go "well what's the problem with that, keeping it simple?". Explaining just what the problem is with this "design by catchphrase" is illustrated by the article. It needs ~100 paragraphs (a quick error prone scan was 86 plus sample code) to explain just why these choices leads to a darkened room with rakes sprinkled all over it. And this article is just about Go channels! Go could get a 100 articles like this written about it, covering various aspects of its design. They all have the same root cause: Go's designers had enough of experts, and it takes longer to explain why something leads to bad outcomes, than to just show the catchphrase level "look at the happy path. Look at it!". I dislike Java more than I dislike Go. But at least Java was designed, and doesn't have this particular meta-problem. When Go was made we knew better than to design languages this way.
- kbolino 2y ago
  Go's designers were experts. They had extensive experience building programming languages and operating systems. But they were working in a bit of a vacuum. Not only were they mostly addressing the internal needs of Google, which is a write-only shop as far as the rest of the software industry is concerned, they also didn't have broad experience across many languages, and instead had deep experience with a few languages.
  - emtel 2y ago
    Rob Pike was definitely not a PL expert and I don’t think he would claim to be. You can read his often-posted critique of C++ here: https://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html?m=1 https://commandcenter.blogspot.com/2012/06/less-is-exponenti... In it, he seems to believe that the primary use of types in programming languages is to build hierarchies. He seems totally unfamiliar with ideas behind ML or haskell.
guilhas 2y ago
Go channels, good or bad, are clearly a step up in concurrency/parallelism concepts and discourse I think that was one of the successes of Go Every big enough concurrent system will conclude sync primitives are dangerous and implement a queue system more similar to channels Mutexes always look easier for starters, but channels/queues will help you model the problem better in the long term, and debug Also as a rule of thumb you should probably handle panics every time you start a new thread/goroutine
jmyeet 2y ago
The biggest mistake I see people make with Go channels is prematurely optimizing their code by making channels buffered. This is almost always a mistake. It seems logical. You don't want your code to block. In reality, you've just made your code unpredictable and there's a good chance you don't know what'll happen when your buffered channel fills up and your code then actually blocks. You may have a deadlock and not realize it. So if the default position is unbuffered channels (which it should be), you then realize at some point that this is an inferior version of cooperative async/await. Another general principle is you want to avoid writing multithreaded application code. If you're locking mutexes or starting threads, you're probably going to have a bad time. An awful lot of code fits the model of serving an RPC or HTTP request and, if you can, you want that code to be single-threaded (async/await is fine).
- sapiogram 2y ago
  > So if the default position is unbuffered channels (which it should be), you then realize at some point that this is an inferior version of cooperative async/await. I feel so validated by this comment.
- franticgecko3 2y ago
  >The biggest mistake I see people make with Go channels is prematurely optimizing their code by making channels buffered. This is almost always a mistake. It seems logical. You don't want your code to block. Thank you. I've fixed a lot of bugs in code that assumes because a channel is buffered it is non-blocking. Channels are always blocking, because they have a fixed capacity; my favorite preemptive fault-finding exercise is to go through a codebase and set all channels to be unbuffered, lo-and-behold there's deadlocks everywhere. If that is the biggest mistake, then the second biggest mistake is attempting to increase performance of an application by increasing channel sizes. A channel is a pipe connecting two workers, if you make the pipe wider the workers do not process their work any faster, it makes them more tolerant of jitter and that's it. I cringe when I see a channel buffer with a size greater than ~100 - it's a a telltale sign of a misguided optimization or finger waving session. I've seen some channels sized at 100k for "performance" reasons, where the consumer is pushing out to the network, say 1ms for processing and network egress. Are you really expecting the consumer to block for 100 seconds, or did you just think bigger number = faster?
liendolucas 2y ago
Putting aside this particular topic, I'm seeing posts talking negatively about the language. I got my feet wet with Go many many years ago and for unknown reasons I never kept digging on it, so... Is it worth learning it? What problems are best solved with it?
- jtolds 2y ago
  Author of the post here, I really like Go! It's my favorite language! It has absolutely nailed high concurrency programming in a way that other languages' solutions make me cringe to think through (await/async are so gross and unnecessary!) If you are intending to do something that has multiple concurrent tasks ongoing at the same time, I would definitely reach for Go (and maybe be very careful or skip entirely using channels). I also would reach for Go if you intend to work with a large group of other software engineers. Go is rigid; when I first started programming I thought I wanted maximum flexibility, but Go brings uniformity to a group of engineers' output in a way that makes the overall team much more productive IMO. Basically, I think Go is the best choice for server-side or backend programming, with an even stronger case when you're working with a team.
  - liendolucas 2y ago
    Thanks for the tip! Will definitely take into account your insights on channels if I decide to dive into it.
    - jtolds 2y ago
      I have written channel code in the last week. It's part of the deal (especially with the context package). I'm just happy to see them restrained.
gwd 2y ago
He mentions the lack of generics; I wonder how he'd make his own channel++ library now that generics are well-established.
ajankovic 2y ago
I had time to spare so I toyed with the example exercise. Now I am not sure if I misunderstood something because solution is fairly simple using only channels: https://go.dev/play/p/tD8cWdKfkKW https://go.dev/play/p/tD8cWdKfkKW
- sateesh 2y ago
  Your example solution has only one player. Your solution won't work when there are multiple players.
  - ajankovic 2y ago
    I don't think you examined the code in full. main spawns 10 go routines that are constantly sending player scores to the game. That means 10 different players are sending their scores concurrently until someone reaches score of 100.
- jtolds 2y ago
  I replied to your comment on my website, but for posterity here, yes, I do think you did a good job for the part about exiting when bestScore > 100. There's nitpicks, but this is fine! It makes sense, and nice use of a select over a send. I did expect that this exercise would come after the first one though, and doing this on top of a solution to the first exercise is a bit harder. That said, I also don't mean to claim either are impossible. It's just tough to reason about.
boruto 2y ago
Quite a change in mood of comments compared to when it was posted last time. https://news.ycombinator.com/item?id=11210578 https://news.ycombinator.com/item?id=11210578
- jtolds 2y ago
  Seriously! This caused such a ruckus when I posted this 9 years ago. I lost some professional acquaintanceships over it! Definitely a different reception.
dingdingdang 2y ago
If channels are the wrong way to do things in Golang, what is the right way?
- fpoling 2y ago
  In Go in many cases channels are unavoidable due to API. As already was pointed out in other threads a good rule of thumb is not to use them in public method signatures. The valid use case for channels is to signal to consumer via channel close in Context.Done() style that something is ready that then can be fetched using separated API. Then if you need to serialize access, just use locks. WorkGroup can replace channels in surprisingly many cases. A message passing queue with priorities implemented on top of mutexes/signals can be used in many cases that require complex interactions between many components.
  - dingdingdang 2y ago
    Cheers for clear summary, I take it you mean sync.WaitGroup and not WorkGroup? As a beginner I started up with WaitGroup usage years ago since it was more intuitive to me.. worked fine afaict.
    - fpoling 2y ago
      Yes, WaitGroup of cause.
      - pdimitar 2y ago
        I am here to lengthen your suffering: of course**
nasretdinov 2y ago
I think since a concept of channel was something new and exciting back when Go was introduced, people (including myself) tried using it everywhere they could. Over time, as you collect your experience with the tool you get better at it, and certainly for shared state management channels are rarely the best option, however there still are quite a few places where you can't do something equivalent to what channels provide easily, which is to block until you've received new data. It just so happens that those situations are quite rare in Go.
codr7 2y ago
Agreed, channels are overrated and overused in Go. Like closures, channels are very flexible and can be used to implement just about anything; that doesn't mean doing so is a good idea. I would likely reach for atomics before mutexes in the game example.
GoRocks12 2y ago
Amusing, like the Blub paradox but backwards. Programmers with no experience in Go think they can critique it before they've understood it. If you don't understand how to use channels, you should learn first. I agree that you might have to learn by experimenting yourself, and that a) there is a small design flaw in Go channels, but one that is easily fixed; and b) the standard documentation does not teach good practices for using channels. First, the design flaw: close(channel) should be idempotent. It is not. Is this a fatal flaw? Hardly. The work around is trivial. Create a wrapper struct with a mutex that allows you to call Close() on the struct, and that effects an idempotent close of the member channel. Yes this is a bit of work, but you do it once, put it in a re-usable library, and never bother to think much about it again. b) poor recommended practices (range over channel). The original article makes this mistake, and it is what causes his problem: you can never use range over a channel in production code. You must always do any select on a channel alongside a shutdown (bailout) channel, so there will always be at least two channels being select-ed on. So yes. The docs could be better. It was immediately obvious to me when I learned Go 12 years ago that nobody at Google ever shuts down their services deliberately. Fortunately I was learning Test Driven Development at the time. So I was forced to figure out the above two rules pretty quickly. Once those two trivial fixes are in place, Go sails. There are Go libraries on github that do this for you. You don't even have to think. But you should. Handling errors is only as verbose as you want it to be. You do realize you can call a function instead of writing if err != nil so much, right? Sheesh. Go _is_ something of a sharp tool. Maybe we need to put a warning on it: for mature audiences only.
- jfauwasdf 2y ago
  > you can never use range over a channel in production code. You must always do any select on a channel alongside a shutdown (bailout) channel, so there will always be at least two channels being select-ed on. What if you spawned a new goroutine that waits for a waitgroup to complete and then closes the channel?
- maleldil 2y ago
  > Go _is_ something of a sharp tool. Maybe we need to put a warning on it: for mature audiences only This has been said for other languages, and it turns out no one is mature enough to consistently make it work without issues. The world has moved on; we can have proper tools that won't shoot your leg off. Why risk misusing the sharp tool when you can just use something safe to accomplish the same thing?
om8 2y ago
Channels are useful when they are really (rarely) needed. IMO Channel API should've been as ugly as reflect API to be considered only in extra cases.
Saul1998zx 2y ago
6451937099
Saul1998zx 2y ago
Si

160 points by mpweiher 2y ago

t8sr 2y ago

When I did my 20% on Go at Google, about 10 years ago, we already had a semi-formal rule that channels must not appear in exported function signatures. It turns out that using CSP in any large, complex codebase is asking for trouble, and that this is true even about projects where members of the core Go team did the CSP. If you take enough steps back and really think about it, the only synchronization primitive that exists is a futex (and maybe atomics). Everything else is an abstraction of some kind. If you're really determined, you can build anything out of anything. That doesn't mean it's always a good idea. Looking back, I'd say channels are far superior to condition variables as a synchronized cross-thread communication mechanism - when I use them these days, it's mostly for that. Locks (mutexes) are really performant and easy to understand and generally better for mutual exclusion. (It's in the name!)

throwaway150 2y ago
What is "20% on Go"? What is it 20% of?
- ramon156 2y ago
  I assume this means "20% of my work on go" aka 1 out of 5 work days working on golang
- NiloCK 2y ago
  Google historically allowed employees to self-direct 20% of their working time (onto any google project I think).
- darkr 2y ago
  At least historically, google engineers had 20% of their time to spend on projects not related to their core role
  - kyrra 2y ago
    This still exists today. For example, I am on the payments team but I have a 20% project working on protobuf. I had to get formal approval from my management chain and someone on the protobuf team. And it is tracked as part of my performance reviews. They just want to make sure I'm not building something useless that nobody wants and that I'm not just wasting the company's time.
    - vrosas 2y ago
      I see why they do this, but man it almost feels like asking your boss for approval on where you go on vacation. Do people get dinged if their 20% time project doesn't pan out, or they lose interest later on?
      - NBJack 2y ago
        Previously it could be anything you wanted. These days, you need formal approval. Google has changed a bit.
      - kyrra 2y ago
        It has nothing to do with success. It's entirely for making sure some one besides the person doing the 20% agrees with the idea behind the project.
        hedora 2y ago
        Lol. They’d be better off giving people the option to work 4 days if they also signed over right of first refusal for hobby projects.
    - rollcat 2y ago
      I never worked at Google (or any other large corp for that matter), but this sounds like the exact opposite of an environment that spawned GMail. As you think back even to the very early days of computing, you'll find individuals or small teams like Grace Hopper, the Unix gang, PARC, etc that managed to change history by "building something useless". Granted, throughout history that happened less than 1% of the time, but it will never happen if you never try. Maybe Google no longer has any space for innovation.
dfawcus 2y ago
How large do you deem to be large in this context? I had success in using a CSP style, with channels in many function signatures in a ~25k line codebase. It had ~15 major types of process, probably about 30 fixed instances overall in a fixed graph, plus a dynamic sub-graph of around 5 processes per 'requested action'. So those sub-graph elements were the only parts which had to deal with tear-down, and clean up. There were then additionally some minor types of 'process' (i.e. goroutines) within many of those major types, but they were easier to reason about as they only communicated with that major element. Multiple requested actions could be present, so there could be multiple sets of those 5 process groups connected, but they had a maximum lifetime of a few minutes. I only ended up using explicit mutexes in two of the major types of process. Where they happened to make most sense, and hence reduced system complexity. There were about 45 instances of the 'go' keyword. (Updated numbers, as I'd initially misremembered/miscounted the number of major processes)
- hedora 2y ago
  How many developers did that scale to? Code bases that I’ve seen that are written in that style are completely illegible. Once the structure of the 30 node graph falls out of the last developer’s head, it’s basically game over. To debug stuff by reading the code, each message ends up having 30 potential destinations. If a request involves N sequential calls, the control flow can be as bad as 30^N paths. Reading the bodies of the methods that are invoked generally doesn’t tell you which of those paths are wired up. In some real world code I have seen, a complicated thing wires up the control flow, so recovering the graph from the source code is equivalent to the halting problem. None of these problems apply to async/await because the compiler can statically figure out what’s being invoked, and IDE’s are generally as good at figuring that out as the compiler.
  - dfawcus 2y ago
    That was two main developers, one doing most of the code and design, the other a largely closed subset of 3 or 4 nodes. Plus three other developers co-opted for implementing some of the nodes. [1] The problem space itself could have probably grown to twice the number of lines of code, but there wouldn't have needed to be any more developers. Possibly only the original two. The others were only added for meeting deadlines. As to the graph, it was fixed, but not a full mesh. A set of pipelines, with no power of N issue, as the collection of places things could talk to was fixed. A simple diagram represented the major message flow between those 30 nodes. Testing of each node was able to be performed in isolation, so UT of each node covered most of the behaviour. The bugs were three deadlocks, one between two major nodes, one with one major node. The logging around the trigger for the deadlock allowed the cause to be determined and fixed. The bugs arose due to time constraints having prevented an analysis of the message flows to detect the loops/locks. So for most messages, there were a limited number of destinations, mostly two, for some 5. For a given "request", the flow of messages to the end of the fixed graph would be passing through 3 major nodes. That then spawned the creation of the dynamic graph, with it having two major flows. One a control flow through another 3, the other a data flow through a different 3. Within that dynamic graph there was a richer flow of messages, but the external flow from it simply had the two major paths. Yes, reading the bodies of the methods does not inform as to the flows. One either had to read the "main" routine which built the graph, or better refer to the graph diagram and message flows in the design document. Essentially a similar problem to dealing with "microservices", or plugable call-backs, where the structure can not easily be determined from the code alone. This is where design documentation is necessary. However I found it easier to comprehend and work with / debug due to each node being a prodable "black box", plus having the graph of connections and message flows. [1] Of those, only the first had any exerience with CSP or Go. The CSP expereince being with a library for C, the Go experience some minimal use a year earlier. The other developers were all new to CSP and Go. The first two developers were "senior" / "experienced".
ChrisSD 2y ago
I think the two basic synchronisation primitives are atomics and thread parking. Atomics allow you to share data between two or more concurrently running threads whereas parking allows you to control which threads are running concurrently. Whatever low-level primitives the OS provides (such as futexes) is more an implementation detail. I would tentatively make the claim that channels (in the abstract) are at heart an interface rather than a type of synchronisation per se. They can be implemented using Mutexes, pure atomics (if each message is a single integer) or any number of different ways. Of course, any specific implementation of a channel will have trade-offs. Some more so than others.
- im3w1l 2y ago
  To me message passing is like it's own thing. It's the most natural way of thinking about information flow in a system consisting of physically separated parts.
- LtWorf 2y ago
  What you think is not very relevant if it doesn't match how CPUs work.
  - ChrisSD 2y ago
    huh?
    - hedora 2y ago
      I think they mean that message channels are an expensive and performance unstable abstraction. You could address the concern by choosing a CPU architecture that included infinite capacity FIFOS that connected its cores into arbitrary runtime directed graphs. Of course, that architecture doesn’t exist. If it did, dispatching an instruction would have infinite tail latency and unbounded power consumption.
i_don_t_know 2y ago
> When I did my 20% on Go at Google, about 10 years ago, we already had a semi-formal rule that channels must not appear in exported function signatures. That sounds reasonable. From what little Erlang/Elixir code I’ve seen, the sending and receiving of messages is also hidden as an implementation detail in modules. The public interface did not expose concurrency or synchronization to callers. You might use them under the hood to implement your functionality, but it’s of no concern to callers, and you’re free to change the implementation without impacting callers.
- throwawaymaths 2y ago
  AND because they're usually hidden as implementation detail, a consumer of your module can create simple mocks of your module (or you can provide one)
catern 2y ago
>If you take enough steps back and really think about it, the only synchronization primitive that exists is a futex (and maybe atomics). Everything else is an abstraction of some kind. You're going to be surprised when you learn that futexes are an abstraction too, ultimately relying on this thing called "cache coherence". And you'll be really surprised when you learn how cache coherence is implemented.

ricardobeat 2y ago

Strange to go all this length without mentioning the approaches that solve the problem in that first example: 1. send a close message on the channel that stops the goroutine 2. use a Context instance - `ctx.Done()` returns a channel you can select on Both are quite easy to grasp and implement.

jtolds 2y ago
Hi! No, I think you've misunderstood the assignment. The example posits that you have a "game" running, which should end when the last player leaves. While only using channels as a synchronization primitive (a la CSP), at what point do you decide the last player has left, and where and when do you call close on the channel?
- blablabla123 2y ago
  I don't think channels should be used for everything. In some cases I think it's possible to end up with very lean code. But yes, if you have a stop channel for the other stop channel it probably means you should build your code around other mechanisms. Since CSP is mentioned, how much would this apply to most applications anyway? If I write a small server program, I probably won't want to write it on paper first. With one possible exception I never heard of anyone writing programs based on CSP (calculations?)
  - franticgecko3 2y ago
    > Since CSP is mentioned, how much would this apply to most applications anyway? If I write a small server program, I probably won't want to write it on paper first. With one possible exception I never heard of anyone writing programs based on CSP (calculations?) CSP is really in the realm of formal methods. No you wouldn't formulate your server program as CSP, but if you were writing software for a medical device, perhaps. This is the FDR4 model checker for CSP, it's a functional programming language that implements CSP semantics and may be used to assert (by exhaustion, IIRC) the correctness of your CSP model. https://cocotec.io/fdr/ https://cocotec.io/fdr/ I believe I'm in the minority of Go developers that have studied CSP, I fell into Go by accident and only took a CSP course at university because it was interesting, however I do give credit to studying CSP for my successes with Go.

regularfry 2y ago

This was 2016. Is it all still true? I know things will be backwards compatible, but I haven't kept track of what else has made it into the toolbox since then.

mort96 2y ago
Channels haven't really changed since then, unless there was some significant evolution between 2016 and ~2018 that I don't know about. 2025 Go code that uses channels looks very similar to 2018 Go code that uses channels.
- regularfry 2y ago
  I'm also wondering about the internals though. There are a couple of places that GC and the hypothetical sufficiently-smart-compiler are called out in the article where you could think there might be improvements possible without breaking existing code.
sapiogram 2y ago
Absolutely nothing has changed at the language level, and for using channels and the `go` keyword directly, there isn't really tooling to help either. Most experienced Golang practitioners have reached the same conclusions as this blog post: Just don't use channels, even for problems that look simple. I used Go professionally for two years, and it's by far the worst thing about the language. The number of footguns is astounding.
- pdimitar 2y ago
  Okay, but what do you use then?
athoscouto 2y ago
Yes. See update 2 FTA for a 2019 study on go concurrency bugs. Most go devs that I know consider using higher level synchronization mechanisms the right way to go (pun intended). sync.WaitGroup and errgroup are two common used options.
deleted 2y ago

anarki8 2y ago

I find myself using channels in async Rust more than any other sync primitives. No more deadlock headaches. Easy to combine multiple channels in one state-keeping loop using combinators. And the dead goroutines problem described in the article doesn't exist in Rust.

ninkendo 2y ago
Same. It’s a pattern I’m reaching for a lot, whenever I have multiple logical things that need to run concurrently. Generally: - A struct that represents the mutable state I’m wrapping - A start(self) method which moves self to a tokio task running a loop reading from an mpsc::Receiver<Command> channel, and returns a Handle object which is cloneable and contains the mpsc::Sender end - The handle can be used to send commands/requests (including one shot channels for replies) - When the last handle is dropped, the mpsc channel is dropped and the loop ends It basically lets me think of each logical concurrent service as being like a tcp server that accepts requests. They can call each other by holding instances of the Handle type and awaiting calls (this can still deadlock if there’s a call cycle and the handling code isn’t put on a background task… in practice I’ve never made this mistake though) Some day I’ll maybe start using an actor framework (like Axum/etc) which formalizes this a bit more, but for now just making these types manually is simple enough.
tuetuopay 2y ago
This article has an eerie feeling now that async rust is production grade and widely used. I do use a lot the basic pattern of `loop { select! { ... } }` that manages its own state. And compared to the article, there's no dead coroutine, and no shared state managed by the coroutine: seeing the `NewGame` function return a `*Game` to the managed struct, this is an invitation for dumb bugs. This would be downright impossible in Rust, and coerces you in an actual CSP pattern where the interaction with the shared state is only through channels. Add a channel for exit, another for bookeeping, and you're golden. I often have a feeling that a lot of the complaints are self-inflicted Go problems. The author briefly touches on them with the special snowflakes that are the stdlib's types. Yes, genericity is one point where channels are different, but the syntax is another one. Why on earth is a `chan <- elem` syntax necessary over `chan.Send(elem)`? This would make non-blocking versions trivial to expose and discover for users (hello Rust's `.try_send()` methods). Oh and related to the first example of "exiting when all players left", we also see the lack of proper API for go channels: you can't query if there still are producers for the channel because gc and pointers and shared channel objetc itself and yadda. Meanwhile in rust, producers are reference-counted and the channel automatically closed when there are no more producers. The native Go channels can't do that (granted, they could, with a wrapper and dedicated sender and receiver types).

noor_z 2y ago

It's very possible I'm just bad at Go but it seems to me that the result of trying to adhere to CSP in my own Go projects is the increasing use of dedicated lifecycle management channels like `shutdownChan`. Time will tell how burdensome this pattern proves to be but it's definitely not trivial to maintain now.

sapiogram 2y ago
You're not bad at Go, literally everyone I know who has tried to do this has concluded it's a bad idea. Just stop using channels, there's a nice language hidden underneath the CSP cruft.
- vrosas 2y ago
  I've found the smartest go engineer in the room is usually the one NOT using channels.
  - pdimitar 2y ago
    Okay, but what are the viable alternatives?
fireflash38 2y ago
Is using a server context a bad idea? Though tbh using it for the cancelation is a shutdown channel in disguise hah.

fpoling 2y ago

I once read a book from 1982 that was arguing about CSP implementation in Ada that it lead to proliferation of threads (called tasks in Ada) and code complexity when mutex -based solutions were simpler. Go implementations of CSP somewhat mitigated the problems raised in the book by supporting buffered channels, but even with that with CSP one end up with unnecessary tasks which also brings the problem of their lifetime management as the article mentioned.

sapiogram 2y ago
Unfortunately, Go also made their channels worse by having their nil semantics be complete lunacy. See the "channel API is inconsistent and just cray-cray" section in the article.
- chuckadams 2y ago
  The fact that Go requires a zero value for everything, including nil for objects like it was Java and the 1990's all over again, is one of the reasons I'm not particularly inclined to use Go for anything. Even PHP is non-nullable-by-default nowadays.
  - NBJack 2y ago
    Wait until you try refactoring in Go. Java has plenty of warts, but the explicit inheritance in it and other languages makes working across large codebases much simpler when you need to restructure something. Structural typing is surprisingly messy in practice.
    - chuckadams 2y ago
      I love structural typing, use it all day long in TypeScript, and wish every language had it. I've never run into issues with refactoring, though I suppose renaming a field in a type when the consumers are only matching by a literal shape might be less than 100% reliable. It looks like Go does support anonymous interfaces, but I'm not aware of how much they're actually used in real-world code (whereas in TS inlined object shapes are fairly common in trivial cases).

anacrolix 2y ago

I've been using Go since 2011. One year less than the author. Channels are bad. No prioritization. No combining with other synchronisation primitives without extra goroutines. In Go, no way to select on a variable number of channels (without more goroutines). The poor type system doesn't let you improve abstractions. Basically anywhere I see a channel in most people's code particular in the public interface, I know it's going to be buggy. And I've seen so many bugs. Lots of abandoned projects are because they started with channels and never dug themselves out. The lure to use channels is too strong for new users. The nil and various strange shapes of channel methods aren't really a problem they're just hard for newbs. Channels in Go should really only be used for signalling, and only if you intend to use a select. They can also act as reducers, fan out in certain cases. Very often in those cases you have a very specific buffer size, and you're still only using them to avoid adding extra goroutines and reverting to pure signalling.

politician 2y ago
One nit: reflect.Select supports a dynamic set of channels. Very few programs need it though, so a rough API isn’t a bad trade-off. In my entire experience with Go, I’ve needed it once, and it worked perfectly.
- lanstin 2y ago
  I almost always only use Channels as the data path between fixed sized pools of workers. At each point I can control if blocking or not, and my code uses all the (allocated) CPUs pretty evenly. Channels are excellent for this data flow design use case. I have a little pain when I do a cli as the work appears during the run and it’s tricky to guarantee you exit when all the work is done and not before. Usually Ihave a sleep one second, wait for wait group, sleep one more second at the end of the CLI main. If my work doesn’t take minutes or hours to run, I generally don’t use Go.
jessekv 2y ago
How about channel channels? https://github.com/twpayne/go-pubsub/blob/master/pubsub.go#L17 https://github.com/twpayne/go-pubsub/blob/master/pubsub.go#L...

zerr 2y ago

Is the tl;dr: instead of channels, just use mutexes (and a shared state) explicitly with goroutines?

guilhas 2y ago
Use the one that fits your problem https://go.dev/wiki/MutexOrChannel https://go.dev/wiki/MutexOrChannel But in any case you will end up using a wrapper on either
- zerr 2y ago
  So akin to Erlang processes.

politician 2y ago

My rule of thumb is that the goroutine that writes to a channel is responsible for closing it. In this case, a deferred call to close the channel in HandlePlayer is sufficient. Still, this example has other issues (naked range over a channel?!) potentially contributing to the author’s confusion. However, this post was also written almost a decade ago, so perhaps it’s a result of being new to the language? If I cared to look, I’d probably be able to find the corresponding HN thread from that year full of arguments about this, hah.

sateesh 2y ago
> In this case, a deferred call to close the channel in HandlePlayer is sufficient It is not clear from the example, but I presume there would multiple players, i.e there will calls of the form: g.HandlePlayer(p1) g.HandlePlayer(p2) .. in such a case one player closing the channel would affect rest of the producers too.
sapiogram 2y ago
> My rule of thumb is that the goroutine that writes to a channel is responsible for closing it. In this case, a deferred call to close the channel in HandlePlayer is sufficient. This isn't your rule of thumb, it's the only practical way to do it. The problems arise when you have multiple goroutines writing to a channel, which is the case here. > Still, this example has other issues (naked range over a channel?!) potentially contributing to the author’s confusion. You sound way more confused than the author. I think you've misunderstood what the (admittedly very abstract) example is supposed to be doing.

franticgecko3 2y ago

I'd like to refute the 'channels are slow' part of this article. If you run a microbenchmark which seems like what has been done, then channels look slow. If you try the contention with thousands of goroutines on a high core count machine, there is a significant inflection point where channels start outperforming sync.Mutex The reason is that sync.Mutex, if left to wait long enough will enter a slow code path and if memory serves, will call out to a kernel futex. The channel will not do this because the mutex that a channel is built with is exists in the go runtime - that's the special sauce the author is complaining doesn't exist but didn't try hard enough to seek it out. Anecdotally, we have ~2m lines of Go and use channels extensively in a message passing style. We do not use channels to increment a shared number, because that's ridiculous and the author is disingenuous in their contrived example. No serious Go shop is using a channel for that.

chuckadams 2y ago
> We do not use channels to increment a shared number, because that's ridiculous and the author is disingenuous in their contrived example. No serious Go shop is using a channel for that. Talk about knocking down strawmen: it's a stand-in for shared state, and understanding that should be a minimum bar for serious discussion.
- franticgecko3 2y ago
  And implying I don't understand toy examples and responding with this is apparently above the bar for serious discussion.
  - deleted 2y ago
    [deleted]
- pdimitar 2y ago
  Dude. Criticizing a contrived example that is never used in production code is a very fair game. You didn't need to go all "but actually!" here.
mrkeen 2y ago

DeathArrow 2y ago

I don't know Go, but can't the situation be improved somehow? Either make channels better or remove them altogether?

sapiogram 2y ago
As a language user? Just stop using them entirely. It's what most people do eventually.
- deleted 2y ago
  [deleted]
ReflectedImage 2y ago
Well the problem is best stated as "I don't know how to use channels and I don't intend to learn". So perhaps some kinda of developer education is the solution?

thomashabets2 2y ago

Unlike the author, I would actually say that Go is bad. This article illustrates my frustration with Go very well, on a meta level. Go's design consistently at every turn chose the simplest (one might say "dumbest", but I don't mean it entirely derogatory) way to do something. It was the simplest most obvious choice made by a very competent engineer. But it was entirely made in isolation, not by a language design expert. Go designs did not actually go out and research language design. It just went with the gut feel of the designers. But that's just it, those rules are there for a reason. It's like the rules of airplane design: Every single rule was written in blood. You toss those rules out (or don't even research them) at your own, and your user's, peril. Go's design reminds me of Brexit, and the famous "The people of this country have had enough of experts". And like with Brexit, it's easy to give a lame catch phrase, which seems convincing and makes people go "well what's the problem with that, keeping it simple?". Explaining just what the problem is with this "design by catchphrase" is illustrated by the article. It needs ~100 paragraphs (a quick error prone scan was 86 plus sample code) to explain just why these choices leads to a darkened room with rakes sprinkled all over it. And this article is just about Go channels! Go could get a 100 articles like this written about it, covering various aspects of its design. They all have the same root cause: Go's designers had enough of experts, and it takes longer to explain why something leads to bad outcomes, than to just show the catchphrase level "look at the happy path. Look at it!". I dislike Java more than I dislike Go. But at least Java was designed, and doesn't have this particular meta-problem. When Go was made we knew better than to design languages this way.

kbolino 2y ago
Go's designers were experts. They had extensive experience building programming languages and operating systems. But they were working in a bit of a vacuum. Not only were they mostly addressing the internal needs of Google, which is a write-only shop as far as the rest of the software industry is concerned, they also didn't have broad experience across many languages, and instead had deep experience with a few languages.
- emtel 2y ago
  Rob Pike was definitely not a PL expert and I don’t think he would claim to be. You can read his often-posted critique of C++ here: https://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html?m=1 https://commandcenter.blogspot.com/2012/06/less-is-exponenti... In it, he seems to believe that the primary use of types in programming languages is to build hierarchies. He seems totally unfamiliar with ideas behind ML or haskell.

guilhas 2y ago

Go channels, good or bad, are clearly a step up in concurrency/parallelism concepts and discourse I think that was one of the successes of Go Every big enough concurrent system will conclude sync primitives are dangerous and implement a queue system more similar to channels Mutexes always look easier for starters, but channels/queues will help you model the problem better in the long term, and debug Also as a rule of thumb you should probably handle panics every time you start a new thread/goroutine

jmyeet 2y ago

The biggest mistake I see people make with Go channels is prematurely optimizing their code by making channels buffered. This is almost always a mistake. It seems logical. You don't want your code to block. In reality, you've just made your code unpredictable and there's a good chance you don't know what'll happen when your buffered channel fills up and your code then actually blocks. You may have a deadlock and not realize it. So if the default position is unbuffered channels (which it should be), you then realize at some point that this is an inferior version of cooperative async/await. Another general principle is you want to avoid writing multithreaded application code. If you're locking mutexes or starting threads, you're probably going to have a bad time. An awful lot of code fits the model of serving an RPC or HTTP request and, if you can, you want that code to be single-threaded (async/await is fine).

sapiogram 2y ago
> So if the default position is unbuffered channels (which it should be), you then realize at some point that this is an inferior version of cooperative async/await. I feel so validated by this comment.
franticgecko3 2y ago
>The biggest mistake I see people make with Go channels is prematurely optimizing their code by making channels buffered. This is almost always a mistake. It seems logical. You don't want your code to block. Thank you. I've fixed a lot of bugs in code that assumes because a channel is buffered it is non-blocking. Channels are always blocking, because they have a fixed capacity; my favorite preemptive fault-finding exercise is to go through a codebase and set all channels to be unbuffered, lo-and-behold there's deadlocks everywhere. If that is the biggest mistake, then the second biggest mistake is attempting to increase performance of an application by increasing channel sizes. A channel is a pipe connecting two workers, if you make the pipe wider the workers do not process their work any faster, it makes them more tolerant of jitter and that's it. I cringe when I see a channel buffer with a size greater than ~100 - it's a a telltale sign of a misguided optimization or finger waving session. I've seen some channels sized at 100k for "performance" reasons, where the consumer is pushing out to the network, say 1ms for processing and network egress. Are you really expecting the consumer to block for 100 seconds, or did you just think bigger number = faster?

liendolucas 2y ago

Putting aside this particular topic, I'm seeing posts talking negatively about the language. I got my feet wet with Go many many years ago and for unknown reasons I never kept digging on it, so... Is it worth learning it? What problems are best solved with it?

jtolds 2y ago
Author of the post here, I really like Go! It's my favorite language! It has absolutely nailed high concurrency programming in a way that other languages' solutions make me cringe to think through (await/async are so gross and unnecessary!) If you are intending to do something that has multiple concurrent tasks ongoing at the same time, I would definitely reach for Go (and maybe be very careful or skip entirely using channels). I also would reach for Go if you intend to work with a large group of other software engineers. Go is rigid; when I first started programming I thought I wanted maximum flexibility, but Go brings uniformity to a group of engineers' output in a way that makes the overall team much more productive IMO. Basically, I think Go is the best choice for server-side or backend programming, with an even stronger case when you're working with a team.
- liendolucas 2y ago
  Thanks for the tip! Will definitely take into account your insights on channels if I decide to dive into it.
  - jtolds 2y ago
    I have written channel code in the last week. It's part of the deal (especially with the context package). I'm just happy to see them restrained.

gwd 2y ago

He mentions the lack of generics; I wonder how he'd make his own channel++ library now that generics are well-established.

ajankovic 2y ago

I had time to spare so I toyed with the example exercise. Now I am not sure if I misunderstood something because solution is fairly simple using only channels: https://go.dev/play/p/tD8cWdKfkKW https://go.dev/play/p/tD8cWdKfkKW

sateesh 2y ago
Your example solution has only one player. Your solution won't work when there are multiple players.
- ajankovic 2y ago
  I don't think you examined the code in full. main spawns 10 go routines that are constantly sending player scores to the game. That means 10 different players are sending their scores concurrently until someone reaches score of 100.
jtolds 2y ago
I replied to your comment on my website, but for posterity here, yes, I do think you did a good job for the part about exiting when bestScore > 100. There's nitpicks, but this is fine! It makes sense, and nice use of a select over a send. I did expect that this exercise would come after the first one though, and doing this on top of a solution to the first exercise is a bit harder. That said, I also don't mean to claim either are impossible. It's just tough to reason about.

boruto 2y ago

Quite a change in mood of comments compared to when it was posted last time. https://news.ycombinator.com/item?id=11210578 https://news.ycombinator.com/item?id=11210578

jtolds 2y ago
Seriously! This caused such a ruckus when I posted this 9 years ago. I lost some professional acquaintanceships over it! Definitely a different reception.

dingdingdang 2y ago

If channels are the wrong way to do things in Golang, what is the right way?

fpoling 2y ago
In Go in many cases channels are unavoidable due to API. As already was pointed out in other threads a good rule of thumb is not to use them in public method signatures. The valid use case for channels is to signal to consumer via channel close in Context.Done() style that something is ready that then can be fetched using separated API. Then if you need to serialize access, just use locks. WorkGroup can replace channels in surprisingly many cases. A message passing queue with priorities implemented on top of mutexes/signals can be used in many cases that require complex interactions between many components.
- dingdingdang 2y ago
  Cheers for clear summary, I take it you mean sync.WaitGroup and not WorkGroup? As a beginner I started up with WaitGroup usage years ago since it was more intuitive to me.. worked fine afaict.
  - fpoling 2y ago
    Yes, WaitGroup of cause.
    - pdimitar 2y ago
      I am here to lengthen your suffering: of course**

nasretdinov 2y ago

I think since a concept of channel was something new and exciting back when Go was introduced, people (including myself) tried using it everywhere they could. Over time, as you collect your experience with the tool you get better at it, and certainly for shared state management channels are rarely the best option, however there still are quite a few places where you can't do something equivalent to what channels provide easily, which is to block until you've received new data. It just so happens that those situations are quite rare in Go.

codr7 2y ago

Agreed, channels are overrated and overused in Go. Like closures, channels are very flexible and can be used to implement just about anything; that doesn't mean doing so is a good idea. I would likely reach for atomics before mutexes in the game example.

GoRocks12 2y ago

Amusing, like the Blub paradox but backwards. Programmers with no experience in Go think they can critique it before they've understood it. If you don't understand how to use channels, you should learn first. I agree that you might have to learn by experimenting yourself, and that a) there is a small design flaw in Go channels, but one that is easily fixed; and b) the standard documentation does not teach good practices for using channels. First, the design flaw: close(channel) should be idempotent. It is not. Is this a fatal flaw? Hardly. The work around is trivial. Create a wrapper struct with a mutex that allows you to call Close() on the struct, and that effects an idempotent close of the member channel. Yes this is a bit of work, but you do it once, put it in a re-usable library, and never bother to think much about it again. b) poor recommended practices (range over channel). The original article makes this mistake, and it is what causes his problem: you can never use range over a channel in production code. You must always do any select on a channel alongside a shutdown (bailout) channel, so there will always be at least two channels being select-ed on. So yes. The docs could be better. It was immediately obvious to me when I learned Go 12 years ago that nobody at Google ever shuts down their services deliberately. Fortunately I was learning Test Driven Development at the time. So I was forced to figure out the above two rules pretty quickly. Once those two trivial fixes are in place, Go sails. There are Go libraries on github that do this for you. You don't even have to think. But you should. Handling errors is only as verbose as you want it to be. You do realize you can call a function instead of writing if err != nil so much, right? Sheesh. Go _is_ something of a sharp tool. Maybe we need to put a warning on it: for mature audiences only.

jfauwasdf 2y ago
> you can never use range over a channel in production code. You must always do any select on a channel alongside a shutdown (bailout) channel, so there will always be at least two channels being select-ed on. What if you spawned a new goroutine that waits for a waitgroup to complete and then closes the channel?
maleldil 2y ago
> Go _is_ something of a sharp tool. Maybe we need to put a warning on it: for mature audiences only This has been said for other languages, and it turns out no one is mature enough to consistently make it work without issues. The world has moved on; we can have proper tools that won't shoot your leg off. Why risk misusing the sharp tool when you can just use something safe to accomplish the same thing?

om8 2y ago

Channels are useful when they are really (rarely) needed. IMO Channel API should've been as ugly as reflect API to be considered only in extra cases.

Saul1998zx 2y ago

6451937099

Saul1998zx 2y ago