19 ms·
Go channels are bad
- Jabbles 11y agoEffective Go has always said: Do not communicate by sharing memory; instead, share memory by communicating. This approach can be taken too far. Reference counts may be best done by putting a mutex around an integer variable, for instance. https://golang.org/doc/effective_go.html#sharing https://golang.org/doc/effective_go.html#sharing
- poizan42 11y agoReference counts are best done using interlocked increment/decrement primitives.
- catnaroek 11y agos/interlocked/atomic/
- chrisseaton 11y agoI wonder if there are any compilers which can replace mutex.lock { x++ } With a 'lock xaddl x 1' instruction.
- nly 11y agoIt's conceivable, if you made mutexes compiler/language intrinsic, but as long as you're calling pthread_mutex_lock, the compiler has to assume that that pthread library, which is linked dynamically, is interchangeable, and can do anything it likes to memory. That includes mutating x
- pcwalton 11y agoThat hasn't inhibited optimizations for a long time. Disassemble a call to printf("Hello world") in optimized clang output and look at what it turns into.
- chrisseaton 11y agoYes if the library is covered by a standard just like the language is, then the compiler can make assumptions. Also threads are a language feature in C and C++ now.
- deleted 11y ago[deleted]
- pjmlp 11y agoJava and .NET will do it if you make use of InterlockedIncrement APIs.
- chrisseaton 11y agoBut I think InterlockedIncrement is just 'lock xaddl x 1', so using InterlockedIncrement would be to do it manually. I'm asking if any compiler can take a statement which uses a high level, general purpose lock and increments a variable inside it using conventional language expressions, and convert it to use 'lock xaddl x 1' (perhaps via InterlockedIncrement or whatever other intrinsics you have) instead. I only know Java well, not .NET, but I'm pretty sure no Java compiler does it.
- pjmlp 11y agoAh, I missed the point.
- mike_hearn 11y agoIt's not quite the same thing but recent JVMs can translate synchronised blocks into Intel TSX transactions, which means multiple threads can run inside the lock at once, with rollback and retry if interference is detected at the hardware (cache line) level. So yeah .... almost. But it's fancy and cutting edge stuff.
- deleted 11y ago[deleted]
- jupp0r 11y agoThere ist std::atomic in recent C++ flavors
- chrisseaton 11y agoRight. My question is whether we can translate locks which only reference a single variable, to use something like std::atomic, automatically.
- api 11y agoI thought Go had gc. Why would you ever need reference counts?
- voidlogic 11y agoNon-memory resources that exist within your application. Example: Perhaps once no instances of an object are in use, you want to remove that object from persistent storage such as a DB.
- api 11y agoDoes Go not have finalizers? These are mostly solved problems since the Smalltalk era. Haven't learned Go yet and from what I read I'd be better off with Rust or something that would stretch my brain more like Haskell. When I read about it I get the sense that we are reinventing stuff from the 90s. But hey, it's hip.
- catnaroek 11y agoSometimes you need to guarantee that resources are cleaned up in a timely fashion. Finalizers don't help here.
- hacknat 11y agoPiggy backing off @catnaroek, go does have finalizers too though.
- pcwalton 11y agoGo does have runtime.SetFinalizer: https://golang.org/pkg/runtime/#SetFinalizer https://golang.org/pkg/runtime/#SetFinalizer But beware of finalizer ordering issues.
- chrsm 11y agoExample: Recently built a small service that responds to requests and walks various files looking for data. The service can be asked to unload (close) the file, but it's hard to say whether it's in-use at the time without some kind of reference count to current requests using the file.
- mike_hearn 11y agoI'm not sure making an absolute statement ("do not...") followed by "... actually do, sometimes" is helpful. How is this different to any other language that gives you a toolbox of synchronisation primitives?
- advanderveer 11y agoI see channels as an architectural option when it comes to structuring the communication between components of my software. Mutexes are another option that are more effective in situations where multiple threads may access the interface of a single structure. E.g I use channels to distribute os.Signals througout my software and a mutex for making a "context" structure thread safe. Right tool for the right job
- jemfinch 11y agoEven when that's the case, it's rare that fixed-size buffered or unbuffered channels are really the best option for communication between different components of your software. A simple mutex-guarded queue is easier to begin with and easier to evolve when requirements change. You can prioritize queued work trivially and transparently; you can add batch processing, monitoring, and resolve other production issues without any undue refactoring: it can all be encapsulated behind your mutex-guarded queue. It's really quite a pity that Go's channel syntax treats channels as unique snowflakes, rather just being sugar for calls into an interface that would allow the underlying channel implementation to differ based on software needs.
- richard_todd 11y agoThat's an excellent example (in a long list) of things that would be possible with generics, or even parameterized packages. They could have provided an interface Channel[T] with syntax sugar if desirable. But as it is, everything in Go that can handle multiple types has snowflake status.
- woodcut 11y agoI find it hard to read something when the language used is so patronising.
- nevir 11y agoHmm, didn't get that vibe at all. Tone came off as a little exasperated to me, but mostly all about giving enough background to back up the claims
- feathj 11y agoThe language is one thing. I am so tired of trying to read articles riddled with gifs.
- arethuza 11y agoI agree - very distracting. However, at least it made me find the "Disable Image Animations" option in my browser! Edit: If you have web developer toolbar installed in Firefox then it is: Images > Disable Images > Disable Image Animations
- plorkyeran 11y agoLuckily the article doesn't use any images for content, so disabling images for the page (or deleting them all via the inspector) was an option, and made the article far more readable. I wish I had the magic power of being able to read text with an animated gif next to it without getting distracted every other word that some people apparently have.
- coldtea 11y agoI'm so tired of reading negative comments about entirely subjective (others might appreciate the gifs) and totally skippable if one doesn't like them (you can also ignore them) elements of a good post.
- woodcut 11y agoCriticising how information is communicated is wholly valid.
- travjones 11y agoThis was a well-written and entertaining post. It represents the kind of self-reflection every programming community should encourage. Too often are devs zealously supportive of their language of choice without considering thoughtful critiques that could make their chosen language even better, and/or present an alternate way of looking at things that makes one better at programming in general.
- lazyjones 11y agoSaving the highscore in a goroutine becomes more interesting if that action can block or simply take a while, both more realistic occurrences than such minimalistic examples.
- deleted 11y ago[deleted]
- dilap 11y agoFunny timing for me -- last Friday I rewrote some code from channels to traditional sync primitives (to the code's improvement), and I was musing in my head that while everyone always says "don't communicate by sharing, share by communicating, yada yada," it doesn't seem to work out that way in practice. I think the article is well-written, and clearly comes from a place of deep experience and understanding. Good stuff.
- gort 11y agoI had a similar experience in the opposite direction. Two weeks ago I moved some code from a mutex-based design (including a map of string to mutex, which itself needed to be behind a mutex) to channels, and I love it, though the result seemed about 10% slower. I guess the message is: everything has its place; don't make a straight-jacket for yourself.
- dilap 11y agoNo doubt it's quite easy to go down ratholes using mutexes and whatnot. It's a very low-level way of synchronizing. The OPs critques of the specific design of channels as implemented by Go seem on-point to me.
- drdaeman 11y agoOfftopic: that animated image is literally nauseating. Consider removing it, or making it animate just once and then halt. It was meant to be "fun" or whatever but, seriously, I wasn't able to read the text when it looped over and over in the corner of the eye.
- tantalor 11y agohttp://superuser.com/questions/23655/how-to-stop-animated-gifs-in-google-chrome http://superuser.com/questions/23655/how-to-stop-animated-gi...
- gkya 11y agohttps://en.wikipedia.org/wiki/List_of_web_browsers https://en.wikipedia.org/wiki/List_of_web_browsers
- jtolds 11y agoAlright, alright, I froze a bunch of the animated gifs. There were too many, I get it.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- joeGrigg 11y agoCan't handle the danknetude of the memes.
- deleted 11y ago[deleted]
- michaelwww 11y agoI felt the same way so I made a bookmarklet to blank out images. I see it's not needed now, but I'm set for future pages that do this. javascript:(function (){var x = document.getElementsByTagName("img");for (i = 0; i < x.length; i++){x[i].setAttribute("src","");}}());
- amelius 11y agoThis article should have been titled: "Go channels considered harmful" :)
- kbenson 11y agoThankfully it wasn't, and we were spared the ensuing discussion about how "..considered harmful" is good/bad/overused/misunderstood/causes cancer.
- falcolas 11y agoInstead we get the debate of whether "and you should feel bad" is appropriate in a title. I don't believe it is, but ultimately click bait is click bait.
- kbenson 11y agoEh, I don't mind click bait titles as long as the article delivers, and the title isn't too egregious in its manipulation. In this case, I think it's pretty well understood by most that the title is poking fun, since taking it truthfully is fairly ridiculous.
- hacknat 11y agoI think I've just come to accept that sychronization is the pain point in any language. It's callbacks, promises, and the single event loop in nodejs. It's channels in golang. No one can come up with a single abstraction for synchronization without it failing in some regard. I code in go quite a bit and I just try to avoid synchronization like the plague. Are there gripes I have with the language? Sure, CS theory states that a thread safe hash table can perform just about as well as a none-thread safe, so why don't we have one in go? However... Coming up with a valid case where a language's synchronization primitive fails and then flaming it as an anti-pattern (for the clicks and the attention, I presume) is trolling and stupid.
- yetihehe 11y ago> No one can come up with a single abstraction for synchronization without it failing in some regard. Erlang did. Or at least it's as close as possible.
- hacknat 11y agoI'm not saying Erlang isn't great, but if you need to pass a large datastructure around between Erlang processes then copy message passing starts to be a lot and you need to share memory. You can do it in Erlang, but I'd hardly call it great, and you're avoiding the sync primitive that Erlang offers.
- catnaroek 11y agoHow about Rust's “share by transferring ownership”? (0) In the general case, whatever object you give to a third party, you don't own anymore. And the type checker enforces this. (1) Unless the object's type supports shallow copying, in which case, you get to keep a usable copy after the move. (2) If the object's type doesn't support shallow copying, but supports deep cloning, you can also keep a copy [well, clone], but only if you explicitly request it. This ensures that communication is always safe, and never more expensive than it needs to be. --- Sorry, I can't post a proper reply because I'm “submitting too fast”, so I'll reply here... The solution consists of multiple steps: (0) Wrap the resource in a RWLock [read-write lock: http://doc.rust-lang.org/std/sync/struct.RwLock.html http://doc.rust-lang.org/std/sync/struct.RwLock.html], which can be either locked by multiple readers or by a single writer. (1) The RWLock itself can't be cloned, so wrap it in an Arc [atomically reference-counted pointer: http://doc.rust-lang.org/std/sync/struct.Arc.html http://doc.rust-lang.org/std/sync/struct.Arc.html], which can be cloned. (2) Clone and send to as many parties as you wish. --- I still can't post a proper reply, so... Rust's ownership and borrowing system is precisely what makes RWLock and Arc work correctly.
- hacknat 11y agoShould have been titled, "Interesting Cases Where Go Channels Fail"...
- david-given 11y agoI've always really, really liked Ada's rendezvous-based concurrency. There's more to it than I can really describe here, but in effect it allows you to treat a thread as an object with methods; calling a method on the object sends a message to the thread. The thread's main code can, at any point, block and wait for a message, or combination of messages. The handling code looks like this: ...some code... accept DoSomething(value: in out integer) do ...some code here... end ...some more code... That causes the thread to block and wait for the message. When the message is sent, the caller blocks, the receiver runs the handler, then the caller resumes. The beauty here is that inside the message handler, you know that the caller is blocked... which means it's safe to pass parameters by pointer[]. Everywhere the parameter's in scope, the parameter is safe to use. The type system won't let the thread store the pointer anywhere without copying the contents first, so you get zero-copy messaging and* it's failsafe. You can also do really cool stuff with timeouts, guards, automatic thread termination, etc. Here's a simple multithreaded queue (syntax and logic not guaranteed, it's been a while): loop select when usage < capacity => accept Push(value: in integer) do data[usage] := value; usage := usage + 1; end; or when usage > 0 => accept Pop(value: out integer) do usage := usage - 1; value := data[usage]; end; or terminate; end select; end loop; Multithreaded! Blocks the client automatically if they pop while the queue's empty or push while it's full! Automatically terminates the thread when the last connection goes away and the thread leaves scope! Thread safe! Readable! I'd love to be able to do this in a more mainstream language. [*] This is a simplification. Ada's pointers are not like other language's pointers.
- msbarnett 11y agoThe older I get, the clearer it is that Ada was the answer to the last 2 decades worth of problems (fast, able to go low-level when you need to, very type safe, easy concurrency primitives) and we all just ignored it because it wasn't fashionable.
- bliti 11y agoYou and me both. I day dream about working on Ada. end of random quibble
- Animats 11y agoGood points. The author points out that channel teardown is hard. He's right. Figuring out how to shut down your Go program cleanly can be difficult, especially since calling "close" on a closed channel causes a panic. You have to send an EOF on each channel so the receiver knows to stop. When you have a pair of channels going in opposite directions between two goroutines, and either end can potentially initiate shutdown, it gets messy. At least in the original implementation, "select" for more than one option was really slow and complex. The single-input case was handled efficiently with generated code, but for N > 1, a very general library mechanism with several heap allocations for each message was used. This means having both a wait for data and a timeout in a select puts you through the slow path. Not good. Someone did an analysis of existing programs and found that N=1 was most common, N=2 was reasonably common, and N>2 was rare. N=2 needs special case support. QNX interprocess messaging has a similar architecture. But they don't have the panic on close problem, and timeout is handled efficiently. So you can generally shut things down by closing something. As each process is notified about the close, it closes any channels with which it is involved, even if some other process has already closed them. The closes thus cascade and everything shuts down cleanly. Processes that time out at a message receive check to see if the rest of the system is still running, and shut down if it's not. Go's "share by communicating" would be more useful if Go had Rust's borrow checker, so you could share data without races. Yes, Go has a run-time race detector, but that's only useful if races are common enough that they occur during testing.
- helper 11y agoThe panic when calling close on a closed channel is a bit annoying. Recently I've been using x/net/context to signal goroutines instead of closing a channel. The CancelContext allows you to call cancel multiple times.
- jerf 11y ago"When you have a pair of channels going in opposite directions between two goroutines, and either end can potentially initiate shutdown, it gets messy." It does get messy to do it correctly, but I've found in the end it comes out less messy to have a channel communicating back to the sender that can be closed if you want the recipient to be able to close channel. I haven't needed it very often, but it happens. It still ends up simpler than hacking around the problem by trying to "close" the channel from the wrong end and the resulting panic handling. For concreteness, at least from what I've experienced, the "messiness" is that if you close one of these channels, you may have to "drain" the other channel lest you let the other side block. If the other side is only using the channel in a "select" block with other options you may not need to but if it ever does a "bare" send you need to wait for the other end to send its close. This can be particularly complicated if for some reason the "draining" process has to do something other than drop the messages on the floor.
- PaulHoule 11y agoPixie dust that makes concurrency problems go away is an antipattern.
- sagichmal 11y agoThis is a frustrating and overly-exasperated post which reaches conclusions that have always been part of the Go canon. APIs should be designed synchronously, and the callers should orchestrate concurrency if they choose -- yes! Channels are useful in some circumstances, but if you just want to synchronize access to shared memory (like the author's example) then you should just use a mutex -- yes! These are well understood truths. Novices to the language have a tendency to overuse channels. Here is Andrew Gerrand addressing precisely this point two years ago at GopherCon: https://www.youtube.com/watch?v=u-kkf76TDHE&t=815 https://www.youtube.com/watch?v=u-kkf76TDHE&t=815 Neither the referenced performance characteristics via Tyler Treat, nor the FUD surrounding channel-based program architecture, invalidate channels generally. One does have to think carefully about ownership hierarchies: only one goroutine gets to close the channel. And if it's in a hot loop, a channel will always perform worse than a mutex: channels use mutexes internally. But plenty of problems are solved very elegantly with channel-based CSP-style message passing. It's unfortunate that articles like this are written and gain traction. The explicit instruction to [new] Go programmers is that they should avoid channels, even that they are badly implemented, and both of those things are false.
- mordocai 11y agoAs a non-go programmer, I'm pretty sure the author made some very good objective arguments that channels are in fact badly implemented.
- sagichmal 11y agoRead Tyler's original article for a less FUDdy take on it. Channels are always slower than mutexes, which is obvious when you understand their implementation. They are definitely not badly implemented as a general rule.
- zeeboo 11y agoThe api definitely is badly implemented and makes them hard to use. That's the point of the post. There are design decisions around channels (sends panicing, close panicing, nil channels blocking) that make them hard to understand, follow, and compose concurrent solutions.
- gravypod 11y agoIn non-critical things (not important to execution speed), is it still acceptable to use go channels? I'm always weary of using a mutex because then I have to spend a much larger amount of time checking to see if it will lock.
- GhotiFish 11y agoa nice feature of this post would be the ability to click on images to hide them. Normally I have ublock to remove distracting elements.
- deleted 11y ago[deleted]
- hamburglar 11y agoNon Go programmer here. Can someone explain the initial goroutine leak that is being addressed? I don't see the issue.
- aardvark179 11y agoI'm not a go programmer, but think of it like this. You start a game, and that starts a goroutine that goes round in a loop getting scores from a channel. You have players which also have references to the channel and who put scores onto it. When all the players have left the only thing that has access to the channel is the game's goroutine. It's not consuming CPU itself because it's simply waiting for something to be put on its channel, but it does still have its stack and other resources, and it now has no way to exit. You can get this sort of resource leak in lots of ways in concurrent systems, and they all essentially boil down to the same thing, a thread or goroutine, or whatever, is waiting on a resource that nothing else has a reference to anymore, and there is no other way end it.
- r_sreeram 11y ago> Can someone explain the initial goroutine leak that is being addressed? The "for score := range g.scores {" loop runs forever, since nothing ever closes the g.scores channel. I.e., the "range" only terminates when the channel is explicitly closed. Even if there are no current senders on the channel, and even if nobody else holds a reference to the channel (and thus nobody else could potentially create a new sender and start sending on that channel), Go doesn't realize it (garbage collection doesn't help here). The "range" waits forever. Thus, all goroutines that run this code (via calls to NewGame(), via "go g.run()") will run forever, and leak, as long as something else in the program is running. When the rest of the program is done, Go will correctly detect that all these leaked goroutines are blocked and thus it's a deadlock, leading Go to panic ("no goroutines can proceed") and terminate.
- deleted 11y ago[deleted]
- divan 11y agoIt's not actually a leak. It's a program explicitly doing 'run goroutine and don't care of it anymore'. If the program logic wants this - it's ok. If author wants it to finish on some condition, but didn't write the condition code (like in this article) - it's a leak, but it's purely author's mistake.
- pklausler 11y agoChannels are great, but I prefer lazy lists.
- b169118 11y agoHow can I play the gifs again?
- deleted 11y ago[deleted]
- divan 11y agoI didn't get the point of example with Game and Player. The code behaves exactly how it's told to. If you need some logic to handle conditions where all players have been disconnected - you should implement it, no matter how. Maybe you want to wait for some time for a new players and teardown only after this timeout. Or, maybe, you want to reuse this game object, moving to some kind of pool (like sync.Pool). Or, perhaps, you really want to wait forever for returning players. It's not 'mutex vs channels' example in any way. It's not 'fix goroutine leak' it's "write the logic you want", it's that simple. Next, channels are slow, really? Send-receive operation on unbuffered channel typically takes around 300ns. Nanoseconds. 300 nanosecond in exchange of nice and safe way to express concurrent things - I wouldn't even call it a tradeoff. It's not slow at all in vast majority of cases. Of course, if you write that software that do care about nanoseconds and channels becomes your bottleneck - congratulations, you're doing great, and you probably have to switch to C++, Rust or even Assembler. But, please, don't mislead people telling them, that channels are slow. They could be slow for your exact case, but it's not the same. I don't really get the tone and arguments of the article. Some of the points are totally valid, but they easily falls into the 'hey folks, be careful about this small thing you may misunderstand at the beginning' category. Pity.
- dllthomas 11y agoI've done work where 300 nanoseconds is a noticeable chunk of my time budget... (not, of course, in go...)
- nemothekid 11y ago>* Of course, if you write that software that do care about nanoseconds and channels becomes your bottleneck - congratulations, you're doing great, and you probably have to switch to C++, Rust or even Assembler.* Thats ridiculous. I could switch my entire language... or I could just use a lock? First off, looking at Tyler's post, he measured unbuffered channels taking 2200ns vs 400ns for the lock solution - a 5x speed up. That is a large gain, especially when dealing a program that may have high lock contention. Switch from Go to C++ or Rust my not even gain you that much in terms of throughput - they are both compiled code and moving to either language will only mainly alleviate of magic STW pauses - acquiring a lock likely won't be any faster. Second, in the point of Game and Player, the logic to handle conditions where players disconnect is still simpler to implement with locks - its 2 lines, and there is no need to bring in sync.Pool, or introduce arbitrary timeouts. Channels are slower than locks. In more complex applications, channels are easier to reason about than locks, but those tends to be in cases where you care more about message passing than state synchronization.
- elcct 11y agoThere is probably a large number of developers who think "OMG my Go code doesn't have any channels and goroutines. Am I doing this right?" If you try to force a solution that isn't quite right for the given problem, then well, have fun. Case presented by the author I would naturally program with Mutexes, as me thinks using channels / goroutines is an overkill for this task.
- fpoling 11y agoThe article presents very similar arguments to those that I read in a book from 1982 or so. It discussed channels in Ada and pointed out that without super smart compilers that would turn channels into mutex operations the code using channels would be slower and more complex due to the need to create an extra threads. Base on that I can predict that in 2050 I will also read an article discussing channels in yet another language and advocating using mutexes instead...
- soroso 11y agoFrom http://go-proverbs.github.io/ http://go-proverbs.github.io/: Channels orchestrate; mutexes serialize.
- tptacek 11y agoYou can express unbounded buffered channels in Go straightforwardly with the stacked channel idiom.
- richard_todd 11y agoI enjoyed the article and nodded along as I read it. But after, I felt like it was overstating its case a little. It puts up a toy implementation that kinda works, and then explains that to make it act correctly in the real world you have to add uglier code. I can't really see blaming the language constructs for that... show me a language where that doesn't happen! I do appreciate that the article tries to deflate some of the hype about channels that you see when first investigating Go (I know I bought into it at first). After a little experience, I settled into a pattern of using channels for large-scale and orderly pipelines, and more typical mutexes and conditions for everything else. They have strengths and weaknesses, like all tools.
- f2f 11y agothe Clive[1] system uses a fork of the Go language which allows readers to close channels (I think it's the most significant difference between the languages, if not the only one): http://syssoftware.blogspot.com/2015/06/lsub-go-changes.html http://syssoftware.blogspot.com/2015/06/lsub-go-changes.html -- 1: http://lsub.org/ls/clive.html http://lsub.org/ls/clive.html
- _ph_ 11y agoI am not a Go veteran, but can see where this article is not helpful. Yes, channels are not a solve-everything. That is, why the Go library also contains mutexes etc. The game serving example could have been fixed by adding a channel to signalize that the game is finished. The game runner function should listen on the "scores" and the "done" channel with a select. Or, not use a channel at all. The channels are great, when you just want a completely safe method of communicating between goroutines, as long as the communication reasonably falls in the "streaming" behavior of the channel model.