4 ms·
Ugh, I just finished writing the XMPP frontend for an XMPP/IRC bot I'm working on (http://www.getinstabot.com http://www.getinstabot.com). The frontends are in
by StavrosK 14y ago
Ugh, I just finished writing the XMPP frontend for an XMPP/IRC bot I'm working on (http://www.getinstabot.com http://www.getinstabot.com). The frontends are in Go for concurrency, and ferry messages back and forth from the channels to the backend.
Let me tell you, that problem is hard. Go coped pretty well, but the final thing is a mess of global states, and it's pretty elegant for what the problem is. I was hoping to avoid having many moving parts, but it ended up needing a lot of shared state between all processes.
Some problems are just hard, and, no matter how well-designed the language is, they'll still be hard. Something I miss from the language after implementing that is the ability to, say, monitor one goroutine from another to see if it returns (so the former can return as well). I know it's possible with channels, but when one goroutine is blocked on network Recv(), there's not much of a chance to listen to channels.
Anyway, yes, parallelism in Go is great, but not everything is magically all rainbows and unicorns (that's Django). Some problems will be messy and dirty, and even more so when you use channels.
- burntsushi 14y ago> I know it's possible with channels, but when one goroutine is blocked on network Recv(), there's not much of a chance to listen to channels. Could you elaborate a bit more on this point? This seems like a natural pattern for a channel/goroutine. Spin up a goroutine that reads from a connection and send whatever is read on a channel. Then other goroutines can synchronize on the channel.
- StavrosK 14y agoSure, but what if you want to shut that goroutine down later on? The best solution that I've found is to just close the connection, the .Recv() will error out immediately.
- lobster_johnson 14y agoYes, that is afaik the only way to abort a synchronous function call such as TCPConn.Read().
- burntsushi 14y agoWhat other solution would you want? You could use `SetDeadline` if you like. But closing the connection seems reasonable too. I think you're framing the problem wrong. You're not asking to shutdown a goroutine, you're asking "How do I abort from a synchronous read from a network connection that is blocked?"
- StavrosK 14y agoDon't get too hung up on that, it was the least of the problems. Closing the connection works fine, it's just a bit unorthodox. The basic structure of the problem was much more problematic to implement.
- pandemicsyn 14y agoheads up. Seems like https://getinstabot.com https://getinstabot.com and http://www.getinstabot.com http://www.getinstabot.com end up in a redirect loop.
- StavrosK 14y agoOh. I forgot about this comment and picked a great time to redirect everything over https. Deploying the fix now, thanks.
- NyxWulf 14y agoObserving other channels, or seeing when a channel does fire is what the select structure is for. It is one of the features that makes channels and goroutines very nice in go. http://golang.org/ref/spec#Select_statements http://golang.org/ref/spec#Select_statements http://blog.golang.org/2010/09/go-concurrency-patterns-timing-out-and.html http://blog.golang.org/2010/09/go-concurrency-patterns-timin... http://talks.golang.org/2012/concurrency.slide#1 http://talks.golang.org/2012/concurrency.slide#1 Video for the slides: http://www.youtube.com/watch?v=f6kdp27TYZs http://www.youtube.com/watch?v=f6kdp27TYZs
- lobster_johnson 14y ago> Something I miss from the language … Does sound like you want to use channels, and break your logic into small independent parts. You will have one goroutine using blocking Read() in a loop and feeding data to some channel. When it's done, you write to another channel that exists only for signaling: defer { doneChannel <- true } for { data := make([]byte, 65535) _, err := conn.Read(data) if err != nil { errorChannel <- err break; } dataChannel <- data } and then in your other goroutine: for { select { case data := <- dataChannel: // Handle data case <- doneChannel: // Other goroutine is done } } The only things shared here are the channels. The way to avoid too much state and moving parts is to break the problem into isolated, manageable parts that communicate with channels. Often you will have hierarchical relationships like this, where one piece of dumb code exists to pass data from something lower down to somewhere higher up. It's not hard, although some of the code gets a bit ugly and disjoint at times, especially in how anything synchronous has to use channels and goroutines. For example, today I wrote a simple worker pool implementation that runs a given function in parallel via goroutines and can adjust the number of workers dynamically at runtime. That function has to be declared not just as "func()" but as "func(abortChannel chan bool)", and the worker function has to honour the abort signal when it arrives from the pool. So channels do leak everywhere, even into APIs. (Yeah, I know I can use "chan struct{}" to avoid any storage, but I think "<- true" looks nicer than "<- struct{} {}".) What is harder is to intelligently handle complex cascading failures. That's what Erlang, with its supervisor tree, is good at. Go's goroutines are "fire and forget" and cannot even be terminated programmatically from elsewhere in the program.
- StavrosK 14y agoYeah, the channel approach was rewrite #3 :P It required me to have shared state in a horrible way as well, ruining the whole thing. There really isn't an elegant/functional way to do it, and sharing channels is error-prone itself. I ended up closing the connection to terminate goroutine B from A, and messaging on an already-shared channel to terminate A from B. Having cascading failures in Go would be fantastic. The problem, unfortunately, is not very amenable to elegant solutions. I might do a writeup at some point.
- 14y ago