18 ms·
Go concurrency patterns
- AndreasFrom 14y agoOff topic: Why are their slides so unusable on iPad!? I can't swipe to the next slide without skipping that and 3 others.
- deleted 14y ago[deleted]
- jlarocco 14y agoThey're annoying in a regular desktop browser, too. I don't know why people feel they need to break regular scrolling for their slides. I'm sure it's interesting content, but I'll wait for a PDF version of the slides.
- 4ad 14y agoTell me when the PDF version has embedded, runnable code snippets.
- masklinn 14y ago> Tell me when the PDF version has embedded, runnable code snippets. When you're making the basic task of seeing the slides impossible, nobody cares about your pointless turd-polishing. Furthermore, PDF readers (at least Adobe's, but third-parties also sometimes do even if not all APIs are present) embed an ECMAScript interpreter, so you can indeed have embedded runnable code snippets in PDF: http://www.adobe.com/devnet/acrobat/javascript.html http://www.adobe.com/devnet/acrobat/javascript.html
- 4ad 14y ago> When you're making the basic task of seeing the slides impossible, nobody cares about your pointless turd-polishing. These are slides for the presenter. Nobody cares if you see them or not. Only the presenter has to care. > PDF readers [...] embed an ECMAScript interpreter So you wrote a Go to Javascript compiler? Can the VM running in the PDF reader implement a web server that can interact with the outside world?
- masklinn 14y ago> These are slides for the presenter. [...] Only the presenter has to care. Irrelevant, the whole thread of inquiry is about seeing the slides locally, you're two comments too late to try that one: you implicitly acknowledged the desire and desirability of personally viewing the slides in your previous comment. > So you wrote a Go to Javascript compiler? Also irrelevant (although there likely is no need to, there's an LLVM Go frontend and there's emscripten), the exact same method used here can be used there. > Can the VM running in the PDF reader implement a web server that can interact with the outside world? That doesn't even remotely make sense.
- jlgreco 14y agoI think you are getting a bit too aggressive about some slides with poor UI. You are asking a lot of the slide authors to minor facilitate secondary usage at the expense of primary usage. It could be a lot worse, they could be like 99% of the other slides in the industry: completely worthless without listening to the accompanying lecture.
- chimeracoder 14y ago> You are asking a lot of the slide authors to minor facilitate secondary usage at the expense of primary usage. I'm in a desktop browser, and I find this incredibly annoying to navigate. If that's not "primary usage" for viewing slides posted online, I don't know what is.
- burntsushi 14y ago> If that's not "primary usage" for viewing slides posted online, I don't know what is. Seriously? The primary usage is presentation..........
- jlgreco 14y agoPrimary considerations are centered around the presenter himself. He is the primary user of the slides, so whichever system he is most comfortable with will be the obvious choice. Asking that the presenter dedicate less time to preparation and more time to constructing some sort of elaborate LLVM/emscripten PDF presentation just for the benefit of secondary users who won't/can't use arrow keys is fairly unreasonable.
- duaneb 14y agoIt doesn't matter if they have runnable code if I can't get to it.
- gizzlon 14y agoOf course is does.. Most other people can get to it
- mseepgood 14y agoThe 'present' tool is open source, you can improve it if you have some JavaScript skills: http://code.google.com/p/go/source/browse/?repo=talks#hg%2Fpresent http://code.google.com/p/go/source/browse/?repo=talks#hg%2Fp... Here's the touch event handling code: http://code.google.com/p/go/source/browse/present/static/slides.js?repo=talks#259 http://code.google.com/p/go/source/browse/present/static/sli...
- enneff 14y agoYes, someone, please do this. I am the author of present, but I just pulled slides.js from another slide deck software. It's on my list to try to fix this, but I have little experience with touch devices. I'm sure someone with experience writing UI code for touch devices could make quick work of this.
- ominous_prime 14y agoHere's a video of the talk from Google I/O 2012: http://www.youtube.com/watch?v=f6kdp27TYZs http://www.youtube.com/watch?v=f6kdp27TYZs
- martinced 14y ago"You don't need to be an expert! Much nicer than dealing with the minutiae of parallelism (threads, semaphores, locks, barriers, etc.)" Great! But then, a few slides later: "Go has 'sync' and 'sync/atomic' packages that provides mutexes, condition variables, etc." : ( Can program written in Go still deadlock or not? I became a big of Clojure's no-brainer "no locks at all, hence no deadlock" approach btw.
- jamesmiller5 14y agoThe packages are available but you don't need to use them and are often encouraged not to. Yes, you can still have all goroutines asleep and the program will halt but that's due to improper concurrent design rather than a program implementation mistake.
- voidlogic 14y agoGo just makes things easy to get right, it doesn't make you get them right. Mutexes, condition vars, atomic operations on primitives have their place. That place is usually in scenarios where channels do not fit well. A program written in Go can deadlock, but the runtime detects deadlock and terminates the program, so at a system level, it does not deadlock. P.S. "No locks at all, hence no deadlock", is this not true, any system with blocking operations can deadlock. You can construct a Go program that only uses channels and get it to deadlock...
- ominous_prime 14y agoYes, you can still deadlock, but the runtime will usually panic saying that no goroutines can advance, which is definitely nicer than just hanging. If the core of your code is synchronized on channels, it becomes very easy to manage.
- kibwen 14y agoHow does the runtime detect this? I'd have suspected that such a thing would be undecidable, but I'm not at all a concurrency expert.
- 14y ago
- Ezra 14y agoVideo of rob giving the talk: www.youtube.com/watch?v=f6kdp27TYZs Also, this makes use of the excellent "Golang Present Tool", which makes most of the code on the slides executable.[1] [1]: http://godoc.org/code.google.com/p/go.talks/present http://godoc.org/code.google.com/p/go.talks/present
- sirclueless 14y agoHow do goroutines get cleaned up? For example, in many of these examples they create a goroutine that is supposed to send a message over a blocking channel eventually. What if I use their "select + time.After()" pattern and the timeout hits and I return. Does the goroutine hang forever? Does it constitute a memory leak? We both hold the channel, so it can't get garbage collected, is Go smart enough to know there aren't any more readers and the goroutine + channel can get cleaned up? Maybe to solve this I use a "quit" channel as they do. Do I need to worry about what happens if a quit message is never sent, or if two routines send one, or if the receiver is multiplexed and only one of many readers gets the message and cleans up correctly. It sounds like the whole malloc() and free() dance all over again, except this time I don't have any nice invariants to reason about such as "Everything that gets malloc()ed needs to be free()d, exactly once". Instead I need to worry that everyone is playing nice and will terminate when asked, and I need to remember to ask exactly once.
- ominous_prime 14y agoI'm not sure this is exactly what you're asking, but returning from a goroutine ends it. The rest is just garbage collected as needed. If you choose to select indefinitely, yes, you will need another channel to signal when you should return, or the goroutine will continue to run, and continue to hold a reference to that first channel.
- sirclueless 14y agoIt doesn't even require "selecting indefinitely." Every time you create a goroutine you need to worry about how it terminates, or you get a memory leak. If it uses channels, you need to make damn sure you perform all the proper operations on it, consume all the values from it eventually, etc. It's like manual memory management, except that the invariants are all implicit and poorly specified, like "Take at least 10 values from this channel, and I will terminate" or "Send me a value on this other channel over here and I will terminate". Sounds like a nightmare to reason about in some cases, which is a shame because Go does so much else Just Right^TM A concrete example, slide 34 [1] contains this code: func main() { c := boring("Joe") for { select { case s := <-c: fmt.Println(s) case <-time.After(1 * time.Second): fmt.Println("You're too slow.") return } } } This is code that looks really simple, but it's only correct because the process is known to terminate. You couldn't, for example, have the body of this function as a subroutine somewhere: every time you execute it a boring() goroutine is going to be left hanging, which is a classic memory leak. What's worse, "c" looks like an average variable I don't need to worry about (yay garbage collection!) but it's not because it escapes to a running goroutine. Correct me if I'm wrong but even if boring() was a much simpler function that returned after generating one value, if the timeout happened first it would be left hanging. In that case the only thing that would make the program correct is that of two racing goroutines, one is expected to return first. That's the kind of thing that keeps me up at night, especially when Go makes it so easy to treat goroutines and channels like every other GC-managed data structure when in fact they are every bit as leaky and difficult as threads. [1]: http://talks.golang.org/2012/concurrency.slide#34 http://talks.golang.org/2012/concurrency.slide#34
- jiggy2011 14y agoOk, I've never actually written anything in go so humour me here. The generally given reasons on HN that threads mapped 1:1 to pthreads are bad seem to be as follows: Shared mutable state is hard. Memory usage is inefficient when you are allocating a fixed size stack per thread and those threads spend most of their time blocked waiting for IO. Let's assume you are using some thread pooling pattern , so thread startup time is not such an issue. Apart from perhaps better syntax with channels etc, how does go fundamentally solve these problems in way that you cannot with standard threads? For example, shared mutable state problems can be mitigated to a degree by going down a "shared nothing" approach and handling shared state through some middleman like a Queue or a SQL/NoSQL DB. I assume go supports some form of pass by reference so you can still make shared mutable state an issue with it if you pass a reference to a goroutine. Each goroutine has it's own stack regardless of how many pthreads exist so there can still be wasted memory on a blocking operation.
- jerf 14y agoNothing stops you from doing anything you want with threads, including using them to implement very safe patterns. The problems are, 1: the thread libraries do not afford those safe patterns, and indeed, afford very unsafe ones (and also ones that compose poorly) 2: libraries written for the ecosystem will end up using the poor patterns 3: lack of compiler enforcement and how easy it is to accidentally mutate something unexpectedly mean that unless you are superhumanly careful you will still screw something up, somewhere. And you are correct that Go does not enforce shared-nothing between the goroutines. It has better affordances on those fronts that conventional C, but it is not enforced as it is in Erlang or Haskell. And while I say the affordances are "better", I still think that people screwing it up will be a practical problem.
- chubot 14y agoYeah, this is exactly right... I have to say Go changed my thinking about concurrency. But now I want to write a very small wrapper around pthreads that lets you write in the actor style. It just adds those "affordances". Go is more or less the actors style, except with the (discouraged) possibility of sharing mutable state... even though for some reason it doesn't seem to be advertised as such. The reason is that I don't think Go can cover what C + Python can. C gives you more low level control and Python is still shorter (and thus quicker). I like Go a lot but I would rather program in C + Python (like I do now) than C + Python + Go. And then the other component to this is de-emphasizing the somewhat-horrible-for-concurrency Python/C API. The library I'm talking about would have channels, and you would have one end open in Python, and one end open in C. Python and C are running in different threads. Rather than the crazy subroutine/callback hell you have now with any nontrivial Python/C binding. So basically I want to fix the C/Python interface, which is the only reason it is awkward to program in C + Python (the languages themselves are both great), rather than adding another language that overlaps highly with both of them. The OS is written in C, so you've never going to get past C. If there was a whole world written in Go, that might be reasonable... but I don't believe in portability layers.
- jws 14y agoSlide #33: Fan-in using select I always get alternating Joe/Ann responses. I'm not seeing that in the code though. It says selectors are chosen pseudo-randomly. I'm expecting Joe or Ann to get a couple quickies in at some point.
- jws 14y agoI seen the RNG get seeded in later examples. This may just be an unseeded RNG and an unfortunate sequence that gives an illusion of an order.
- Cyranix 14y agoIf you added a trivial wait of random length before each response, mightn't you see non-alternating behavior at times?
- gizzlon 14y agoNot GP, but I also saw this.. Increasing the timeout to 0><10 seconds does give the expected output.. But shouldn't I see this from time-to-time with 1 second as well? Hmm..
- gizzlon 14y agoMy rand.Intn() was always returning the same value. Seem like you have to to seed it manually (once, in main() for example): rand.Seed( time.Now().UTC().UnixNano()) http://stackoverflow.com/questions/12321133/golang-random-number-generator-how-to-seed-properly http://stackoverflow.com/questions/12321133/golang-random-nu... Edit: It's in the documentation, but you have to look for it. "Seed uses the provided seed value to initialize the generator to a deterministic state. If Seed is not called, the generator behaves as if seeded by Seed(1)"
- masklinn 14y ago> Rough analogy: writing to a file by name (process, Erlang) vs. writing to a file descriptor (channel, Go). That analogy doesn't really work, messages can be sent to an Erlang process by pid, how this pid came to be obtained isn't necessarily through naming (you can also register a process to a name, in which case it is indeed like writing to a file by name), and some of the things messages are sent to are not actual processes (e.g. ports).
- jlgreco 14y agoThe way I see it is that you write to files with an fd, but and fd can point to more than just files.
- masklinn 14y ago> some of the things messages are sent to are not actual processes (e.g. ports).
- jlgreco 14y agoAh, fair enough. I don't know erlang. Can a process have multiple distinct channels it receives on?
- masklinn 14y agoNope, each process has a single mailbox. On the other hand, processes don't have to go through their mailbox sequentially, they can prioritize messages (through pattern matching) and handle these first even if they are the last message in the mailbox (it's called a "selective receive"), so you get the same feature trivially by tagging messages instead of sending a message to a different channel.
- deleted 14y ago[deleted]
- RyanZAG 14y agoHow do goroutines compare to things like futures in eg. java? The ultimate structure seems much the same: start off a worker, and then you wait for the result with a timeout. As an example, the following code gives the same result: In Go: c := make(chan Result) go func() { c <- Web(query) } () go func() { c <- Image(query) } () go func() { c <- Video(query) } () for i := 0; i < 3; i++ { result := <-c results = append(results, result) } In Java: ExecutorService executorService = Executors.newCachedThreadPool(); Future[] futures = new Future[]{ executorService.submit(new Web(query)), executorService.submit(new Image(query)), executorService.submit(new Video(query)), }; for (int i = 0; i < 3; i++) { Result result = futures[i].get(); results.add(result); }
- newobj 14y agoFind an example that shows using a timeout channel along with another async activity on another channel, and then switching on that. Then you will start to see some elegance versus the Java way. (Sorry, in a hurry otherwise I'd provide the link myself :(
- laureny 14y agoPlease do show how to do this in Go when you have a few minutes, I'm really curious (and happy to write it in Java once I see what you mean).
- ciniglio 14y agoI think they mean something like: c = make(chan bool) go doSomeWork(c) select { case b := <- c: # do something here if something happened on the channel case <- time.After(5 * time.Second): # timed out :-( }
- RyanZAG 14y agoWell the timeout can be handled in java too Result result = future.get(5, TimeUnit.Second); if (result == null) // do something if timed out else // do something with result My original question was phrased badly though, and what I meant to ask was: is the underlying implementation faster? Since Golang is designed as a systems language, is the channel/gorouting system a lot more performant than the Executor/future method of Java? EDIT: I see what newobj is talking about now: With the Java method, the 'task queue' in this case is very rigid and would be hard to add futures from multiple locations, while the goroutine method allows easy access to add new messages to the channel from anywhere. You could probably emulate the goroutine method using a synchronized queue or similar, but the goroutine version handles it automatically.
- duaneb 14y agoCompletely off topic, but this is literally the worst presentation software I've ever seen and I hate it when it pops up. It didn't fit in my screen, so I zoomed out (or whatever the pinch movement does), at which point horizontal scrolling stopped working, even when I zoomed back in. IIRC it didn't work at all on the ipad. Is it really so hard to have forward/back buttons on the slide somewhere?
- kevingadd 14y agoGoogle presenters don't care about you unless you're running Chrome and have a keyboard. Pro-tip: If you load it with JS disabled, all the slides show up as plain text. That might let you at least consume the slides' content. EDIT: Pro-tip #2: It looks like if you tap the edge of the next/previous slide, it should scroll onto screen. Kind of tricky, though.
- smosher 14y agoThe phrase "concurrency patterns" tells me we are not yet talking about the concurrency-oriented language of the future. This is in the sense that patterns usually deal with the weak spots of the language, and lo, we must build weird constructs like doing fanout (not relevant to the original problem.) Go isn't alone here, in fact I see it every time someone thinks: "Erlang is nice, but gee, PIDs are so darn crude." In my mind, a language that makes concurrency the real priority (and I don't mean to say Go, Rust, et al. shouldn't be tempered by other concerns) will not require intermediate patterns to map the problem to the code. In other words, something close to a 1:1 correspondence should exist between problem concurrency and language support, including the nature of communications. I'd like to draw a comparison to memory management in C. It's a far simpler problem, but it's easy to illustrate that we don't (normally) need "allocation patterns" that go beyond the problem of memory needs. You can sum it up: "If you need memory, allocate it. Free it when you are done." The rules are simple and the advanced memory allocation techniques aren't required to map to the problem.
- 4ad 14y ago> patterns usually deal with the weak spots of the language Design patterns, as the term is usually understood, do. Using the word "pattern" to refer to organisational structures that arise in practice is useful and distinct than preaching design patterns like some people do for C++.