7 ms·
Concurrency Models: Go vs Erlang
- VeejayRampay 14y agoVery interesting article, thank you. On a slighly related note, you should fix the > / < in the code snippets :)
- PixelPusher 14y agoI haven't used Go yet, but love Erlang. Honestly can't imagine a better way to build fault tolerant applications without the help of a supervision tree, errors that bubble up, and fast process restarts. Would love to try out Scala or Go though, but Erlang has served me well so far. Not sure what they offer that's similar to OTP.
- alexatkeplar 14y agoCheck out Akka for Scala/Java - http://akka.io/ http://akka.io/ I am genuinely puzzled why people embark on building systems which they intend to be massively scalable using languages which don't have a technology like Erlang/OTP or Akka to support distribution across multiple boxes.
- drdaeman 14y agoThis. The fact that, as opposed to to Goroutines, Erlang processes can be transparently running on some other node is the most important difference.
- shinolajla1234 14y agoAkka does that as well, and you're right, it's very important for writing declarative, distributed logic. Akka's ActorRef abstracts over the physical location of the actor, just as Erlang's PIDs do.
- drdaeman 14y agoSo does Cloud Haskell (which borrows and improves on many ideas from Erlang/OTP). http://haskell-distributed.github.com/documentation.html http://haskell-distributed.github.com/documentation.html
- genwin 14y agoIn Go, channels can be between boxes, so a channel could be used to trigger or communicate with Go code running on another box.
- danieldk 14y agoOh, could you please show me some code that does this (a 'channel between boxes'), and supports sending of exactly the same data types that you send over a regular Go channel? Using the old deprecated netchan package doesn't count ;).
- genwin 14y agoI haven't done it myself. The idea is from the book The Go Programming Language Phrasebook. I've misplaced the book, but this link shows it's probably netchan: https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/A41IQ-7DqyI https://groups.google.com/forum/?fromgroups=#!topic/golang-n...
- granitepail 14y agoThere's gob ( http://golang.org/pkg/encoding/gob/ http://golang.org/pkg/encoding/gob/ ). Not going to claim it's on the level of node communication in Erlang, but it does make data (de)serializaiton incredibly simple. With only a few lines of code you could have a channel reading from and writing to a socket to another machine.
- rdtsc 14y agoThere is nothing like the pleasure of using the right tool for the right job. Using C++, Java or even Python for highly concurrent and fault tolerant systems is a bit like hammering screws into the wall. It can be done with enough effort -- but try a screw driver drill and see what a difference it makes. The secret sauce is all about fault tolerance, everything else amazingly and logically leads from it: isolation, hot code reloading, distribution (running nodes on multiple machines)
- xradionut 14y agoYou can use Python on top of Erlang.
- mseepgood 14y agoA supervisor is easily implemented in Go: http://code.google.com/p/gosup/source/browse/supervisor/supervisor.go http://code.google.com/p/gosup/source/browse/supervisor/supe...
- jerf 14y agoErlang's error checking model is a great deal more like Go's than he thinks. Erlang is in the "exceptions are exceptional" camp too, and idiomatic code should not be throwing exceptions around willy-nilly. Go is noticably more fragile with errors. Unhandled exceptions (which are just a fact of life unless you're a perfect programmer) will result in the entire program terminating if you don't have something that handles it. This behavior is forced on it precisely because of the shared memory model (one of the actual big differences); if one goroutine has f'ed up, you simply don't know what the state of your program is anymore. (Theoretically you could do better than that, but not simply.) Since Erlang memory is isolated, it can kill just that one process, and the other processes can pick up the pieces. (Not necessarily perfectly or without loss, but in practice, really quite well.) Consequently, for any serious Go program, you're still going to have to choose an exception handling policy, it's not as if it has gotten away from exceptions. Failures are a fact of life... for all you know, memory was corrupted. Again, the difference here is not "error handling policy" but the longer term consequences of shared vs. isolated memory spaces. If you just type up idiomatic Erlang OTP code, you have to go out of your way to not have a bulletproof server; if you just type up idiomatic Go code it's on you to make sure you're not excessively sharing state and that you aren't going to see your entire server doing tens of thousands of things come down due to one unhandled exception. Go programmers need to be more worried about error handling in practice than Erlang programmers, since Erlang programmers aren't facing the termination of the entire world if they screw up one thing. There's also a recurring pattern in newer language advocates in which they will in one year claim it's a good thing that they don't have X, and next year tout to the high heavens how wonderful it is that they just implemented X. I went around with the Haskell world on a couple of those issues ("no, strings are not linked lists of numbers", "yes they are you're just not thinking functionally and inductively dude, and by the way, six months later, check out this totally radical ByteString library, and when that still turns out not to be stringy enough hey, check out Data.Text six months later..."). Thinking you can get away without OTP is likely to be one of those for Go. No. You need it, though I have questions about whether it can even be built in Go, because one of the other actual differences between the languages... ... which is Channels vs. Processes. Go has channels, but you can't tell who or what has them, and there's no such thing as a "goroutine reference". By contrast, Erlang has processes, but no way to tell what messages they may send or receive, and there's no such thing as a "channel". Again, this has major impacts on how the system is structured, in particular because it is meaningful to talk about how to restart a "process" in a way that it is not meaningful to talk about how to restart a "channel". Go advocates desperately, desperately need to avoid the temptation to explain to themselves why Erlang isn't as good, because then they'll fail to learn the lessons that Erlang learned the easy way. There's a lot of ways to improve on Erlang, but let me tell you that despite those opportunities, Erlang as a language is one of the wisest languages around, you do not want to try to exceed it by starting from scratch. Learn from it, please, I want a better Erlang than Erlang, but talking yourself into how Go is already better than Erlang isn't going to get you there.
- gordonguthrie 14y agoFew quick points Erlang doesn't have threads - it has processes. Thread's share state, processes don't. The processes are created within the Erlang cluster - they can't be operating system processes because they might be running on a different instance of the operating system. Try and Catch are actually very rare constructs in Erlang - usually only at the boundaries of the system with the rest of the world. In my 10 years experience there is about 1 try/catch per 20k - 25k lines of production code. Erlang doesn't handle errors - you let your process crash and let OTP restart it. The article seriously underestimates the importance of OTP. An Operating System is a set of libraries that means your unwritten application can do things like write to persistent storage, speak to a network, run in a cronjob, have a GUI, etc, etc... To have a reliable computer system you need at least 2 computers and OTP is an Application System than runs across multiple physical boxes. This means that your unwritten software can failover if a box dies, recover in the presence of gross errors, cluster up, etc, etc... (Google App Engine is another example of an A/S). The first cluster we ever deployed was a shambles. We moved a webserver from a London box to a Californian one and its performance went through the floor. Because of a typo we had actually only moved the web bit leaving the database running in London.
- rdtsc 14y ago> Thread's share state, processes don't. Not only that they also have separate heaps -- truly concurrent GC is possible without too much "clever" code.
- chimeracoder 14y ago> In Go, you handle errors, and hence you manage everything about your goroutines and in-process servers. In Erlang, the language handles errors, and you have to manage the consequences of those errors. The Go implementation is more straightforward, while the Erlang implementation involves more callbacks and indirection. I haven't used Erlang, but that sums up my experience with Go very well. Coming from Python, the hardest thing for me to adjust to was not the concurrency model (which is incredibly straightforward), but the error handling. That said, the adjustment was worth the effort. I've come to dislike the idea of my functions failing (panic()nig) because of some nested function call four, five, six levels deep. Returning error codes makes the error handling explicit, and because the compiler complains about unused lvalues, it makes it easy to spot code where the errors are ignored (just look for underscores)[0]. Contrast to Python, where it's almost impossible to tell whether or not I need to wrap a given call in a try/except block without digging deeper into the code. It makes writing concurrent code much more logical, because it couples the error handling more closely with the spot at which the error occurs, while at the same time giving the option to let errors 'bubble up' as needed. It's not quite the same way that Lisp handles conditions so, so elegantly[1], but it's about the closest I'd expect in a C-style language. [0] Or you could not assign the single return value of a function that returns an error, but since pretty much everything returns an error (whether or not you pay attention to it), 'naked' function calls are just as suspicious. [1] http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html http://www.gigamonkeys.com/book/beyond-exception-handling-co...
- masklinn 14y ago> I haven't used Erlang Don't take anything he said about error handling in Erlang at face value, because very little applies to idiomatic erlang. Basically the only thing which is correct is: > In Erlang, it is idiomatic to let your functions fail And even that has to be stretched: in Erlang, it is idiomatic to let your process fail (throwing exceptions is rare, catching them is rarer still, the average Erlang program will likely do neither "procedurally"). Because idiomatic erlang separates processing an error recovery, and an other process will handle error recovery for the failing one. Furthermore he conflates error handling and failing, when Erlang very much separates them (error handling is done via return values, in a terser yet more explicit way than Go) The rest is worse.
- douglasisshiny 14y agoI would love to see a comparison between Erlang and Clojure. I do Rails stuff but eventually want to try a functional language in which concurrency isn't an afterthought. I know Erlang achieves concurrency through message passing / message queues, but that's about it. And I have only watched a handful of Rich Hickey talks and have read some introductory material, but I understand that concurrency is achieved through the STM with refs, agents and atoms. I'm leaning towards Clojure now (partially because I want to read SICP), but at the very least a comparison between Erlang and Clojure would be very interesting (not just limited to concurrency). Edit: I have thought about Go lang, too, but Rich Hickey's talks about functional programming (especially immutability) have won me over.
- stonemetal 14y agoIt is closer to the Go end of the spectrum. It provides agents, they are an asynchronous message handler kind of similar to a goroutine or an Erlang process. When an agent gets an uncaught exception it caches it, then any subsequent interactions will immediately throw an exception, until the agent's errors are cleared.(http://clojure.org/agents http://clojure.org/agents) Which sounds kind of reasonable but I could see it causing problems as you start to see exceptions thrown by code that had no hand in creating the problem and may not know how to fix it.
- reddit_clone 14y agoThe closest thing to Erlang/Otp is (Scala/)Akka. Akka is a library modeled pretty much 1-1 after Erlang/Otp. As a language Scala supports both functional and imperative programming. I recommend you check it out too.
- shinolajla1234 14y agoThat's not entirely true. Erlang OTP allows for selective receive, and will stash unhandled messages to the side. When a received message is handled, all unhandled messages are retried because the actor may have changed what messages it can handle as a result. This is fine in Erlang - if the actor process crashes for exceeding allocated memory, it has no impact on other actors. The original Scala Actors did this as well, with predictable results. Scala runs on the JVM, which is a monolithic process. The usage of selective receive meant that over time, actors would accumulate enough unhandled messages to result in a possible OutOfMemoryError, from which there is no recovery on the JVM. Akka does not do selective receive, so it does not entirely follow OTP. That said, it is inspired by OTP, as evidenced by the original name of the project, Scala OTP. It's just optimized for the JVM.
- eternalban 14y agohttp://erlang.org/pipermail/erlang-questions/2003-February/007355.html http://erlang.org/pipermail/erlang-questions/2003-February/0...
- puzza007 14y agoHere's a good description of how Erlang does scheduling http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-scheduling.html http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-...
- jeremyjh 14y agoIdiomatic Erlang actually uses multiple return values (in a tuple) in a way that seems similar to Go to me. For example an attempt to open a database connection might return either {ok, Conn} or {error, Reason}. If you have a match for the error then you can handle it right there. If you don't have a match, yes its a runtime exception and OTP will handle it for you. You can also use exceptions for control-flow but that is less idiomatic. Just because Go doesn't offer you supervision trees doesn't mean they wouldn't be helpful. In fact I'll be surprised if there isn't a decent library for them in a couple of years.
- gordonguthrie 14y ago> Just because Go doesn't offer you supervision trees doesn't mean they wouldn't be helpful. In fact I'll be surprised if there isn't a decent library for them in a couple of years. Supervision trees only work because the worker processes they supervise are supervisable. You can't just bolt them on. If you are building things in threads (ie with shared state) then 'supervision' consists of closing them ALL on any error in ONE and restarting. Not so useful
- dannywoodz 14y ago> For instance, what if there are a pair of processes that work in unison and one of them dies? That would depend upon whether the programmer linked them or not. If they're linked, the exception propagates and potentially kills the peer process. If they're nodes in a supervision tree, it's up to the supervisor as to whether to: respawn the dead process; kill the peer and then restart them both; or even to kill the peer and die itself, passing the error up the tree. It's a matter of resource cleanup and consistency. Not so much what happens if one of the processes dies, but rather what happens if only one of them does?