8 ms·

Why Erlang?

156 points by hukl 14y ago

namidark 14y ago
I'm curious to know what type of CPU & Memory utilization there was on the ruby backends vs the erlang backends between the two games
- ConstantineXVI 14y ago
  He states that the Erlang server only needed a single box vs. the 80-200 for the Ruby; I'd wager the Erlang is dramatically lower in terms of resources consumed (or at least much more efficient)
Scorponok 14y ago
I've been curious about this for a while: If you can do concurrent processing so easily by message passing, why not just write Java or C++ that just passes messages around instead of sharing state? That way you don't have to learn a whole new programming language to do it.
- oconnore 14y ago
  Why not just whip up a simple method dispatch mechanism in C and skip Java/C++ entirely? edit: </internet sarcasm> ducks
  - Scorponok 14y ago
    Sure, if you like. I like (for example) the RAII and templating features of C++, but if you want to just use C go for it. I guess what I really mean is, what does Erlang offer beyond "concurrency is easy if you don't share state"?
    - rprospero 14y ago
      I believe the point that oconnore was trying to make is that asking what Erlang offer besides message passing concurrency is like asking what C++ offers beyond RAII and templates. You could add that sort of functionality to C, but it's not going to be as nice as just using C++. A lot of the things which make Erlang ugly are safe guards that prevent the kinds of bugs that allow you to accidentally share state. For instance, if I send a pointer between distributed systems, my program is hosed. Any distributed C++ library is going to either be extremely limited in the kinds of objects it can send or, more likely, just declare via fiat that it's the programmers responsibility not to send anything with a pointer. That's okay until the moment that you import a third party library and you have to go through every object to make sure that it doesn't use a pointer somewhere as a private variable. Meanwhile, with Erlang, there aren't any pointers, so I never even have to think about this stuff.
- my2cents 14y ago
  Yes, which one can I use?
- boofar 14y ago
  Keep in mind that in a language like erlang this is enforced for all it's users. So the whole eco-system is build around that premise.
- rudiger 14y ago
  See Akka: http://akka.io/ http://akka.io/
- ryanmolden 14y ago
  You can do that, but as far as I know Erlang (not far, mind you) it takes care of a ton of details for you. For instance to send a message to another actor, whether they are on you machine or a remote machine you just need a PID (process identifier, an Erlang id concept similar to, but not the same as an OS level PID). In C++ you would either need to deal with local/remote differences in message transport (ignoring the non-trivial nature of serializing arbitrary types in C++) or create an abstraction that would do so on your behalf. Second, in Erlang all actors have their own heaps, so their local data is spatially close in memory by definition making actors NUMA/cache friendly (assuming the runtime is "doing it right") heap allocated data passed between actors in C++ wouldn't have that attribute by default (again, you could make it so with some effort, but it certainly isn't free). Third actors in Erlang don't use full OS level threads, so the overhead of spawning say 10k of them is not the overhead of spawning 10k C++ threads (also ignoring that C++ really only acknowledged the existence of threads in C++11). These are a few issues/benefits, but as I said you could do all of this in any language mostly, but in Erlang a lot of the gross/tricky details are already handled for you.
  - jimbokun 14y ago
    In other words, it would require Greenspunning Erlang.
- luriel 14y ago
  Rob Pike and the rest of the (ex)Bell Labs/Unix gang have been on both sides of this for over 20 years: both with Squeak/Alef/Limbo, then switching to using C with libthread (http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread not to be confused with pthreads, for details see: http://swtch.com/~rsc/thread/ http://swtch.com/~rsc/thread/ ), but in the end having language support for concurrency is something that is really worth it, and one of the reasons the ended up building Go.
- trimbo 14y ago
  That's what I've done on maybe my last 5 projects. In C++, Java and Python. Not sure why this is so tough for people to understand. You usually don't need a whole new language to use a paradigm.
  - chc 14y ago
    You don't need anything but assembler. Yet you just named three languages and not a one was assembler. Once we've admitted that the niceties offered by more specialized languages are worthwhile, that "you don't need a new language" argument becomes a whole lot less compelling. For all you know, maybe programming this stuff in C++ is like programming sequential code in assembler — dismissing it just because the tool you have can be made to do the same thing isn't bad (I'd never disparage people who accomplish things), but it doesn't make a very compelling case for anything besides your own comfort zone.
    - trimbo 14y ago
      What you've just laid out is what I guess people call a "straw man". No one is arguing assembler over C here, just like no one is arguing the virtues of vacuum tubes over the transistor. I'll say it again: the benefits of the concurrency model described in the original article can probably be achieved in peoples' existing/preferred dev env. It's easier and faster to look into that then to throw what you have under the bus for erlang.
      - chc 14y ago
        If you want to get into logical parlance, it isn't a straw man, it's reductio ad absurdum. That is, you take somebody's reasoning and apply it to a different situation that illustrates the problems in the reasoning more clearly. In this case, your thinking appears to be, "I can accomplish the same thing in C++, so why use Erlang?" My point is that an assembly programmer might just as well say, "I can accomplish the same thing in assembly, so why use C++?" The answer is that it's a lot easier and more natural to write OO code in C++ than it is to use, say, assembler macros. The fact that something is possible is less interesting than how simple and well-supported it is. Again, I'm not saying your choice is necessarily wrong. For your use case, it might have been right. The benefits of using a language with pervasive and highly developed support for the paradigm might not outweigh the benefits of using C++ for you. But that reflects more on your personal circumstances than on the benefits of Erlang in general.
- sepeth 14y ago
  I guess it is easier deal with them and using them is more idiomatic in Erlang. For example you can write your own persistent data structures in Java and use them in your concurrent program. Actually you don't have to write your own, Clojure's persistent data structures written in Java and you can use them in Java, but these data structures makes much more sense with the rest of Clojure's tools, syntax, semantics...
- cageface 14y ago
  I heard Jonas Bonér (Akka guy) talk about his motivations for building Akka. He said he started the project because he fell in love with Erlang but couldn't convince enough companies to deploy it. So Akka is a port is the essential ideas to Scala and Java.
- monopede 14y ago
  You can build a library for doing Erlang-style programming, but there are lots of features that you can't reproduce. (1) In C++ you may get hard crashes, so to get reliability you need to use high-overhead OS processes. Erlang allows you to hundreds of thousands (yes, really) of processes because they are really lightweight (76 words + heap/stack). Also, copying within the same address space has much lower overhead (no system call/context switch) and serialisation is much simpler. (2) Because of immutable data you can do have nice fault tolerance as follows. Imagine you have a request handler function that takes a state and a request. However, the request is malformed or triggers a bug in the handler. In a language that doesn't enforce immutability you have to restart the handler process because the state may be in an inconsistent form. In Erlang you would just discard the request (or send an error message) and handle the next request with the unmodified state. (3) Pattern matching makes receiving a message very convenient. I probably forgot a few things, but this gives you an idea.
- nirvana 14y ago
  You could re-implement erlang in C++, but then you'd end up with erlang again. Why not just use erlang in the first place? I don't believe it is possible to do real concurrency on the JVM, without rewriting key parts of the JVM.
  - trimbo 14y ago
    > You could re-implement erlang in C++, but then you'd end up with erlang again. Why not just use erlang in the first place? Because rewriting or shimming the 10MM lines of code in the libraries one depends on outweighs this one particular benefit of erlang. > I don't believe it is possible to do real concurrency on the JVM, without rewriting key parts of the JVM Define "real concurrency".
    - nirvana 14y ago
      This one particular benefit is not available in any other language, unless you start with a relatively low level language like C and recreate it. Multi-threading is doable in any number of languages, but the problems inherent with it are why people move to concurrency. Concurrency needs to be supported in the language itself. You'd spend 20 years recreating this in C++ vs 2 weeks learning erlang.
- sparkie 14y ago
  The feature those languages are missing in order to do actors correctly, is enforced isolation of state. While you can emulate actors and message passing in nearly any language, you need this guarantee so that any actor can be executed anywhere at any time, without worrying that it is going to cause a race condition or hit a mutex somewhere. The big sin in Java and C++ which makes them quite unsuitable is "static" It might be possible to achieve that isolation if you stick to strict coding practices, but the point where it's going to fail is when you import another library, written by someone else. Because C++, Java and many other languages (including F#, Scala and others which have message passing libraries) do not have the enforced isolation of state, nor the ability to add notation to indicate that some piece of code is free of side effects (including all of it's dependencies), you're always going to have the risk of breakage when importing a library. The only way you can be sure that a library is actor-safe is to read it's source code - and if the source code is not available, then you're out of luck. Java and C++ could have the potential to do actors correctly by adding the notation for side-effect-free code, using custom annotations/attributes on all functions which are actor-safe, and using some compiler extension or static analysis tools to prove correctness. I'm not aware of anything that does this for the mentioned languages though. On the other hand, if you implement message passing in any purely functional programming language, your actors are automatically safe for free.
andyl 14y ago
Erlang is indeed impressive. But there is a big learning curve (at least for me!) and a whole new tool-set to support. It would be great if a Ruby/ZeroMQ based framework modelled on Erlang became widely adopted.
- clutchski 14y ago
  Not exactly what you're asking for, but check out Elixir, a Ruby inspired language that targets the Erlang VM: http://blog.plataformatec.com.br/2011/03/why-rubyists-should-try-elixir/ http://blog.plataformatec.com.br/2011/03/why-rubyists-should...
- nirvana 14y ago
  That wouldn't really work. You can't just bolt on concurrency support. IF you spend about 2 weeks working on erlang, it becomes really easy to use. The learning curve is not nearly as steep as it appears-- its just that the syntax initially looks so alien that its quite scary. I know the feeling, I went thru it, but it is well worth it.
- reddit_clone 14y ago
  Check out Akka Actor framework in scala. It does actors very well. Asynchronous and highly vertically scalable.
- masklinn 14y ago
  Most of the learning curve really is OTP. And as with Cocoa, the only way to skip those is to reinvent them, or not even reach an understanding of the problem they're solving. I'd expect an equivalent toolset on a different platform to have an equivalent learning curve if it existed at all.
- davidw 14y ago
  Big learning curve, pretty piecemeal support for lots of things compared to popular languages, and a mental model that is difficult for less advanced programmers spell something that will likely never be all that popular. Here's something I wrote about it 5 years ago: http://journal.dedasys.com/2007/09/22/erlang http://journal.dedasys.com/2007/09/22/erlang
luriel 14y ago
On a related note, I have heard of quite a few game companies that are using Go for their server side code.
- teamonkey 14y ago
  That's quite interesting. I'd like to know what kind of games. Care to name names?
  - Jare 14y ago
    The server code for AirMech by Carbon Games is written in Go: see http://carbongames.com/ http://carbongames.com/ Basically, a modern Herzog Zwei.
  - ZephyrP 14y ago
    I was recently talking with an Erlang developer working on the Battlestar Galactica MMO
  - zemo 14y ago
    not sure of specific games, but ngmoco has a Go stack they've open-sourced. They say they use it on their games, but I'm not sure which ones. https://github.com/ngmoco/falcore https://github.com/ngmoco/falcore
revertts 14y ago
What I gather from the slides: 1. server loads the data from S3 on start (sessions are sticky to a particular server) 2. user data gets mutated on that server for the lifetime of the session 3. the data gets written back to S3 on session end How are failures handled in 2? If a box goes down, do you not only lose the open connections but also any data that has been changed in that session? If these are game sessions it seems like they'll be long-lived; do you periodically dump back to S3?
halayli 14y ago
In lthread, a coroutine lib, (http://github.com/halayli/lthread http://github.com/halayli/lthread) you can make blocking calls inside coroutines. It gives you the lightness of event-driven, the advantages of threads, and the simplicity of sequential statements. And it can scale over cores.
- jerf 14y ago
  Remember, Erlang scales between machines. Erlang's actually all about reliability. It's the secret decoder ring to its design, not the actor model or message passing, which serve as the ways it gets to reliability. (Even to some extent its syntax. Excepting comma-period-semicolon which is just dumb.) And one of the principles of Erlang is that you don't have reliability when you are running on only one set of computer hardware.
  - halayli 14y ago
    lthread allows you to do so by message passing.
- DanielHimmelein 14y ago
  The problem with lthread is that it does not really provide blocking calls inside coroutines (you simply cannot do this with current OSes like Linux or Windows). Just take a look at the docs: If you need to execute an expensive computation or make a blocking call inside an lthread, you can surround the block of code with lthread_compute_begin() and lthread_compute_end(), which moves the lthread into an lthread_compute_scheduler that runs in its own pthread to avoid blocking other lthreads. lthread_compute_schedulers are created when needed and they die after 60 seconds of inactivity. lthread_compute_begin() tries to pick an already created and free lthread_compute_scheduler before it creates a new one. It just does the blocking call on its own pthread. That simply does not scale the way Erlang does. One day there is hopefully an OS where each file, socket, device, etc. is an actor with a message queue. Then you can really be concurrent :-). Even Erlang can then be more concurrent...
  - halayli 14y ago
ZephyrP 14y ago
I think this is a great article but misses some of what I think are the most essential points of what makes Erlang great. Superficially, Erlang is Actor Model for functionalists. I could go on about the various advantages this model provides, but as other posters have so eruditely pointed out, MPI provides the same abstract framework for dealing with the problems that actors are intended to solve. I'm not a professional erlang developer, but I've based and written a new NoSQL database thats intended to store billions of records in Erlang, altogether clocking in at less than 1000 lines of code, and after this experience, I can safely say that I will never try to build distributed apps, save some niche cases, in another language (even Haskell). Here's what stands out about Erlang to me, in contract to comparable options. Firstly, HiPE and BEAM may be the most advanced virtual machine systems in the world. People working with JVM will claim this an exaggeration, but in terms of reliability and speed, my experience with HiPE is unmatched. Secondly, and perhaps more importantly, OTP. Open Telephony Platform is the single best collected library for dealing with various problems that I have ever seen. The Generics (gen_server, gen_fsm, etc.) are a quick, fast and easy solution to problems and edge cases that would bog down even the best MPI developer. For example, dealing with internal state as a finite state machine is not new, but it's not nearly as popular as it should be, perhaps to this end it's apt to say that Erlang does for parallelism and distribution what Ruby on Rails did for web development -- It forces you to make intelligent decisions. The systems responsible for controlling emergency services, global telecom infrastructure, many large MMOs, and countless other applications are Erlang. If you trust 911, you can trust Erlang. I could go on, Hot code loading while old processes still depend on usable code? Built in failover? Function takeover? Automatic workload offloading? Good luck if it's not Erlang.
- talentdeficit 14y ago
  HiPE is the native code compiler, BEAM is the VM. otp is fantastic, however
- eternalban 14y ago
  Erlang is a CSP system, first and foremost, and not actors. [CSP style concurrency on JVM]: https://www.youtube.com/watch?v=37NaHRE0Sqw https://www.youtube.com/watch?v=37NaHRE0Sqw (Kilim, it should be noted, was benchmarked as scaling better than Erlang/OTP.) http://lambda-the-ultimate.org/node/4350 http://lambda-the-ultimate.org/node/4350 (paper in top comment is worth the read.) Disclaimer: I think Erlang/OTP rocks. But JVM has its moments, as well, and in fairness has earned the "most advanced VM" title.
silentbicycle 14y ago
A lot of intros to Erlang miss an important point: Erlang is about fault tolerance, not concurrency. The concurrency is a means to an end -- if your systems has to survive process crashes, hardware failures, and so on, it needs to have multiple components that can supervise and restart each other. Unlike (say) node.js, in Erlang, error handling is priority #1. Its ability to juggle lots of concurrent connections is just a consequence of its priority on localizing errors. Also, Erlang tries to address many failure cases that other languages/platforms do not; this isn't free. It's often a good trade-off (making systems resilient after the fact can be quite an undertaking), but worth noting.
boothead 14y ago
Can anyone comment on writing erlang style apps in Haskell? Haskell appears to have many of the required ingredients: Cheap multi threading, immutability and the Cloud Haskell[1] framework, with the added benefits of speed and the awesome type system. I currently use eventlet and ZeroMQ in python to emulate a similar (and probably crappier) style of writing message passing concurrency apps. With the addition of protocol buffers this type of architecture is easily applicable to a multi language set up. I'd be very interested to see what others are doing. Has anyone who's used erlang tried playing with Haskell in the same role? [1] https://github.com/jepst/CloudHaskell https://github.com/jepst/CloudHaskell
- jlouis 14y ago
  Yep, see my Combinatorrent and eTorrent projects, respectively. The long story short is here: http://jlouisramblings.blogspot.com/2010/04/haskell-vs-erlang-for-bittorent-clients.html http://jlouisramblings.blogspot.com/2010/04/haskell-vs-erlan...
  - boothead 14y ago
    Thanks for the link, I remember reading it a while back. Have you played with the cloud haskell stuff at all? I seems you were able to achieve most of what you wanted with STM channels for combinatorrent - do you think that the presence of cloud haskell might have made any difference?
    - jlouis 14y ago
      Cloud Haskell came after my work, so I do not know how it fares. I remember that I saw it and thought that there were neat ideas in there. My guess is that it would simplify the amount of work I had to do in code since CH would subsume many of my lines.
- dons 14y ago
  They're targetting slightly different points on the power/safety spectrum. * Erlang: fault tolerance, distributed systems, absolute performance less important. * Haskell: correctness important, multicore+shared memory performance a main focus, distributed systems less of a focus (cloud haskell) I wouldn't write my webserver in Erlang, for example, but my distributed object store, that's another story.
nirvana 14y ago
Frankly, I'm surprised (but I shouldn't be) that so many people are desperate for an alternative to Erlang. Going so far as to try to pretend that libraries bolted on to other languages give you the same thing at lower cost. This is impossible, and you'd understand that if you understood what erlang does. Stop that, right now. You're supposed to be hackers. New technology isn't supposed to scare you this much. Erlang doesn't use the C style syntax. BIG WHOOP. Yeah, it looks "weird" at first but don't be scared off by it. You can't get the fault tolerance (and consequential scalability) of erlang in any other language/platform. Period. Full Stop. It simply doesn't exist. Yeah, theoretically, one could build it with some low level language like C. You whipping up crappy C++ code using an "actor" library is not the same thing, by a long shot. So, unless you want to spend 20 years reinventing the wheel, why not just use the wheel. Yeah, I know its round and you're used to square. I promise you that you'll really enjoy the much smoother ride. Don't be a blub programmer. Learning erlang takes you about 2 weeks. The syntax is perfectly fathomable once you put in those two weeks. But be serious about it. Do a chapter each night from the armstrong book. This really is one of those languages that separates the hackers from the hacks. Not because its difficult, but because the hackers are not scared of it, while the hacks are. Seriously. Stop rationalizing.
- sparkie 14y ago
  While I wouldn't disagree with your points, I think you've chosen a fairly narrow view of reasons people might want an alternative to erlang. An obvious case you've omitted, is that people already have existing codebases written in other languages, and rewriting them all in erlang is not an option. Attaching an actor library to "X" language can give us many of the features offered by erlang with our existing codebase, and allow us to continue using our existing skills and paradigms. Erlang does an excellent job at solving certain issues, but it's lacking in many other areas that "X" might do a better job of. If my project needs both the fault tolerance and concurrency of erlang, and feature "A" which language "X" is good at, but erlang not so good at, then choosing erlang all the time might not be the right solution. I'm personally a fan of polyglot programming. We should pick the most suitable languages for each task, then we wouldn't need to keep re-inventing programming languages and libraries to emulate other languages. It's quite an idealistic view, but there's certainly ways we can improve the interoperability between languages, including erlang's - which is quite limited so far. An example of such approach is the (now discontinued) Axum project from Microsoft Research. It offered many of the ideas of Erlang, but built onto the .NET framework and largely compatible with existing code. Axum could call, and could even contain almost any C# code inside it, although it was a restricted version of C# which had isolated states, and no static variables - something that high standard codebases omit anyway, regardless of the whether language allows it, because it's been known good practice for a long time that shared state causes problems. The Actor Model existed long before Erlang, and will feature in many other languages other than it - whether or not they are successful is a different matter, but programming language designers would be wise to have a good knowledge of Erlang, and many other languages, so they're not re-inventing the wheel badly. Concurrency and fault tolerance should be considered from the start. I personally think we can do better than Erlang - or at least offer the same without sacrificing every other paradigm to achieve the aims of erlang. (And Haskell will probably be the language to better it).

156 points by hukl 14y ago

I'm curious to know what type of CPU & Memory utilization there was on the ruby backends vs the erlang backends between the two games

ConstantineXVI 14y ago
He states that the Erlang server only needed a single box vs. the 80-200 for the Ruby; I'd wager the Erlang is dramatically lower in terms of resources consumed (or at least much more efficient)

Scorponok 14y ago

I've been curious about this for a while: If you can do concurrent processing so easily by message passing, why not just write Java or C++ that just passes messages around instead of sharing state? That way you don't have to learn a whole new programming language to do it.

oconnore 14y ago
Why not just whip up a simple method dispatch mechanism in C and skip Java/C++ entirely? edit: </internet sarcasm> ducks
- Scorponok 14y ago
  Sure, if you like. I like (for example) the RAII and templating features of C++, but if you want to just use C go for it. I guess what I really mean is, what does Erlang offer beyond "concurrency is easy if you don't share state"?
  - rprospero 14y ago
    I believe the point that oconnore was trying to make is that asking what Erlang offer besides message passing concurrency is like asking what C++ offers beyond RAII and templates. You could add that sort of functionality to C, but it's not going to be as nice as just using C++. A lot of the things which make Erlang ugly are safe guards that prevent the kinds of bugs that allow you to accidentally share state. For instance, if I send a pointer between distributed systems, my program is hosed. Any distributed C++ library is going to either be extremely limited in the kinds of objects it can send or, more likely, just declare via fiat that it's the programmers responsibility not to send anything with a pointer. That's okay until the moment that you import a third party library and you have to go through every object to make sure that it doesn't use a pointer somewhere as a private variable. Meanwhile, with Erlang, there aren't any pointers, so I never even have to think about this stuff.
my2cents 14y ago
Yes, which one can I use?
boofar 14y ago
Keep in mind that in a language like erlang this is enforced for all it's users. So the whole eco-system is build around that premise.
rudiger 14y ago
See Akka: http://akka.io/ http://akka.io/
ryanmolden 14y ago
You can do that, but as far as I know Erlang (not far, mind you) it takes care of a ton of details for you. For instance to send a message to another actor, whether they are on you machine or a remote machine you just need a PID (process identifier, an Erlang id concept similar to, but not the same as an OS level PID). In C++ you would either need to deal with local/remote differences in message transport (ignoring the non-trivial nature of serializing arbitrary types in C++) or create an abstraction that would do so on your behalf. Second, in Erlang all actors have their own heaps, so their local data is spatially close in memory by definition making actors NUMA/cache friendly (assuming the runtime is "doing it right") heap allocated data passed between actors in C++ wouldn't have that attribute by default (again, you could make it so with some effort, but it certainly isn't free). Third actors in Erlang don't use full OS level threads, so the overhead of spawning say 10k of them is not the overhead of spawning 10k C++ threads (also ignoring that C++ really only acknowledged the existence of threads in C++11). These are a few issues/benefits, but as I said you could do all of this in any language mostly, but in Erlang a lot of the gross/tricky details are already handled for you.
- jimbokun 14y ago
  In other words, it would require Greenspunning Erlang.
luriel 14y ago
Rob Pike and the rest of the (ex)Bell Labs/Unix gang have been on both sides of this for over 20 years: both with Squeak/Alef/Limbo, then switching to using C with libthread (http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread not to be confused with pthreads, for details see: http://swtch.com/~rsc/thread/ http://swtch.com/~rsc/thread/ ), but in the end having language support for concurrency is something that is really worth it, and one of the reasons the ended up building Go.
trimbo 14y ago
That's what I've done on maybe my last 5 projects. In C++, Java and Python. Not sure why this is so tough for people to understand. You usually don't need a whole new language to use a paradigm.
- chc 14y ago
  You don't need anything but assembler. Yet you just named three languages and not a one was assembler. Once we've admitted that the niceties offered by more specialized languages are worthwhile, that "you don't need a new language" argument becomes a whole lot less compelling. For all you know, maybe programming this stuff in C++ is like programming sequential code in assembler — dismissing it just because the tool you have can be made to do the same thing isn't bad (I'd never disparage people who accomplish things), but it doesn't make a very compelling case for anything besides your own comfort zone.
  - trimbo 14y ago
    What you've just laid out is what I guess people call a "straw man". No one is arguing assembler over C here, just like no one is arguing the virtues of vacuum tubes over the transistor. I'll say it again: the benefits of the concurrency model described in the original article can probably be achieved in peoples' existing/preferred dev env. It's easier and faster to look into that then to throw what you have under the bus for erlang.
    - chc 14y ago
      If you want to get into logical parlance, it isn't a straw man, it's reductio ad absurdum. That is, you take somebody's reasoning and apply it to a different situation that illustrates the problems in the reasoning more clearly. In this case, your thinking appears to be, "I can accomplish the same thing in C++, so why use Erlang?" My point is that an assembly programmer might just as well say, "I can accomplish the same thing in assembly, so why use C++?" The answer is that it's a lot easier and more natural to write OO code in C++ than it is to use, say, assembler macros. The fact that something is possible is less interesting than how simple and well-supported it is. Again, I'm not saying your choice is necessarily wrong. For your use case, it might have been right. The benefits of using a language with pervasive and highly developed support for the paradigm might not outweigh the benefits of using C++ for you. But that reflects more on your personal circumstances than on the benefits of Erlang in general.
sepeth 14y ago
I guess it is easier deal with them and using them is more idiomatic in Erlang. For example you can write your own persistent data structures in Java and use them in your concurrent program. Actually you don't have to write your own, Clojure's persistent data structures written in Java and you can use them in Java, but these data structures makes much more sense with the rest of Clojure's tools, syntax, semantics...
cageface 14y ago
I heard Jonas Bonér (Akka guy) talk about his motivations for building Akka. He said he started the project because he fell in love with Erlang but couldn't convince enough companies to deploy it. So Akka is a port is the essential ideas to Scala and Java.
monopede 14y ago
You can build a library for doing Erlang-style programming, but there are lots of features that you can't reproduce. (1) In C++ you may get hard crashes, so to get reliability you need to use high-overhead OS processes. Erlang allows you to hundreds of thousands (yes, really) of processes because they are really lightweight (76 words + heap/stack). Also, copying within the same address space has much lower overhead (no system call/context switch) and serialisation is much simpler. (2) Because of immutable data you can do have nice fault tolerance as follows. Imagine you have a request handler function that takes a state and a request. However, the request is malformed or triggers a bug in the handler. In a language that doesn't enforce immutability you have to restart the handler process because the state may be in an inconsistent form. In Erlang you would just discard the request (or send an error message) and handle the next request with the unmodified state. (3) Pattern matching makes receiving a message very convenient. I probably forgot a few things, but this gives you an idea.
nirvana 14y ago
You could re-implement erlang in C++, but then you'd end up with erlang again. Why not just use erlang in the first place? I don't believe it is possible to do real concurrency on the JVM, without rewriting key parts of the JVM.
- trimbo 14y ago
  > You could re-implement erlang in C++, but then you'd end up with erlang again. Why not just use erlang in the first place? Because rewriting or shimming the 10MM lines of code in the libraries one depends on outweighs this one particular benefit of erlang. > I don't believe it is possible to do real concurrency on the JVM, without rewriting key parts of the JVM Define "real concurrency".
  - nirvana 14y ago
    This one particular benefit is not available in any other language, unless you start with a relatively low level language like C and recreate it. Multi-threading is doable in any number of languages, but the problems inherent with it are why people move to concurrency. Concurrency needs to be supported in the language itself. You'd spend 20 years recreating this in C++ vs 2 weeks learning erlang.
sparkie 14y ago
The feature those languages are missing in order to do actors correctly, is enforced isolation of state. While you can emulate actors and message passing in nearly any language, you need this guarantee so that any actor can be executed anywhere at any time, without worrying that it is going to cause a race condition or hit a mutex somewhere. The big sin in Java and C++ which makes them quite unsuitable is "static" It might be possible to achieve that isolation if you stick to strict coding practices, but the point where it's going to fail is when you import another library, written by someone else. Because C++, Java and many other languages (including F#, Scala and others which have message passing libraries) do not have the enforced isolation of state, nor the ability to add notation to indicate that some piece of code is free of side effects (including all of it's dependencies), you're always going to have the risk of breakage when importing a library. The only way you can be sure that a library is actor-safe is to read it's source code - and if the source code is not available, then you're out of luck. Java and C++ could have the potential to do actors correctly by adding the notation for side-effect-free code, using custom annotations/attributes on all functions which are actor-safe, and using some compiler extension or static analysis tools to prove correctness. I'm not aware of anything that does this for the mentioned languages though. On the other hand, if you implement message passing in any purely functional programming language, your actors are automatically safe for free.

andyl 14y ago

Erlang is indeed impressive. But there is a big learning curve (at least for me!) and a whole new tool-set to support. It would be great if a Ruby/ZeroMQ based framework modelled on Erlang became widely adopted.

clutchski 14y ago
Not exactly what you're asking for, but check out Elixir, a Ruby inspired language that targets the Erlang VM: http://blog.plataformatec.com.br/2011/03/why-rubyists-should-try-elixir/ http://blog.plataformatec.com.br/2011/03/why-rubyists-should...
nirvana 14y ago
That wouldn't really work. You can't just bolt on concurrency support. IF you spend about 2 weeks working on erlang, it becomes really easy to use. The learning curve is not nearly as steep as it appears-- its just that the syntax initially looks so alien that its quite scary. I know the feeling, I went thru it, but it is well worth it.
reddit_clone 14y ago
Check out Akka Actor framework in scala. It does actors very well. Asynchronous and highly vertically scalable.
masklinn 14y ago
Most of the learning curve really is OTP. And as with Cocoa, the only way to skip those is to reinvent them, or not even reach an understanding of the problem they're solving. I'd expect an equivalent toolset on a different platform to have an equivalent learning curve if it existed at all.
davidw 14y ago
Big learning curve, pretty piecemeal support for lots of things compared to popular languages, and a mental model that is difficult for less advanced programmers spell something that will likely never be all that popular. Here's something I wrote about it 5 years ago: http://journal.dedasys.com/2007/09/22/erlang http://journal.dedasys.com/2007/09/22/erlang

luriel 14y ago

On a related note, I have heard of quite a few game companies that are using Go for their server side code.

teamonkey 14y ago
That's quite interesting. I'd like to know what kind of games. Care to name names?
- Jare 14y ago
  The server code for AirMech by Carbon Games is written in Go: see http://carbongames.com/ http://carbongames.com/ Basically, a modern Herzog Zwei.
- ZephyrP 14y ago
  I was recently talking with an Erlang developer working on the Battlestar Galactica MMO
- zemo 14y ago
  not sure of specific games, but ngmoco has a Go stack they've open-sourced. They say they use it on their games, but I'm not sure which ones. https://github.com/ngmoco/falcore https://github.com/ngmoco/falcore

revertts 14y ago

What I gather from the slides: 1. server loads the data from S3 on start (sessions are sticky to a particular server) 2. user data gets mutated on that server for the lifetime of the session 3. the data gets written back to S3 on session end How are failures handled in 2? If a box goes down, do you not only lose the open connections but also any data that has been changed in that session? If these are game sessions it seems like they'll be long-lived; do you periodically dump back to S3?

halayli 14y ago

In lthread, a coroutine lib, (http://github.com/halayli/lthread http://github.com/halayli/lthread) you can make blocking calls inside coroutines. It gives you the lightness of event-driven, the advantages of threads, and the simplicity of sequential statements. And it can scale over cores.

jerf 14y ago
Remember, Erlang scales between machines. Erlang's actually all about reliability. It's the secret decoder ring to its design, not the actor model or message passing, which serve as the ways it gets to reliability. (Even to some extent its syntax. Excepting comma-period-semicolon which is just dumb.) And one of the principles of Erlang is that you don't have reliability when you are running on only one set of computer hardware.
- halayli 14y ago
  lthread allows you to do so by message passing.
DanielHimmelein 14y ago
The problem with lthread is that it does not really provide blocking calls inside coroutines (you simply cannot do this with current OSes like Linux or Windows). Just take a look at the docs: If you need to execute an expensive computation or make a blocking call inside an lthread, you can surround the block of code with lthread_compute_begin() and lthread_compute_end(), which moves the lthread into an lthread_compute_scheduler that runs in its own pthread to avoid blocking other lthreads. lthread_compute_schedulers are created when needed and they die after 60 seconds of inactivity. lthread_compute_begin() tries to pick an already created and free lthread_compute_scheduler before it creates a new one. It just does the blocking call on its own pthread. That simply does not scale the way Erlang does. One day there is hopefully an OS where each file, socket, device, etc. is an actor with a message queue. Then you can really be concurrent :-). Even Erlang can then be more concurrent...
- halayli 14y ago

ZephyrP 14y ago

I think this is a great article but misses some of what I think are the most essential points of what makes Erlang great. Superficially, Erlang is Actor Model for functionalists. I could go on about the various advantages this model provides, but as other posters have so eruditely pointed out, MPI provides the same abstract framework for dealing with the problems that actors are intended to solve. I'm not a professional erlang developer, but I've based and written a new NoSQL database thats intended to store billions of records in Erlang, altogether clocking in at less than 1000 lines of code, and after this experience, I can safely say that I will never try to build distributed apps, save some niche cases, in another language (even Haskell). Here's what stands out about Erlang to me, in contract to comparable options. Firstly, HiPE and BEAM may be the most advanced virtual machine systems in the world. People working with JVM will claim this an exaggeration, but in terms of reliability and speed, my experience with HiPE is unmatched. Secondly, and perhaps more importantly, OTP. Open Telephony Platform is the single best collected library for dealing with various problems that I have ever seen. The Generics (gen_server, gen_fsm, etc.) are a quick, fast and easy solution to problems and edge cases that would bog down even the best MPI developer. For example, dealing with internal state as a finite state machine is not new, but it's not nearly as popular as it should be, perhaps to this end it's apt to say that Erlang does for parallelism and distribution what Ruby on Rails did for web development -- It forces you to make intelligent decisions. The systems responsible for controlling emergency services, global telecom infrastructure, many large MMOs, and countless other applications are Erlang. If you trust 911, you can trust Erlang. I could go on, Hot code loading while old processes still depend on usable code? Built in failover? Function takeover? Automatic workload offloading? Good luck if it's not Erlang.

talentdeficit 14y ago
HiPE is the native code compiler, BEAM is the VM. otp is fantastic, however
eternalban 14y ago
Erlang is a CSP system, first and foremost, and not actors. [CSP style concurrency on JVM]: https://www.youtube.com/watch?v=37NaHRE0Sqw https://www.youtube.com/watch?v=37NaHRE0Sqw (Kilim, it should be noted, was benchmarked as scaling better than Erlang/OTP.) http://lambda-the-ultimate.org/node/4350 http://lambda-the-ultimate.org/node/4350 (paper in top comment is worth the read.) Disclaimer: I think Erlang/OTP rocks. But JVM has its moments, as well, and in fairness has earned the "most advanced VM" title.

silentbicycle 14y ago

A lot of intros to Erlang miss an important point: Erlang is about fault tolerance, not concurrency. The concurrency is a means to an end -- if your systems has to survive process crashes, hardware failures, and so on, it needs to have multiple components that can supervise and restart each other. Unlike (say) node.js, in Erlang, error handling is priority #1. Its ability to juggle lots of concurrent connections is just a consequence of its priority on localizing errors. Also, Erlang tries to address many failure cases that other languages/platforms do not; this isn't free. It's often a good trade-off (making systems resilient after the fact can be quite an undertaking), but worth noting.

boothead 14y ago

Can anyone comment on writing erlang style apps in Haskell? Haskell appears to have many of the required ingredients: Cheap multi threading, immutability and the Cloud Haskell[1] framework, with the added benefits of speed and the awesome type system. I currently use eventlet and ZeroMQ in python to emulate a similar (and probably crappier) style of writing message passing concurrency apps. With the addition of protocol buffers this type of architecture is easily applicable to a multi language set up. I'd be very interested to see what others are doing. Has anyone who's used erlang tried playing with Haskell in the same role? [1] https://github.com/jepst/CloudHaskell https://github.com/jepst/CloudHaskell

jlouis 14y ago
Yep, see my Combinatorrent and eTorrent projects, respectively. The long story short is here: http://jlouisramblings.blogspot.com/2010/04/haskell-vs-erlang-for-bittorent-clients.html http://jlouisramblings.blogspot.com/2010/04/haskell-vs-erlan...
- boothead 14y ago
  Thanks for the link, I remember reading it a while back. Have you played with the cloud haskell stuff at all? I seems you were able to achieve most of what you wanted with STM channels for combinatorrent - do you think that the presence of cloud haskell might have made any difference?
  - jlouis 14y ago
    Cloud Haskell came after my work, so I do not know how it fares. I remember that I saw it and thought that there were neat ideas in there. My guess is that it would simplify the amount of work I had to do in code since CH would subsume many of my lines.
dons 14y ago
They're targetting slightly different points on the power/safety spectrum. * Erlang: fault tolerance, distributed systems, absolute performance less important. * Haskell: correctness important, multicore+shared memory performance a main focus, distributed systems less of a focus (cloud haskell) I wouldn't write my webserver in Erlang, for example, but my distributed object store, that's another story.

nirvana 14y ago

Frankly, I'm surprised (but I shouldn't be) that so many people are desperate for an alternative to Erlang. Going so far as to try to pretend that libraries bolted on to other languages give you the same thing at lower cost. This is impossible, and you'd understand that if you understood what erlang does. Stop that, right now. You're supposed to be hackers. New technology isn't supposed to scare you this much. Erlang doesn't use the C style syntax. BIG WHOOP. Yeah, it looks "weird" at first but don't be scared off by it. You can't get the fault tolerance (and consequential scalability) of erlang in any other language/platform. Period. Full Stop. It simply doesn't exist. Yeah, theoretically, one could build it with some low level language like C. You whipping up crappy C++ code using an "actor" library is not the same thing, by a long shot. So, unless you want to spend 20 years reinventing the wheel, why not just use the wheel. Yeah, I know its round and you're used to square. I promise you that you'll really enjoy the much smoother ride. Don't be a blub programmer. Learning erlang takes you about 2 weeks. The syntax is perfectly fathomable once you put in those two weeks. But be serious about it. Do a chapter each night from the armstrong book. This really is one of those languages that separates the hackers from the hacks. Not because its difficult, but because the hackers are not scared of it, while the hacks are. Seriously. Stop rationalizing.

sparkie 14y ago
While I wouldn't disagree with your points, I think you've chosen a fairly narrow view of reasons people might want an alternative to erlang. An obvious case you've omitted, is that people already have existing codebases written in other languages, and rewriting them all in erlang is not an option. Attaching an actor library to "X" language can give us many of the features offered by erlang with our existing codebase, and allow us to continue using our existing skills and paradigms. Erlang does an excellent job at solving certain issues, but it's lacking in many other areas that "X" might do a better job of. If my project needs both the fault tolerance and concurrency of erlang, and feature "A" which language "X" is good at, but erlang not so good at, then choosing erlang all the time might not be the right solution. I'm personally a fan of polyglot programming. We should pick the most suitable languages for each task, then we wouldn't need to keep re-inventing programming languages and libraries to emulate other languages. It's quite an idealistic view, but there's certainly ways we can improve the interoperability between languages, including erlang's - which is quite limited so far. An example of such approach is the (now discontinued) Axum project from Microsoft Research. It offered many of the ideas of Erlang, but built onto the .NET framework and largely compatible with existing code. Axum could call, and could even contain almost any C# code inside it, although it was a restricted version of C# which had isolated states, and no static variables - something that high standard codebases omit anyway, regardless of the whether language allows it, because it's been known good practice for a long time that shared state causes problems. The Actor Model existed long before Erlang, and will feature in many other languages other than it - whether or not they are successful is a different matter, but programming language designers would be wise to have a good knowledge of Erlang, and many other languages, so they're not re-inventing the wheel badly. Concurrency and fault tolerance should be considered from the start. I personally think we can do better than Erlang - or at least offer the same without sacrificing every other paradigm to achieve the aims of erlang. (And Haskell will probably be the language to better it).