15 ms·

What's all this fuss about Erlang? (2007)

231 points by krat0sprakhar 10y ago

chrisseaton 10y ago
> Your Erlang program should just run N times faster on an N core processor But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever. If your program has sequential or less parallel phases or needs to communicate then you are subject to Amdahl's law like you always were. Armstrong has that claim in the Erlang book and I was gobsmacked to see it written down with no caveat or mention of the limits from Amdahl's law whatsoever. I was sure it was a joke and it would be followed by 'ha ha ha of course not - nobody knows how to achieve that despite decades of intensive research', but no it's a serious claim made with a straight face. Erlang's solved it! Erlang helps you write parallel programs... but only if you program is entirely parallel in the first place.
- geodel 10y ago
  I searched many times the comparison of Erlang BEAM and JVM regarding performance but couldn't find much. May be it is not done, or maybe it is not publicly available. At my work some folks keep pushing RabbitMQ over ActiveMQ for no other good reason than it is in Erlang so we will get much better performance. I know it is not true but could not find web links to support that.
  - novembermike 10y ago
    BEAM is slower than the JVM for sequential code but it doesn't have a block the world GC and it has preemptive multi processing. It'll have worse steady state performance but a lot of the worst cases that Java can have are heavily mitigated.
    - jerf 10y ago
      It's also realllly good at routing; the control logic is a somewhat slow more-or-less interpreted language speed but then the act of routing bytes around runs at native speeds (as long as you keep things in binaries and don't convert them to "strings"), with the Erlang framework making sure by the nature of how it works that there's no contention or anything getting in the way because everything's immutable. It's sort of like how people talk about how NumPy can be so useful; Python is itself one of the slowest languages (especially since when it comes to NumPy we can't necessarily count on PyPy, though that's coming along) but it doesn't matter because you just use the slow language to script up the computation you want to do, which then goes and runs in highly-optimized assembler implementations. You can look at Erlang being very similar when being used in its native domain for networking, using a slower language to decide how to route bytes around. So it's quite possible for competent Erlang to beat out competent Java for such a task. Java optimized to within an inch of its life can certainly beat Erlang, but it's going to be a much uglier code base to get there.
    - pjmlp 10y ago
      You are making the error of comparing BEAM with a specific JVM implementation. Not all available JVMs on the market have "a block the world GC" and the way multi-processing is done is implementation defined.
      - novembermike 10y ago
        I could be wrong but I was under the impression that the four major JVM gc's all had a stop the world phase and while multi-processing is implementation specific I'm not aware of a robust pre-emptive multiprocessing model built for the JVM. If you have robust examples that are contrary to this I'd be interested in hearing about them.
        pdpi 10y ago
        Azul doesn't. It's not a differnt GC for the hotspot VM, it's a separate VM altogether
- floatboth 10y ago
  Obviously said "program" must be a server that services independent requests. I mean, not obviously, but if you work in telecom, web, etc. that's pretty much all programs you see.
  - chrisseaton 10y ago
    I get that, but as I say we already knew how to scale those linearly in most programming languages. Erlang brought nothing new there. It does have advances in things like reloading and supervision.
    - mnd 10y ago
      Erlang was invented 30 years ago to solve problems that were not easily solvable by the back then prevalent programming languages. The claim that Erlang hasn’t brought anything new to the table couldn’t be further from the truth. Designing concurrent, fault-tolerant programs was not a problem solved in a satisfying way by the programming languages and technologies that were mainstream back then, which is exactly why Ericsson pursued the path of inventing their own programming language. I suggest everyone to read Joe Armstrong’s paper “Making reliable distributed systems in the presence of software errors”. It’s easy to read and mind expanding even if you do not intend to use Erlang. It can also save one from public embarrassment when one claims ridiculous things as parent and many other remarks in the other comments.
      - chrisseaton 10y ago
        I acknowledged the areas where Erlang did bring advancements such as fault tolerance. I read that paper, more than once, and his thesis, when I was doing my own PhD on parallelism, so I know the area and related work fairly well. I don't think I'm embarrassing myself when I say that it's silly to claim that an arbitrary Erlang program should achieve linear scalability. But we don't need to debate with vague claims to originality - as I say Amdahl has done the maths and shows that it is only possible for a program that is embarrassingly parallel.
        10y ago
- timClicks 10y ago
  That's true, but when I read that sentence I place the emphasis on should. I see it as aspirational. A better way to phrase this would perhaps would be to say that perfectly preemptive scheduler should be able to keep all cores hot. Blocking of one Erlang process should not halt the program's progress. As Erlang requires message passing overhead by design, it will never see perfectly linear scaling behaviour. Still, its scheduler, its GC model and M:N threading implementation are extremely robust and can be relied upon to write extremely reliable networked services.
  - nextos 10y ago
    I also interpret it this way. If you recall the famous Wide Finder benchmark for parallel programming by Tim Bray, back in 2007 as well [1], those were essentially the results. Languages like Erlang or JoCaml (Ocaml plus joint calculus), whose models are inherently parallel yield simple and fast solutions. Other languages too, but they require a lot of expert optimization and ugly code. [1] https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Find...
  - chrisseaton 10y ago
    > That's true, but when I read that sentence I place the emphasis on should. I see it as aspirational. Despite all the arguing I'm taking part in, this is probably the true and reasonable explanation. I read 'should' as 'if it doesn't there's a problem'.
    - inopinatus 10y ago
      Yes, indeed, don't forget his very next line is "Is this true? Almost. It’s early still, but we’re optimistic (Wildly optimistic; I haven’t seen this kind of optimism for the last 20 years!)."
  - amelius 10y ago
    > As Erlang requires message passing overhead by design That's Erlang's main weakness. For parallel code, structurally shared data is immensely useful for processes running in the same address space. With Erlang, you have to give that up. OTOH, not having this possibility makes distributing processes to other nodes a lot simpler.
- zzzcpan 10y ago
  > If you have one of those And how would you have one of those? Erlang encourages you to design and think about programs in a certain way, that typical C, Java or Python does not. I've seen Armstrong in one of his talks to actually emphasize the importance of learning distributed systems if you wanted Erlang to be useful for you. That's way beyond "embarrassingly parallel" point of view, Amdahl's law has no use here.
- devishard 10y ago
  >> Your Erlang program should just run N times faster on an N core processor > But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever. You've made two claims, one irrelevant and one false: 1. only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized. 2. it's trivial to write a program that's faster in X language I'm not sure how toot define trivial, but I've yet to find a language that can communicate between threads as performant-ly. Even languages like Clojure which use similar thread semantics can't do what Erlang can because the underlying threads aren't as lightweight. Spinning up a million threads in Erlang isn't even unusual, whereas in any of the languages you mention it's either crippling-ly slow (Python or Java) or very difficult to synchronize (C most, but Python and Java aren't easy).
  - BeetleB 10y ago
    >only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized. OK. Not an Erlang user, but I can't let this statement go. I've studied parallel numerical algorithms. Many/most of them will involve blocking because you're waiting for results from other nodes. If you're saying Erlang has somehow found a way to do those numerical algorithms without having to wait, then I'd love to see all those textbooks rewritten. Amdahl's Law reigns supreme.
    - aeturnum 10y ago
      I'm not devishard, but I parsed his statement slightly differently. He's not saying that Erlang makes things parallel magically. Rather, he's saying that Erlang forces tasks that /could/ be parallel to be parallel by default. Thus, Erlang will tend to maximize the sections of your program that are run in parallel compared to other languages.
- di4na 10y ago
  You mistake concurrency with parallelism. If each of your "synchronous" call to a method is an asynchronous message, suddenly, all your program is embarasingly concurrent. Wait... was that what Alan Kay was saying with SmallTalk ? Would we have all lost our mind on RPC and synchronous calls for so long? Yes. Yes we did.
  - ww520 10y ago
    "Embarrassingly concurrent" is not embarrassingly parallel. Parallelism is not merely making things concurrent nor making synchronous calls into concurrent async calls.
    - di4na 10y ago
      once you are concurrent, what stop you from being distributed to multiple cores ?
      - ww520 10y ago
        I hate to do this but I have to quote you, "You mistake concurrency with parallelism."
        di4na 10y ago
        Not really. Once again. What stop completely concurrent code to run on different schedulers ?
        ww520 10y ago
        What you described is concurrency. What OP was talking about is parallelism.
  - sbov 10y ago
    If every call is an async message but the next step in your algorithm always depends upon the result/outcome of that async message you are programming a sequential algorithm wrapped in concurrent clothing. I would claim that even if you use Erlang and distribute it across multiple processes your program doesn't actually exhibit concurrency. If you want to argue otherwise that's fine, but you just push the problem out: your program is "concurrent" but you gain zero benefits of concurrency, and when you distribute it across multiple cores to try to make it run in parallel it might be concurrent but by definition it will not be parallel. Either way you want to slice it you have a big problem with this line of thought. Edit: from my experience it is easy to write Erlang programs that exhibit something not exactly the same as, but similar to, this sort of behavior.
- PopsiclePete 10y ago
  "Trivial" to write parallel programs in C? You should stop doing whatever it is that you're doing, because you are in the top 0.00000000001% of programmers, in the Universe, and I want to work on whatever multi-billion dollar startup you undoubtedly have going on.
  - chrisseaton 10y ago
    I said it is trivial to parallelise a C program that implements an embarrassingly parallel algorithm. Embarrassingly parallel means no resources need to be shared, and there needs to be no communication, so no need for locks or messages, even in C. That's why I say it'd be trivial to parallelise it. Just add a pthread_start. Of course almost no applications are embarrassingly parallel.
    - m00x 10y ago
      And then you also need to implement mailboxes to receive new data, supervisors to watch the threads and revive them when they fail, initialization methods to get back to the original state, etc. You can trivialize any issue if you ignore all the facts, but it doesn't make your argument have any correctness.
      - BeetleB 10y ago
        I'm really surprised at the arguing in this thread. Embarrassingly parallel algorithms are trivial in pretty much every language. That is why is embarrassing. >And then you also need to implement mailboxes to receive new data If you did, it is not embarrassingly parallel. You know, a huge amount of parallel work is done in C/C++. It was the language used for multiple parallel algorithms courses in my university. Being able to do basic MPI type stuff in C does not make you a top programmer. And not being able to do an embarrassingly parallel algorithm in C would ring alarm bells.
        ww520 10y ago
        People's obsession with language choice often cloud their judgement.
aaron-lebo 10y ago
I think the "more cores == faster" benefit is overstated. Without jit Erlang is so slow that languages without concurrency support will kill it. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. Not to mention it's 10 years later and we haven't seen massively parallel architectures take off. I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.
- zitterbewegung 10y ago
  The ways that massively parallel architectures have taken off are in GPU programming and they interact with them using the languages that they would have already done. C/C++ libraries that either use them directly or provide an interface to Python / Lua / Java etc...
- davidw 10y ago
  > fold the good parts of Erlang/BEAM into them A lot easier said than done, because there are a lot of really fundamental decisions about Erlang that make it what it is. You can't just slap on a scheduler for instance, or Erlang style processes. Go is probably the closest thing, and it's still lacking some things like the supervision tree. Erlang is certainly not 'the answer' in terms of computationally intensive anything, for the foreseeable future.
  - djKianoosh 10y ago
    so many people (including me from time to time) misunderstand "computaionally expensive" that I think Elixir/erlang/beam will serve a lot of systems/apps well for a long time until they really see a need for "intense" computation
    - davidw 10y ago
      Oh, absolutely - you can do tons of things with Erlang, just as people do with Ruby, Python, PHP, Tcl, etc... But that article is a bit hype-y.
davidw 10y ago
This was back in a brief period when Erlang was being hyped by people as a potential Next Big Thing. Unfortunately it never really got that much momentum though, until Elixir and Phoenix came along.
- derefr 10y ago
  I've always thought that Erlang would be more successful if it had a better deployment story, along the lines of Go or Rust: e.g. if you had the option of baking an Erlang runtime into a static binary. As it stands, software like ejabberd and rabbitmq are pretty ugly in OS package form (because distro maintainers think such packages need to rely on a common, usually very old, Erlang runtime, instead of allowing them to vendor it in as you're supposed to.) Now that I think of it, Canonical's push toward "snaps" (sandboxed and vendored—but not fully containerized—software bundles, as a package format) might bring a flourishing of Erlang-based application software, given that it's much more compatible with Erlang's style of release-management.
  - djKianoosh 10y ago
    please, more sources and links? i would love to package up an entire elixir app as an rpm to a rhel7 target that has no internet connection. so the package/binary needs to have everything including erlang. i am not afraid of compiling from source. is the answer distillery or something else entirely?
    - IanCal 10y ago
      Does it need to be an RPM or just something easy to install office? The release process I've used for Erlang before packaged things up into a folder with everything you needed to run.
      - djKianoosh 10y ago
        rpm is preferred. the unix teams i deal with and the checkboxes they have to check.. err i mean they have to comply with some restrictrictions so from what i understand so far, rpm package would require the fewest amount of hoops to go through
slashdotdash 10y ago
Dave Thomas has described Elixir as follows: "Elixir took the Erlang virtual machine, BEAM, and put a sensible face on it. It gives you all the power of Erlang plus a powerful macro system."[1] [1] https://startlearningelixir.com/elixir-for-rubyists https://startlearningelixir.com/elixir-for-rubyists
- jbernardo95 10y ago
  In other words, Elixir is the rocket that will make Erlang go big.
  - rvirding 10y ago
    Well, it's a skin on the rocket that will make it big. While they generally say that Elixir runs on top of the Erlang VM there happens to be a big fat Erlang/OTP layer in-between which Elixir and its libraries make full use of. This is just smart of course but not mentioning doesn't paint the whole picture.
    - yellowapple 10y ago
      I'd say Elixir is the skin, like you say: without it, all your rocket pieces are out in the open and not particularly aerodynamic, and they'll start breaking off and you won't be going to space today. There are other skins, though, like LFE and Erlang itself. The Erlang VM is the engine. You can make a beautiful, sleek, aerodynamic rocket, but it ain't going anywhere unless you literally light a fire under its butt. It helps that BEAM helps the rocket go fast :) Of course, you need some sort of structural piece to hold your engine and your aerodynamic skin in place, lest your rocket crumples itself up into a ball at launch. That's where OTP comes in: providing a robust structure for your rocket. Last but not least, you need to launch it. Rebar and Mix work reasonably well as launch clamps in this really contrived metaphor. You also have exrm (or Distillery nowadays, I guess) that works as the VAB, in which your rocket is put together so it can be launched.
      - mnd 10y ago
hota_mazi 10y ago
> How do programmers fix these problems? They don’t. They pray. Ah! No, Joe, that's what you do. Most of developers have figured out how to do concurrency and parallelism with mutable state and locks. It's not as hard as you make it sound and it's supported in a lot of languages which, as opposed to Erlang, are statically typed, hence much more suitable to the kind of large scale software you describe. And yes, that was already true in 2007.
- douche 10y ago
  You have a more optimistic view of the average level of skill and knowledge of developers than I would. Basic concurrency bugs strike all the time, and a lot of people don't know what they don't know, until they've been burned a few times.
- zaphar 10y ago
  If I am totally honest. Most of the code where I was pretty sure I had figured out the whole mutable states and locks thing has another developer looking at it right now and cursing me under his breath.
  - patates 10y ago
    I'm investigating an issue with a deployment where a process keeps crashing with the log.fatal("not reachable"). I already fixed 3 potential race conditions and still not sure if and when I'll get my next email basically telling me "haha it's still reachable". Yesterday my neighbor came to my door to ask if everything is okay because I cried "aber warum?" (but why?) out loud. tl;dr: Don't be that programmer :)
- gamache 10y ago
  Do you realize how silly this sounds? You are disregarding not only Erlang, but decades of research and development on concurrent programming best practices. Immutability has been promoted for parallel programming since the 1970s. For the past fifteen years, even languages like C++ have moved to promote this style of programming. (Also, Erlang has a compile-time static type checker, if type safety is something you're after.)
eternalban 10y ago
Erlang's secret sauce is OTP.
- yellowapple 10y ago
  OTP is what makes Erlang Web Scale™.
NotThe1Pct 10y ago
I looked at Erlang some time ago with a consultant, who was trying to get something Erlang going in our firm, and we compared a simple queue with boost, which is not really optimized for speed. I think the C++ version was sort of 20x faster. I assume the dude was not that unskilled since he was selling his Erlang solutions for many years but it was bad enough that it left no room for a second meeting.
lngnmn 10y ago
A practical language based on well-researched first principles. Details in the armstrong_thesis_2003.pdf * java is unsuitable * no pthreads, please (wrong concept) * immutable data (no locks - no problem) * isolated lightweight processes (share nothing) * communication by message-passing (grom Kay's OOP) * dynamic typing (good-enough, quick prototyping) * pattern matching (on receive) * supervision hierarchies (fault tolerance) * reusable components for server development I probably forgot something
- XorNot 10y ago
  I've not found immutable data to be the big win people tout it as. In most of what I do acting on stale data is just as bad, which means I wind up needing a lock anyway. Doing a < b and taking an action based on that result is as harmful with stale data.
  - gardenhead 10y ago
    But the data can't be stale, because `a` and `b` are immutable
  - kqr 10y ago
    I'd like to tell you that stale data will always be a thing in any serious application because light only travels so fast. Therefore, you'll need to have sensible logic in the face of stale data either way, so it's not so bad as you think. ...and while that's true, I'd still like to hear the opinion of someone more knowledgeable because it's an issue that's been bugging me too.
  - omgwtfpmr 10y ago
    This is a fair point: immutable data is just part of the solution. To get the win, you need to toss out all references to the old data when you update it: then you ensure that you can't access "stale" data. This requires some discipline, either manual (e.g., reusing the same variable name for the old and new bindings to ensure the stale binding is shadowed) or automatic (e.g., linear types).
- ww520 10y ago
rdtsc 10y ago
I posted this in another thread on what I like about Erlang (I'll shamelessly copy and paste it here): * A better concurrency model. Lightweight processes vs callbacks and futures. Or even channels. I find processes as concurrency units maps better to problems I had to solve. So there is less impedance mismatch. That makes a huge deal in a larger project. (eg.: a user request as an isolated process vs a chain of callbacks). OS design got this right years ago -- think how most modern popular operating systems represent concurrency - an isolated process. * Safety & fault tolerance. Processes are isolated. So you can build very robust systems. That simply puts money in your pocket just due to ops costs. A non-critical / experimental part of backend is crashing at 4am and restarting? No, problem, keep sleeping. Rest of the service will stay up and you can fix it in the morning. * Debuggability and inspection. BEAM VM comes with built in ability to debug, inspect and trace out of the box. That has saved so much time (and money) over the years. * Hot code reload. This is a first class feature. We don't rely on it to do upgrades. But it proved invaluable to fix an issues or two for a high value customer without taking down the services. Or simply to add extra logging to monitor a pathological edge case. Some people might prefer Elixir and that's fine, they are great friends. I personally like Erlang, I think it is simpler. Someone new coming from Python or Ruby might like it better and also it has very beginner friendly community. Jose Valim and and other authors really put user-friendliness and approach-ability at the forefront. I really like that, mad props to them.
B1FF_PSUVM 10y ago
Incidentally, does anyone have a screenshot of the "all Erlang" HN front page of similar vintage (perhaps 2009 or so)?
KirinDave 10y ago
For folks who like Erlang, also check out Pony.
thinkling 10y ago
Are there online classes (eg MOOCs) teaching Erlang? I looked at Coursera, EdX, and Udacity recently and didn't find anything.
- cxa 10y ago
  There are some good introductory videos at https://www.cs.kent.ac.uk/ErlangMasterClasses/ https://www.cs.kent.ac.uk/ErlangMasterClasses/.
soyiuz 10y ago
How to make sure I don't read your post: > Our site has been optimized for use with newer browsers. We also require that your browser has JavaScript enabled. > It looks as if your browser has JavaScript disabled. > This site has information about enabling JavaScript, if that's something you want to do. Really? I am just trying read a few paragraphs of text on a fairly static-looking site.
- LukeShu 10y ago
  It's even worse than that. It is static. It would work without javascript if they didn't put <noscript> <meta http-equiv="refresh" content="2;url=https://pragprog.com/no_js"> </noscript>
  - LukeShu 10y ago
    As a slight followup: In the Advanced settings panel for NoScript, there's an option to disable <meta> redirects in <noscript> blocks.
- slrz 10y ago
  I actually saw the text just fine before that stupid error message showed up.
andrewvijay 10y ago
"How do programmers fix these problems? They don’t. They pray." :D LOLed at this!!
kriro 10y ago
I'm not well versed in the entire (phone) network space. Ericson switch uptime (powered by Erlang) is always cited and admittedly initially drew me to the whole ecosystem. Well mostly the "it's used in a network that has basically worked all these years so it must be rather robust one way or another" line of thinking. However what I'm wondering is...there's other vendors and Ericson isn't powering the entire network. What software stacks are used by other vendors? I'm assuming it's mostly C-ish stuff with some ASM sprinkles like in Cisco IOS?
- hellofunk 10y ago
  Ericson makes a switch that some other networks use. That switch contains lots of Erlang inside it.

231 points by krat0sprakhar 10y ago

> Your Erlang program should just run N times faster on an N core processor But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever. If your program has sequential or less parallel phases or needs to communicate then you are subject to Amdahl's law like you always were. Armstrong has that claim in the Erlang book and I was gobsmacked to see it written down with no caveat or mention of the limits from Amdahl's law whatsoever. I was sure it was a joke and it would be followed by 'ha ha ha of course not - nobody knows how to achieve that despite decades of intensive research', but no it's a serious claim made with a straight face. Erlang's solved it! Erlang helps you write parallel programs... but only if you program is entirely parallel in the first place.

geodel 10y ago
I searched many times the comparison of Erlang BEAM and JVM regarding performance but couldn't find much. May be it is not done, or maybe it is not publicly available. At my work some folks keep pushing RabbitMQ over ActiveMQ for no other good reason than it is in Erlang so we will get much better performance. I know it is not true but could not find web links to support that.
- novembermike 10y ago
  BEAM is slower than the JVM for sequential code but it doesn't have a block the world GC and it has preemptive multi processing. It'll have worse steady state performance but a lot of the worst cases that Java can have are heavily mitigated.
  - jerf 10y ago
    It's also realllly good at routing; the control logic is a somewhat slow more-or-less interpreted language speed but then the act of routing bytes around runs at native speeds (as long as you keep things in binaries and don't convert them to "strings"), with the Erlang framework making sure by the nature of how it works that there's no contention or anything getting in the way because everything's immutable. It's sort of like how people talk about how NumPy can be so useful; Python is itself one of the slowest languages (especially since when it comes to NumPy we can't necessarily count on PyPy, though that's coming along) but it doesn't matter because you just use the slow language to script up the computation you want to do, which then goes and runs in highly-optimized assembler implementations. You can look at Erlang being very similar when being used in its native domain for networking, using a slower language to decide how to route bytes around. So it's quite possible for competent Erlang to beat out competent Java for such a task. Java optimized to within an inch of its life can certainly beat Erlang, but it's going to be a much uglier code base to get there.
  - pjmlp 10y ago
    You are making the error of comparing BEAM with a specific JVM implementation. Not all available JVMs on the market have "a block the world GC" and the way multi-processing is done is implementation defined.
    - novembermike 10y ago
      I could be wrong but I was under the impression that the four major JVM gc's all had a stop the world phase and while multi-processing is implementation specific I'm not aware of a robust pre-emptive multiprocessing model built for the JVM. If you have robust examples that are contrary to this I'd be interested in hearing about them.
      - pdpi 10y ago
        Azul doesn't. It's not a differnt GC for the hotspot VM, it's a separate VM altogether
floatboth 10y ago
Obviously said "program" must be a server that services independent requests. I mean, not obviously, but if you work in telecom, web, etc. that's pretty much all programs you see.
- chrisseaton 10y ago
  I get that, but as I say we already knew how to scale those linearly in most programming languages. Erlang brought nothing new there. It does have advances in things like reloading and supervision.
  - mnd 10y ago
    Erlang was invented 30 years ago to solve problems that were not easily solvable by the back then prevalent programming languages. The claim that Erlang hasn’t brought anything new to the table couldn’t be further from the truth. Designing concurrent, fault-tolerant programs was not a problem solved in a satisfying way by the programming languages and technologies that were mainstream back then, which is exactly why Ericsson pursued the path of inventing their own programming language. I suggest everyone to read Joe Armstrong’s paper “Making reliable distributed systems in the presence of software errors”. It’s easy to read and mind expanding even if you do not intend to use Erlang. It can also save one from public embarrassment when one claims ridiculous things as parent and many other remarks in the other comments.
    - chrisseaton 10y ago
      I acknowledged the areas where Erlang did bring advancements such as fault tolerance. I read that paper, more than once, and his thesis, when I was doing my own PhD on parallelism, so I know the area and related work fairly well. I don't think I'm embarrassing myself when I say that it's silly to claim that an arbitrary Erlang program should achieve linear scalability. But we don't need to debate with vague claims to originality - as I say Amdahl has done the maths and shows that it is only possible for a program that is embarrassingly parallel.
      - 10y ago
timClicks 10y ago
That's true, but when I read that sentence I place the emphasis on should. I see it as aspirational. A better way to phrase this would perhaps would be to say that perfectly preemptive scheduler should be able to keep all cores hot. Blocking of one Erlang process should not halt the program's progress. As Erlang requires message passing overhead by design, it will never see perfectly linear scaling behaviour. Still, its scheduler, its GC model and M:N threading implementation are extremely robust and can be relied upon to write extremely reliable networked services.
- nextos 10y ago
  I also interpret it this way. If you recall the famous Wide Finder benchmark for parallel programming by Tim Bray, back in 2007 as well [1], those were essentially the results. Languages like Erlang or JoCaml (Ocaml plus joint calculus), whose models are inherently parallel yield simple and fast solutions. Other languages too, but they require a lot of expert optimization and ugly code. [1] https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Find...
- chrisseaton 10y ago
  > That's true, but when I read that sentence I place the emphasis on should. I see it as aspirational. Despite all the arguing I'm taking part in, this is probably the true and reasonable explanation. I read 'should' as 'if it doesn't there's a problem'.
  - inopinatus 10y ago
    Yes, indeed, don't forget his very next line is "Is this true? Almost. It’s early still, but we’re optimistic (Wildly optimistic; I haven’t seen this kind of optimism for the last 20 years!)."
- amelius 10y ago
  > As Erlang requires message passing overhead by design That's Erlang's main weakness. For parallel code, structurally shared data is immensely useful for processes running in the same address space. With Erlang, you have to give that up. OTOH, not having this possibility makes distributing processes to other nodes a lot simpler.
zzzcpan 10y ago
> If you have one of those And how would you have one of those? Erlang encourages you to design and think about programs in a certain way, that typical C, Java or Python does not. I've seen Armstrong in one of his talks to actually emphasize the importance of learning distributed systems if you wanted Erlang to be useful for you. That's way beyond "embarrassingly parallel" point of view, Amdahl's law has no use here.
devishard 10y ago
>> Your Erlang program should just run N times faster on an N core processor > But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever. You've made two claims, one irrelevant and one false: 1. only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized. 2. it's trivial to write a program that's faster in X language I'm not sure how toot define trivial, but I've yet to find a language that can communicate between threads as performant-ly. Even languages like Clojure which use similar thread semantics can't do what Erlang can because the underlying threads aren't as lightweight. Spinning up a million threads in Erlang isn't even unusual, whereas in any of the languages you mention it's either crippling-ly slow (Python or Java) or very difficult to synchronize (C most, but Python and Java aren't easy).
- BeetleB 10y ago
  >only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized. OK. Not an Erlang user, but I can't let this statement go. I've studied parallel numerical algorithms. Many/most of them will involve blocking because you're waiting for results from other nodes. If you're saying Erlang has somehow found a way to do those numerical algorithms without having to wait, then I'd love to see all those textbooks rewritten. Amdahl's Law reigns supreme.
  - aeturnum 10y ago
    I'm not devishard, but I parsed his statement slightly differently. He's not saying that Erlang makes things parallel magically. Rather, he's saying that Erlang forces tasks that /could/ be parallel to be parallel by default. Thus, Erlang will tend to maximize the sections of your program that are run in parallel compared to other languages.
di4na 10y ago
You mistake concurrency with parallelism. If each of your "synchronous" call to a method is an asynchronous message, suddenly, all your program is embarasingly concurrent. Wait... was that what Alan Kay was saying with SmallTalk ? Would we have all lost our mind on RPC and synchronous calls for so long? Yes. Yes we did.
- ww520 10y ago
  "Embarrassingly concurrent" is not embarrassingly parallel. Parallelism is not merely making things concurrent nor making synchronous calls into concurrent async calls.
  - di4na 10y ago
    once you are concurrent, what stop you from being distributed to multiple cores ?
    - ww520 10y ago
      I hate to do this but I have to quote you, "You mistake concurrency with parallelism."
      - di4na 10y ago
        Not really. Once again. What stop completely concurrent code to run on different schedulers ?
        ww520 10y ago
        What you described is concurrency. What OP was talking about is parallelism.
- sbov 10y ago
  If every call is an async message but the next step in your algorithm always depends upon the result/outcome of that async message you are programming a sequential algorithm wrapped in concurrent clothing. I would claim that even if you use Erlang and distribute it across multiple processes your program doesn't actually exhibit concurrency. If you want to argue otherwise that's fine, but you just push the problem out: your program is "concurrent" but you gain zero benefits of concurrency, and when you distribute it across multiple cores to try to make it run in parallel it might be concurrent but by definition it will not be parallel. Either way you want to slice it you have a big problem with this line of thought. Edit: from my experience it is easy to write Erlang programs that exhibit something not exactly the same as, but similar to, this sort of behavior.
PopsiclePete 10y ago
"Trivial" to write parallel programs in C? You should stop doing whatever it is that you're doing, because you are in the top 0.00000000001% of programmers, in the Universe, and I want to work on whatever multi-billion dollar startup you undoubtedly have going on.
- chrisseaton 10y ago
  I said it is trivial to parallelise a C program that implements an embarrassingly parallel algorithm. Embarrassingly parallel means no resources need to be shared, and there needs to be no communication, so no need for locks or messages, even in C. That's why I say it'd be trivial to parallelise it. Just add a pthread_start. Of course almost no applications are embarrassingly parallel.
  - m00x 10y ago
    And then you also need to implement mailboxes to receive new data, supervisors to watch the threads and revive them when they fail, initialization methods to get back to the original state, etc. You can trivialize any issue if you ignore all the facts, but it doesn't make your argument have any correctness.
    - BeetleB 10y ago
      I'm really surprised at the arguing in this thread. Embarrassingly parallel algorithms are trivial in pretty much every language. That is why is embarrassing. >And then you also need to implement mailboxes to receive new data If you did, it is not embarrassingly parallel. You know, a huge amount of parallel work is done in C/C++. It was the language used for multiple parallel algorithms courses in my university. Being able to do basic MPI type stuff in C does not make you a top programmer. And not being able to do an embarrassingly parallel algorithm in C would ring alarm bells.
      - ww520 10y ago
        People's obsession with language choice often cloud their judgement.

aaron-lebo 10y ago

I think the "more cores == faster" benefit is overstated. Without jit Erlang is so slow that languages without concurrency support will kill it. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. Not to mention it's 10 years later and we haven't seen massively parallel architectures take off. I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.

zitterbewegung 10y ago
The ways that massively parallel architectures have taken off are in GPU programming and they interact with them using the languages that they would have already done. C/C++ libraries that either use them directly or provide an interface to Python / Lua / Java etc...
davidw 10y ago
> fold the good parts of Erlang/BEAM into them A lot easier said than done, because there are a lot of really fundamental decisions about Erlang that make it what it is. You can't just slap on a scheduler for instance, or Erlang style processes. Go is probably the closest thing, and it's still lacking some things like the supervision tree. Erlang is certainly not 'the answer' in terms of computationally intensive anything, for the foreseeable future.
- djKianoosh 10y ago
  so many people (including me from time to time) misunderstand "computaionally expensive" that I think Elixir/erlang/beam will serve a lot of systems/apps well for a long time until they really see a need for "intense" computation
  - davidw 10y ago
    Oh, absolutely - you can do tons of things with Erlang, just as people do with Ruby, Python, PHP, Tcl, etc... But that article is a bit hype-y.

davidw 10y ago

This was back in a brief period when Erlang was being hyped by people as a potential Next Big Thing. Unfortunately it never really got that much momentum though, until Elixir and Phoenix came along.

derefr 10y ago
I've always thought that Erlang would be more successful if it had a better deployment story, along the lines of Go or Rust: e.g. if you had the option of baking an Erlang runtime into a static binary. As it stands, software like ejabberd and rabbitmq are pretty ugly in OS package form (because distro maintainers think such packages need to rely on a common, usually very old, Erlang runtime, instead of allowing them to vendor it in as you're supposed to.) Now that I think of it, Canonical's push toward "snaps" (sandboxed and vendored—but not fully containerized—software bundles, as a package format) might bring a flourishing of Erlang-based application software, given that it's much more compatible with Erlang's style of release-management.
- djKianoosh 10y ago
  please, more sources and links? i would love to package up an entire elixir app as an rpm to a rhel7 target that has no internet connection. so the package/binary needs to have everything including erlang. i am not afraid of compiling from source. is the answer distillery or something else entirely?
  - IanCal 10y ago
    Does it need to be an RPM or just something easy to install office? The release process I've used for Erlang before packaged things up into a folder with everything you needed to run.
    - djKianoosh 10y ago
      rpm is preferred. the unix teams i deal with and the checkboxes they have to check.. err i mean they have to comply with some restrictrictions so from what i understand so far, rpm package would require the fewest amount of hoops to go through

slashdotdash 10y ago

Dave Thomas has described Elixir as follows: "Elixir took the Erlang virtual machine, BEAM, and put a sensible face on it. It gives you all the power of Erlang plus a powerful macro system."[1] [1] https://startlearningelixir.com/elixir-for-rubyists https://startlearningelixir.com/elixir-for-rubyists

jbernardo95 10y ago
In other words, Elixir is the rocket that will make Erlang go big.
- rvirding 10y ago
  Well, it's a skin on the rocket that will make it big. While they generally say that Elixir runs on top of the Erlang VM there happens to be a big fat Erlang/OTP layer in-between which Elixir and its libraries make full use of. This is just smart of course but not mentioning doesn't paint the whole picture.
  - yellowapple 10y ago
    I'd say Elixir is the skin, like you say: without it, all your rocket pieces are out in the open and not particularly aerodynamic, and they'll start breaking off and you won't be going to space today. There are other skins, though, like LFE and Erlang itself. The Erlang VM is the engine. You can make a beautiful, sleek, aerodynamic rocket, but it ain't going anywhere unless you literally light a fire under its butt. It helps that BEAM helps the rocket go fast :) Of course, you need some sort of structural piece to hold your engine and your aerodynamic skin in place, lest your rocket crumples itself up into a ball at launch. That's where OTP comes in: providing a robust structure for your rocket. Last but not least, you need to launch it. Rebar and Mix work reasonably well as launch clamps in this really contrived metaphor. You also have exrm (or Distillery nowadays, I guess) that works as the VAB, in which your rocket is put together so it can be launched.
    - mnd 10y ago

hota_mazi 10y ago

> How do programmers fix these problems? They don’t. They pray. Ah! No, Joe, that's what you do. Most of developers have figured out how to do concurrency and parallelism with mutable state and locks. It's not as hard as you make it sound and it's supported in a lot of languages which, as opposed to Erlang, are statically typed, hence much more suitable to the kind of large scale software you describe. And yes, that was already true in 2007.

douche 10y ago
You have a more optimistic view of the average level of skill and knowledge of developers than I would. Basic concurrency bugs strike all the time, and a lot of people don't know what they don't know, until they've been burned a few times.
zaphar 10y ago
If I am totally honest. Most of the code where I was pretty sure I had figured out the whole mutable states and locks thing has another developer looking at it right now and cursing me under his breath.
- patates 10y ago
  I'm investigating an issue with a deployment where a process keeps crashing with the log.fatal("not reachable"). I already fixed 3 potential race conditions and still not sure if and when I'll get my next email basically telling me "haha it's still reachable". Yesterday my neighbor came to my door to ask if everything is okay because I cried "aber warum?" (but why?) out loud. tl;dr: Don't be that programmer :)
gamache 10y ago
Do you realize how silly this sounds? You are disregarding not only Erlang, but decades of research and development on concurrent programming best practices. Immutability has been promoted for parallel programming since the 1970s. For the past fifteen years, even languages like C++ have moved to promote this style of programming. (Also, Erlang has a compile-time static type checker, if type safety is something you're after.)

eternalban 10y ago

Erlang's secret sauce is OTP.

yellowapple 10y ago
OTP is what makes Erlang Web Scale™.

NotThe1Pct 10y ago

I looked at Erlang some time ago with a consultant, who was trying to get something Erlang going in our firm, and we compared a simple queue with boost, which is not really optimized for speed. I think the C++ version was sort of 20x faster. I assume the dude was not that unskilled since he was selling his Erlang solutions for many years but it was bad enough that it left no room for a second meeting.

lngnmn 10y ago

A practical language based on well-researched first principles. Details in the armstrong_thesis_2003.pdf * java is unsuitable * no pthreads, please (wrong concept) * immutable data (no locks - no problem) * isolated lightweight processes (share nothing) * communication by message-passing (grom Kay's OOP) * dynamic typing (good-enough, quick prototyping) * pattern matching (on receive) * supervision hierarchies (fault tolerance) * reusable components for server development I probably forgot something

XorNot 10y ago
I've not found immutable data to be the big win people tout it as. In most of what I do acting on stale data is just as bad, which means I wind up needing a lock anyway. Doing a < b and taking an action based on that result is as harmful with stale data.
- gardenhead 10y ago
  But the data can't be stale, because `a` and `b` are immutable
- kqr 10y ago
  I'd like to tell you that stale data will always be a thing in any serious application because light only travels so fast. Therefore, you'll need to have sensible logic in the face of stale data either way, so it's not so bad as you think. ...and while that's true, I'd still like to hear the opinion of someone more knowledgeable because it's an issue that's been bugging me too.
- omgwtfpmr 10y ago
  This is a fair point: immutable data is just part of the solution. To get the win, you need to toss out all references to the old data when you update it: then you ensure that you can't access "stale" data. This requires some discipline, either manual (e.g., reusing the same variable name for the old and new bindings to ensure the stale binding is shadowed) or automatic (e.g., linear types).
ww520 10y ago

rdtsc 10y ago

I posted this in another thread on what I like about Erlang (I'll shamelessly copy and paste it here): * A better concurrency model. Lightweight processes vs callbacks and futures. Or even channels. I find processes as concurrency units maps better to problems I had to solve. So there is less impedance mismatch. That makes a huge deal in a larger project. (eg.: a user request as an isolated process vs a chain of callbacks). OS design got this right years ago -- think how most modern popular operating systems represent concurrency - an isolated process. * Safety & fault tolerance. Processes are isolated. So you can build very robust systems. That simply puts money in your pocket just due to ops costs. A non-critical / experimental part of backend is crashing at 4am and restarting? No, problem, keep sleeping. Rest of the service will stay up and you can fix it in the morning. * Debuggability and inspection. BEAM VM comes with built in ability to debug, inspect and trace out of the box. That has saved so much time (and money) over the years. * Hot code reload. This is a first class feature. We don't rely on it to do upgrades. But it proved invaluable to fix an issues or two for a high value customer without taking down the services. Or simply to add extra logging to monitor a pathological edge case. Some people might prefer Elixir and that's fine, they are great friends. I personally like Erlang, I think it is simpler. Someone new coming from Python or Ruby might like it better and also it has very beginner friendly community. Jose Valim and and other authors really put user-friendliness and approach-ability at the forefront. I really like that, mad props to them.

B1FF_PSUVM 10y ago

Incidentally, does anyone have a screenshot of the "all Erlang" HN front page of similar vintage (perhaps 2009 or so)?

KirinDave 10y ago

For folks who like Erlang, also check out Pony.

thinkling 10y ago

Are there online classes (eg MOOCs) teaching Erlang? I looked at Coursera, EdX, and Udacity recently and didn't find anything.

cxa 10y ago
There are some good introductory videos at https://www.cs.kent.ac.uk/ErlangMasterClasses/ https://www.cs.kent.ac.uk/ErlangMasterClasses/.

soyiuz 10y ago

How to make sure I don't read your post: > Our site has been optimized for use with newer browsers. We also require that your browser has JavaScript enabled. > It looks as if your browser has JavaScript disabled. > This site has information about enabling JavaScript, if that's something you want to do. Really? I am just trying read a few paragraphs of text on a fairly static-looking site.

LukeShu 10y ago
It's even worse than that. It is static. It would work without javascript if they didn't put <noscript> <meta http-equiv="refresh" content="2;url=https://pragprog.com/no_js"> </noscript>
- LukeShu 10y ago
  As a slight followup: In the Advanced settings panel for NoScript, there's an option to disable <meta> redirects in <noscript> blocks.
slrz 10y ago
I actually saw the text just fine before that stupid error message showed up.

andrewvijay 10y ago

"How do programmers fix these problems? They don’t. They pray." :D LOLed at this!!

kriro 10y ago

I'm not well versed in the entire (phone) network space. Ericson switch uptime (powered by Erlang) is always cited and admittedly initially drew me to the whole ecosystem. Well mostly the "it's used in a network that has basically worked all these years so it must be rather robust one way or another" line of thinking. However what I'm wondering is...there's other vendors and Ericson isn't powering the entire network. What software stacks are used by other vendors? I'm assuming it's mostly C-ish stuff with some ASM sprinkles like in Cisco IOS?

hellofunk 10y ago
Ericson makes a switch that some other networks use. That switch contains lots of Erlang inside it.