8 ms·
> Your Erlang program should just run N times faster on an N core processor But only if your program is embarrassingly parallel with at least N times available
by chrisseaton 10y ago
> Your Erlang program should just run N times faster on an N core processor
But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever.
If your program has sequential or less parallel phases or needs to communicate then you are subject to Amdahl's law like you always were.
Armstrong has that claim in the Erlang book and I was gobsmacked to see it written down with no caveat or mention of the limits from Amdahl's law whatsoever. I was sure it was a joke and it would be followed by 'ha ha ha of course not - nobody knows how to achieve that despite decades of intensive research', but no it's a serious claim made with a straight face. Erlang's solved it!
Erlang helps you write parallel programs... but only if you program is entirely parallel in the first place.
- geodel 10y agoI searched many times the comparison of Erlang BEAM and JVM regarding performance but couldn't find much. May be it is not done, or maybe it is not publicly available. At my work some folks keep pushing RabbitMQ over ActiveMQ for no other good reason than it is in Erlang so we will get much better performance. I know it is not true but could not find web links to support that.
- novembermike 10y agoBEAM is slower than the JVM for sequential code but it doesn't have a block the world GC and it has preemptive multi processing. It'll have worse steady state performance but a lot of the worst cases that Java can have are heavily mitigated.
- jerf 10y agoIt's also realllly good at routing; the control logic is a somewhat slow more-or-less interpreted language speed but then the act of routing bytes around runs at native speeds (as long as you keep things in binaries and don't convert them to "strings"), with the Erlang framework making sure by the nature of how it works that there's no contention or anything getting in the way because everything's immutable. It's sort of like how people talk about how NumPy can be so useful; Python is itself one of the slowest languages (especially since when it comes to NumPy we can't necessarily count on PyPy, though that's coming along) but it doesn't matter because you just use the slow language to script up the computation you want to do, which then goes and runs in highly-optimized assembler implementations. You can look at Erlang being very similar when being used in its native domain for networking, using a slower language to decide how to route bytes around. So it's quite possible for competent Erlang to beat out competent Java for such a task. Java optimized to within an inch of its life can certainly beat Erlang, but it's going to be a much uglier code base to get there.
- pjmlp 10y agoYou are making the error of comparing BEAM with a specific JVM implementation. Not all available JVMs on the market have "a block the world GC" and the way multi-processing is done is implementation defined.
- novembermike 10y agoI could be wrong but I was under the impression that the four major JVM gc's all had a stop the world phase and while multi-processing is implementation specific I'm not aware of a robust pre-emptive multiprocessing model built for the JVM. If you have robust examples that are contrary to this I'd be interested in hearing about them.
- pdpi 10y agoAzul doesn't. It's not a differnt GC for the hotspot VM, it's a separate VM altogether
- floatboth 10y agoObviously said "program" must be a server that services independent requests. I mean, not obviously, but if you work in telecom, web, etc. that's pretty much all programs you see.
- chrisseaton 10y agoI get that, but as I say we already knew how to scale those linearly in most programming languages. Erlang brought nothing new there. It does have advances in things like reloading and supervision.
- mnd 10y agoErlang was invented 30 years ago to solve problems that were not easily solvable by the back then prevalent programming languages. The claim that Erlang hasn’t brought anything new to the table couldn’t be further from the truth. Designing concurrent, fault-tolerant programs was not a problem solved in a satisfying way by the programming languages and technologies that were mainstream back then, which is exactly why Ericsson pursued the path of inventing their own programming language. I suggest everyone to read Joe Armstrong’s paper “Making reliable distributed systems in the presence of software errors”. It’s easy to read and mind expanding even if you do not intend to use Erlang. It can also save one from public embarrassment when one claims ridiculous things as parent and many other remarks in the other comments.
- chrisseaton 10y agoI acknowledged the areas where Erlang did bring advancements such as fault tolerance. I read that paper, more than once, and his thesis, when I was doing my own PhD on parallelism, so I know the area and related work fairly well. I don't think I'm embarrassing myself when I say that it's silly to claim that an arbitrary Erlang program should achieve linear scalability. But we don't need to debate with vague claims to originality - as I say Amdahl has done the maths and shows that it is only possible for a program that is embarrassingly parallel.
- 10y ago
- timClicks 10y agoThat's true, but when I read that sentence I place the emphasis on should. I see it as aspirational. A better way to phrase this would perhaps would be to say that perfectly preemptive scheduler should be able to keep all cores hot. Blocking of one Erlang process should not halt the program's progress. As Erlang requires message passing overhead by design, it will never see perfectly linear scaling behaviour. Still, its scheduler, its GC model and M:N threading implementation are extremely robust and can be relied upon to write extremely reliable networked services.
- nextos 10y agoI also interpret it this way. If you recall the famous Wide Finder benchmark for parallel programming by Tim Bray, back in 2007 as well [1], those were essentially the results. Languages like Erlang or JoCaml (Ocaml plus joint calculus), whose models are inherently parallel yield simple and fast solutions. Other languages too, but they require a lot of expert optimization and ugly code. [1] https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Find...
- chrisseaton 10y ago> That's true, but when I read that sentence I place the emphasis on should. I see it as aspirational. Despite all the arguing I'm taking part in, this is probably the true and reasonable explanation. I read 'should' as 'if it doesn't there's a problem'.
- inopinatus 10y agoYes, indeed, don't forget his very next line is "Is this true? Almost. It’s early still, but we’re optimistic (Wildly optimistic; I haven’t seen this kind of optimism for the last 20 years!)."
- amelius 10y ago> As Erlang requires message passing overhead by design That's Erlang's main weakness. For parallel code, structurally shared data is immensely useful for processes running in the same address space. With Erlang, you have to give that up. OTOH, not having this possibility makes distributing processes to other nodes a lot simpler.
- zzzcpan 10y ago> If you have one of those And how would you have one of those? Erlang encourages you to design and think about programs in a certain way, that typical C, Java or Python does not. I've seen Armstrong in one of his talks to actually emphasize the importance of learning distributed systems if you wanted Erlang to be useful for you. That's way beyond "embarrassingly parallel" point of view, Amdahl's law has no use here.
- devishard 10y ago>> Your Erlang program should just run N times faster on an N core processor > But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever. You've made two claims, one irrelevant and one false: 1. only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized. 2. it's trivial to write a program that's faster in X language I'm not sure how toot define trivial, but I've yet to find a language that can communicate between threads as performant-ly. Even languages like Clojure which use similar thread semantics can't do what Erlang can because the underlying threads aren't as lightweight. Spinning up a million threads in Erlang isn't even unusual, whereas in any of the languages you mention it's either crippling-ly slow (Python or Java) or very difficult to synchronize (C most, but Python and Java aren't easy).
- BeetleB 10y ago>only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized. OK. Not an Erlang user, but I can't let this statement go. I've studied parallel numerical algorithms. Many/most of them will involve blocking because you're waiting for results from other nodes. If you're saying Erlang has somehow found a way to do those numerical algorithms without having to wait, then I'd love to see all those textbooks rewritten. Amdahl's Law reigns supreme.
- aeturnum 10y agoI'm not devishard, but I parsed his statement slightly differently. He's not saying that Erlang makes things parallel magically. Rather, he's saying that Erlang forces tasks that /could/ be parallel to be parallel by default. Thus, Erlang will tend to maximize the sections of your program that are run in parallel compared to other languages.
- di4na 10y agoYou mistake concurrency with parallelism. If each of your "synchronous" call to a method is an asynchronous message, suddenly, all your program is embarasingly concurrent. Wait... was that what Alan Kay was saying with SmallTalk ? Would we have all lost our mind on RPC and synchronous calls for so long? Yes. Yes we did.
- ww520 10y ago"Embarrassingly concurrent" is not embarrassingly parallel. Parallelism is not merely making things concurrent nor making synchronous calls into concurrent async calls.
- di4na 10y agoonce you are concurrent, what stop you from being distributed to multiple cores ?
- sbov 10y agoIf every call is an async message but the next step in your algorithm always depends upon the result/outcome of that async message you are programming a sequential algorithm wrapped in concurrent clothing. I would claim that even if you use Erlang and distribute it across multiple processes your program doesn't actually exhibit concurrency. If you want to argue otherwise that's fine, but you just push the problem out: your program is "concurrent" but you gain zero benefits of concurrency, and when you distribute it across multiple cores to try to make it run in parallel it might be concurrent but by definition it will not be parallel. Either way you want to slice it you have a big problem with this line of thought. Edit: from my experience it is easy to write Erlang programs that exhibit something not exactly the same as, but similar to, this sort of behavior.
- PopsiclePete 10y ago"Trivial" to write parallel programs in C? You should stop doing whatever it is that you're doing, because you are in the top 0.00000000001% of programmers, in the Universe, and I want to work on whatever multi-billion dollar startup you undoubtedly have going on.
- chrisseaton 10y agoI said it is trivial to parallelise a C program that implements an embarrassingly parallel algorithm. Embarrassingly parallel means no resources need to be shared, and there needs to be no communication, so no need for locks or messages, even in C. That's why I say it'd be trivial to parallelise it. Just add a pthread_start. Of course almost no applications are embarrassingly parallel.
- m00x 10y agoAnd then you also need to implement mailboxes to receive new data, supervisors to watch the threads and revive them when they fail, initialization methods to get back to the original state, etc. You can trivialize any issue if you ignore all the facts, but it doesn't make your argument have any correctness.
- BeetleB 10y agoI'm really surprised at the arguing in this thread. Embarrassingly parallel algorithms are trivial in pretty much every language. That is why is embarrassing. >And then you also need to implement mailboxes to receive new data If you did, it is not embarrassingly parallel. You know, a huge amount of parallel work is done in C/C++. It was the language used for multiple parallel algorithms courses in my university. Being able to do basic MPI type stuff in C does not make you a top programmer. And not being able to do an embarrassingly parallel algorithm in C would ring alarm bells.
- ww520 10y agoPeople's obsession with language choice often cloud their judgement.