5 ms·

The Trouble with Erlang Concurrency

33 points by fogus 15y ago

asymptotic 15y ago
tl;dr: The Actor model of concurrency used in Erlang doesn't lend itself well to key/value caches. The fact that Mnesia isn't mentioned once indicates the author is insufficiently knowledgeable about Erlang to write an informed article. This isn't an ad hominem attack because he didn't even mention Mnesia, hence this is an attack on the content of the article rather than his credentials. (This rather anal mention of ad hominem is relevant because the author brings it up in the comments). To those that follow, you'll learn more reading: Erlang - mnesia (Erlang documentation) http://www.erlang.org/doc/man/mnesia.html http://www.erlang.org/doc/man/mnesia.html CAP theorem https://secure.wikimedia.org/wikipedia/en/wiki/CAP_theorem https://secure.wikimedia.org/wikipedia/en/wiki/CAP_theorem And, of course, if you want to use something that someone has "just done", you would simply use Riak, which uses Erlang to implement an Amazon Dynamo type distributed key-value store: http://wiki.basho.com/ http://wiki.basho.com/ Not that difficult. A few Google's and you're set. EDIT: On reading my post I realise that Mnesia is an OTP application, rather than an Erlang built-in, and that Riak is an Erlang-based 3rd party library. However, using canonical libraries is part-and-parcel with learning about a language, IMO.
- jerf 15y ago
  MNesia is basically a built-in, inasmuch as I'm not sure you can find an Erlang install anywhere that doesn't have it (barring somebody going in and specifically plinking it). It ships in the open source release. (OTP application just means that it conforms to the OTP interfaces, so for instance you can "application:start(mnesia)", or depend on it in other OTP applications such as ones you've authored, etc.) Riak is legitimately a third party library, but does provide an existence proof that the specific problem chosen to highlight Erlang's weaknesses does still seem to work. That said, the author has a point. But by privileging actors as much as it does, Erlang gets a lot of great stuff out of it that have no effective competition in other languages, OTP in particular. Haskell may offer every sensible concurrency construct under the sun, but you don't get OTP to speak of. (Some efforts are underway, but they are new and I'd be surprised if they are stable yet.) Erlang is definitely niche, but it fits that niche very well and the niche is larger than it may initially appear... but it certainly isn't the best choice for all purposes. Nothing is, and nothing can be. The argument is a bit klunky but the point is still ultimately correct.
signa11 15y ago
another point to mention here is that you can always partition the keys (as the author points out). however, imho, author errs w.r.t simplicity of usage in java vs in erlang. once things are wrapped up in a module, it should be almost similar. on a side note, ConcurrentHashMap in java also ends up using a hashing scheme to locate the appropriate segment for queries. another approach to partitioning the keys would be to use a bloom-filter. nodes can periodically exchange their filter-databases either in its entirety or a compressed delta etc.
wmf 15y ago
There's no one true approach to concurrency, so any language that only provides one style is not a general-purpose language. IMO that's fine as long as people understand what they're getting into.
- tbrownaw 15y ago
  The lack of the One True Concurrency Model is because different models are faster or simpler for solving certain problems, not because some models are incapable of being used to solve some problems.
beza1e1 15y ago
The thesis of the author is: There is at least one problem, which can be done better with a ConcurrentHashMap than with Actors. I do not see, why you cannot replicate the ConcurrentHashMap behavior in Erlang? > Now, you could do some clever stuff and partition the map up into N actual maps, one for each of N processes and farm the requests out to the set of processes based on the hash-code. But that’s getting complex[...]. Well, his problem seems to be that nobody has already included something like this in Erlang/OTP, whereas there is java.util.concurrent.ConcurrentHashMap since Java 5. That is an issue with the standard library, but not a flaw of the Actor model. This "clever stuff" is present in ConcurrentHashMap, too.
mustpax 15y ago
Three words: distributed hash table. Erlang has numerous well worn libraries and packages that provide this functionality. The fact that you are even thinking about creating a single process to handle all lookups means you don't understand message passing based concurrency yet. This article takes a deeply interesting problem (paradigms in distributed computing) and manages to discuss nothing of substance.
sofuture 15y ago
The problem with Erlang, is it doesn’t let you do that. Erlang is religious about concurrency. Erlang allows any concurrency model as long as it’s actors. Unlike Java/C++/Scala it doesn’t provide concurrency constructs that allow you to create other concurrency models. It works at a higher level of abstraction. That's not even close to true. Getting different building blocks from a language/runtime is not the same as not getting any building blocks. Erlang absolutely thrashes the JVM with it's concurrency features, and easily extends across cores and machines like not much else.
- tbrownaw 15y ago
  > That's not even close to true. Getting different building blocks from a language/runtime is not the same as not getting any building blocks. As an example of this, how to write a mutex in erlang: http://james-iry.blogspot.com/2009/04/erlang-style-actors-are-all-about_16.html http://james-iry.blogspot.com/2009/04/erlang-style-actors-ar...
schleyfox 15y ago
The example shows that the author really doesn't understand what concurrency is for Erlang. A concurrent data structure in Java may be more convenient and have lower overhead and complexity for one OS process on one box. Erlang isn't designed for that sort of problem, but makes it very simple to distribute across many machines. Concurrency in Erlang is a result of the goals of distribution in the pursuit of fault tolerance rather than the endgame.
evgen 15y ago
If anyone would like to see a nice example of how someone who actually knows Erlang would approach the same sort of problem that they OP tried and failed to examine in "the Erlang way" you would be well-advised to check out Jay Nelson' "Concurrent Caching" paper ( http://www.duomark.com/erlang/publications/acm2006.pdf http://www.duomark.com/erlang/publications/acm2006.pdf). If one were to do the same thing today there would be a lot more third-party libraries to lean on for additional functionality within the solution-space and to enhance the feature set.
olov 15y ago
I found the blog comments far more interesting than the post itself.

33 points by fogus 15y ago

tl;dr: The Actor model of concurrency used in Erlang doesn't lend itself well to key/value caches. The fact that Mnesia isn't mentioned once indicates the author is insufficiently knowledgeable about Erlang to write an informed article. This isn't an ad hominem attack because he didn't even mention Mnesia, hence this is an attack on the content of the article rather than his credentials. (This rather anal mention of ad hominem is relevant because the author brings it up in the comments). To those that follow, you'll learn more reading: Erlang - mnesia (Erlang documentation) http://www.erlang.org/doc/man/mnesia.html http://www.erlang.org/doc/man/mnesia.html CAP theorem https://secure.wikimedia.org/wikipedia/en/wiki/CAP_theorem https://secure.wikimedia.org/wikipedia/en/wiki/CAP_theorem And, of course, if you want to use something that someone has "just done", you would simply use Riak, which uses Erlang to implement an Amazon Dynamo type distributed key-value store: http://wiki.basho.com/ http://wiki.basho.com/ Not that difficult. A few Google's and you're set. EDIT: On reading my post I realise that Mnesia is an OTP application, rather than an Erlang built-in, and that Riak is an Erlang-based 3rd party library. However, using canonical libraries is part-and-parcel with learning about a language, IMO.

jerf 15y ago
MNesia is basically a built-in, inasmuch as I'm not sure you can find an Erlang install anywhere that doesn't have it (barring somebody going in and specifically plinking it). It ships in the open source release. (OTP application just means that it conforms to the OTP interfaces, so for instance you can "application:start(mnesia)", or depend on it in other OTP applications such as ones you've authored, etc.) Riak is legitimately a third party library, but does provide an existence proof that the specific problem chosen to highlight Erlang's weaknesses does still seem to work. That said, the author has a point. But by privileging actors as much as it does, Erlang gets a lot of great stuff out of it that have no effective competition in other languages, OTP in particular. Haskell may offer every sensible concurrency construct under the sun, but you don't get OTP to speak of. (Some efforts are underway, but they are new and I'd be surprised if they are stable yet.) Erlang is definitely niche, but it fits that niche very well and the niche is larger than it may initially appear... but it certainly isn't the best choice for all purposes. Nothing is, and nothing can be. The argument is a bit klunky but the point is still ultimately correct.

signa11 15y ago

another point to mention here is that you can always partition the keys (as the author points out). however, imho, author errs w.r.t simplicity of usage in java vs in erlang. once things are wrapped up in a module, it should be almost similar. on a side note, ConcurrentHashMap in java also ends up using a hashing scheme to locate the appropriate segment for queries. another approach to partitioning the keys would be to use a bloom-filter. nodes can periodically exchange their filter-databases either in its entirety or a compressed delta etc.

wmf 15y ago

There's no one true approach to concurrency, so any language that only provides one style is not a general-purpose language. IMO that's fine as long as people understand what they're getting into.

tbrownaw 15y ago
The lack of the One True Concurrency Model is because different models are faster or simpler for solving certain problems, not because some models are incapable of being used to solve some problems.

beza1e1 15y ago

The thesis of the author is: There is at least one problem, which can be done better with a ConcurrentHashMap than with Actors. I do not see, why you cannot replicate the ConcurrentHashMap behavior in Erlang? > Now, you could do some clever stuff and partition the map up into N actual maps, one for each of N processes and farm the requests out to the set of processes based on the hash-code. But that’s getting complex[...]. Well, his problem seems to be that nobody has already included something like this in Erlang/OTP, whereas there is java.util.concurrent.ConcurrentHashMap since Java 5. That is an issue with the standard library, but not a flaw of the Actor model. This "clever stuff" is present in ConcurrentHashMap, too.

mustpax 15y ago

Three words: distributed hash table. Erlang has numerous well worn libraries and packages that provide this functionality. The fact that you are even thinking about creating a single process to handle all lookups means you don't understand message passing based concurrency yet. This article takes a deeply interesting problem (paradigms in distributed computing) and manages to discuss nothing of substance.

sofuture 15y ago

The problem with Erlang, is it doesn’t let you do that. Erlang is religious about concurrency. Erlang allows any concurrency model as long as it’s actors. Unlike Java/C++/Scala it doesn’t provide concurrency constructs that allow you to create other concurrency models. It works at a higher level of abstraction. That's not even close to true. Getting different building blocks from a language/runtime is not the same as not getting any building blocks. Erlang absolutely thrashes the JVM with it's concurrency features, and easily extends across cores and machines like not much else.

tbrownaw 15y ago
> That's not even close to true. Getting different building blocks from a language/runtime is not the same as not getting any building blocks. As an example of this, how to write a mutex in erlang: http://james-iry.blogspot.com/2009/04/erlang-style-actors-are-all-about_16.html http://james-iry.blogspot.com/2009/04/erlang-style-actors-ar...

schleyfox 15y ago

The example shows that the author really doesn't understand what concurrency is for Erlang. A concurrent data structure in Java may be more convenient and have lower overhead and complexity for one OS process on one box. Erlang isn't designed for that sort of problem, but makes it very simple to distribute across many machines. Concurrency in Erlang is a result of the goals of distribution in the pursuit of fault tolerance rather than the endgame.

evgen 15y ago

If anyone would like to see a nice example of how someone who actually knows Erlang would approach the same sort of problem that they OP tried and failed to examine in "the Erlang way" you would be well-advised to check out Jay Nelson' "Concurrent Caching" paper ( http://www.duomark.com/erlang/publications/acm2006.pdf http://www.duomark.com/erlang/publications/acm2006.pdf). If one were to do the same thing today there would be a lot more third-party libraries to lean on for additional functionality within the solution-space and to enhance the feature set.

olov 15y ago

I found the blog comments far more interesting than the post itself.