9 ms·

Scaling Erlang to run on machines with hundreds of thousands of cores

75 points by adg001 15y ago

karterk 15y ago
Direct link to the project: http://www.release-project.eu/ http://www.release-project.eu/ Synopsis of the project: http://www.macs.hw.ac.uk/~trinder/RELEASEfactsheet.pdf http://www.macs.hw.ac.uk/~trinder/RELEASEfactsheet.pdf
wmf 15y ago
The description is a little confusing; they're talking about clusters, not really machines.
- rdtsc 15y ago
  They talk about everything. Right now the logical hierarchy of concurrency units goes something like: host : node : process So your cluster can have multiple hosts, each running multiple nodes, each node running multiple processes. All is handled transparently by the language syntax -- just sending a message to a process. I can see the problem being solved on how can these nodes and processes on each host take advantage of multiple cores on that host. Erlang processes are not backed by OS threads in a one-to-one fashion. So how would one allocate these cores to running processes and also how to load balance between 2 nodes for example. One node has 10 processes, the other has 100000 processes. There are 1000 cores available. How to best make use of those cores.
- nirvana 15y ago
  In erlang there is not necessarily any difference between a machine and a core. If you have a CPU with 100,000 cores, then you could run the same erlang program on it that you run on a cluster with 100,000 machines with the only changes being addressing the nodes on the cores rather than over the network.
  - pjscott 15y ago
    No difference in the programming model, perhaps, but the internals are very different.
gtani 15y ago
" near-perfect scalability of the Big Bang benchmark on a simulated 128-core SPARC" http://erlang.org/pipermail/erlang-questions/2011-July/060297.html http://erlang.org/pipermail/erlang-questions/2011-July/06029... Also, sounds similar to grant to Scala team, Jan 11 http://www.scala-lang.org/node/8579 http://www.scala-lang.org/node/8579
nirvana 15y ago
Erlang is the future, and the past of concurrency. Any developer worth their salt really should be learning it. If you will take a bit of time to do so, you'll discover that rather than the incomprehensible scary pain you expect when you first see it, it is quite logical, and quite a joy to work with. In fact, it is a litmus test for me. When I first started looking at NoSQL solutions, I quickly heard of MongoDB. When I found out it was written in C, I suspected that it wouldn't be distributed and it wouldn't be concurrent... and in researching further, those suspicions came true. (by "distributed" I mean, homogeneously like Riak, not the brittle master-server-sharding setup.) Trying to write a truly distributed, concurrent application in C is an exercise in frustration-- you'll have to produce halfway done poor implementation of erlang to do it, or you'll deal with pain every single day. But if you start with erlang, you get these features essentially for free. Plus a couple decades of seriously engineered OTP to go with it. I know a lot of "developers" these days never use anything more .... intense.... than a scripting language. But if you're ready to dive into the deep end, it isn't replicating the experience of writing assembly in C, it is going towards the future-- erlang.
- mxavier 15y ago
  I'm fairly new when it comes to working with Erlang. We've developed an application we use internally for essentially doing distributed, fault-tolerant timers. I, however, would caution against some of the platitudes you use about it being the future of concurrency. I also think it is kind of silly to imply that a coder isn't "worth their salt" if they aren't learning Erlang. It isn't a panacea. If you read some modern Erlang guides like Learn You Some Erlang, for example, you'll note that a responsible advocate of Erlang will caution against drinking too much kool-aid. Erlang, for example, is admitted as being a poor choice for lots of string manipulation because of the way strings are implemented in the language. I believe that same guide mentions that it may not be the best choice for heavy number crunching. Developers worth their salt will not just hear the word concurrency and start writing everything in Erlang. It is much better to consider the problem and ensure that the problem domain matches Erlang's particular strengths. I also want to mention that in my experience, Erlang applications are pure hell to deploy for people who don't have years of OTP deployment under their belts. So much of having a manageable Erlang app depends on the developers being well versed in a lot of subtle, and very easy to get wrong conventions. We actually ended up needing to start a support contract with Erlang Solutions themselves to help us demystify getting a sane deployment process going for our app. This was an expensive endeavor. Yes, it was primarily because we were new to the language, but it also had a lot to do with a lack of community documentation/discussion about deployment procedures. I won't even get into the joy of trying to decipher Erlang's famously cryptic crash reports/stack traces. tl;dr: Erlang isn't concurrency pixie dust. A developer isn't a failure if they aren't currently learning it, nor are they a success if they just decide to use Erlang for every class of problem. It is an interesting language to work with and solves certain problems very well, others not so well.
tsotha 15y ago
How big is the demand for applications that run on hundreds of thousands of cores? I can think of a few things, like weather prediction, but is this really a sizable market?

75 points by adg001 15y ago

Direct link to the project: http://www.release-project.eu/ http://www.release-project.eu/ Synopsis of the project: http://www.macs.hw.ac.uk/~trinder/RELEASEfactsheet.pdf http://www.macs.hw.ac.uk/~trinder/RELEASEfactsheet.pdf

wmf 15y ago

The description is a little confusing; they're talking about clusters, not really machines.

rdtsc 15y ago
They talk about everything. Right now the logical hierarchy of concurrency units goes something like: host : node : process So your cluster can have multiple hosts, each running multiple nodes, each node running multiple processes. All is handled transparently by the language syntax -- just sending a message to a process. I can see the problem being solved on how can these nodes and processes on each host take advantage of multiple cores on that host. Erlang processes are not backed by OS threads in a one-to-one fashion. So how would one allocate these cores to running processes and also how to load balance between 2 nodes for example. One node has 10 processes, the other has 100000 processes. There are 1000 cores available. How to best make use of those cores.
nirvana 15y ago
In erlang there is not necessarily any difference between a machine and a core. If you have a CPU with 100,000 cores, then you could run the same erlang program on it that you run on a cluster with 100,000 machines with the only changes being addressing the nodes on the cores rather than over the network.
- pjscott 15y ago
  No difference in the programming model, perhaps, but the internals are very different.

gtani 15y ago

" near-perfect scalability of the Big Bang benchmark on a simulated 128-core SPARC" http://erlang.org/pipermail/erlang-questions/2011-July/060297.html http://erlang.org/pipermail/erlang-questions/2011-July/06029... Also, sounds similar to grant to Scala team, Jan 11 http://www.scala-lang.org/node/8579 http://www.scala-lang.org/node/8579

nirvana 15y ago

Erlang is the future, and the past of concurrency. Any developer worth their salt really should be learning it. If you will take a bit of time to do so, you'll discover that rather than the incomprehensible scary pain you expect when you first see it, it is quite logical, and quite a joy to work with. In fact, it is a litmus test for me. When I first started looking at NoSQL solutions, I quickly heard of MongoDB. When I found out it was written in C, I suspected that it wouldn't be distributed and it wouldn't be concurrent... and in researching further, those suspicions came true. (by "distributed" I mean, homogeneously like Riak, not the brittle master-server-sharding setup.) Trying to write a truly distributed, concurrent application in C is an exercise in frustration-- you'll have to produce halfway done poor implementation of erlang to do it, or you'll deal with pain every single day. But if you start with erlang, you get these features essentially for free. Plus a couple decades of seriously engineered OTP to go with it. I know a lot of "developers" these days never use anything more .... intense.... than a scripting language. But if you're ready to dive into the deep end, it isn't replicating the experience of writing assembly in C, it is going towards the future-- erlang.

mxavier 15y ago
I'm fairly new when it comes to working with Erlang. We've developed an application we use internally for essentially doing distributed, fault-tolerant timers. I, however, would caution against some of the platitudes you use about it being the future of concurrency. I also think it is kind of silly to imply that a coder isn't "worth their salt" if they aren't learning Erlang. It isn't a panacea. If you read some modern Erlang guides like Learn You Some Erlang, for example, you'll note that a responsible advocate of Erlang will caution against drinking too much kool-aid. Erlang, for example, is admitted as being a poor choice for lots of string manipulation because of the way strings are implemented in the language. I believe that same guide mentions that it may not be the best choice for heavy number crunching. Developers worth their salt will not just hear the word concurrency and start writing everything in Erlang. It is much better to consider the problem and ensure that the problem domain matches Erlang's particular strengths. I also want to mention that in my experience, Erlang applications are pure hell to deploy for people who don't have years of OTP deployment under their belts. So much of having a manageable Erlang app depends on the developers being well versed in a lot of subtle, and very easy to get wrong conventions. We actually ended up needing to start a support contract with Erlang Solutions themselves to help us demystify getting a sane deployment process going for our app. This was an expensive endeavor. Yes, it was primarily because we were new to the language, but it also had a lot to do with a lack of community documentation/discussion about deployment procedures. I won't even get into the joy of trying to decipher Erlang's famously cryptic crash reports/stack traces. tl;dr: Erlang isn't concurrency pixie dust. A developer isn't a failure if they aren't currently learning it, nor are they a success if they just decide to use Erlang for every class of problem. It is an interesting language to work with and solves certain problems very well, others not so well.

tsotha 15y ago

How big is the demand for applications that run on hundreds of thousands of cores? I can think of a few things, like weather prediction, but is this really a sizable market?