4 ms·

I think the "more cores == faster" benefit is overstated. Without jit Erlang is so slow that languages without concurrency support will kill it. It's even more

I think the "more cores == faster" benefit is overstated. Without jit Erlang is so slow that languages without concurrency support will kill it. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. Not to mention it's 10 years later and we haven't seen massively parallel architectures take off. I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.

zitterbewegung 10y ago
The ways that massively parallel architectures have taken off are in GPU programming and they interact with them using the languages that they would have already done. C/C++ libraries that either use them directly or provide an interface to Python / Lua / Java etc...
davidw 10y ago
> fold the good parts of Erlang/BEAM into them A lot easier said than done, because there are a lot of really fundamental decisions about Erlang that make it what it is. You can't just slap on a scheduler for instance, or Erlang style processes. Go is probably the closest thing, and it's still lacking some things like the supervision tree. Erlang is certainly not 'the answer' in terms of computationally intensive anything, for the foreseeable future.
- djKianoosh 10y ago
  so many people (including me from time to time) misunderstand "computaionally expensive" that I think Elixir/erlang/beam will serve a lot of systems/apps well for a long time until they really see a need for "intense" computation
  - davidw 10y ago
    Oh, absolutely - you can do tons of things with Erlang, just as people do with Ruby, Python, PHP, Tcl, etc... But that article is a bit hype-y.
    - djKianoosh 10y ago
      well, of course it is ;) consider the source.
- lliamander 10y ago
  > Go is probably the closest thing, and it's still lacking some things like the supervision tree. From what I've read, Go's runtime just isn't compatible with the kind fault-tolerance that Erlang provides (nor was it meant to). It lacks process monitoring, lacks process isolation, has a pause-the-world garbage collector, etc. Not a knock against Go, just pointing out that the design goals were different.
jcoffland 10y ago
Agreed. I don't think there is a big fuss about Erlang anymore. Still it's an interesting language from an acidemic perspective if not from a practical one.
- lliamander 10y ago
  Erlang is certainly inaccessible, but I don't think academic is really the right word for it. It was born out of very practical business needs, and the reason it is still around is mainly because it has done an excellent job of serving those use cases. What they haven't done (or to the extent that other languages have) is make the language and runtime accessible to the broader programming community. The result is that the people who (start to) use Erlang are the kind of people for whom no other tool will help them solve their problems. Although I have not used Elixir, it seems to me that it was designed as a language that is easy enough to adopt for the wider programming community, while still leveraging the concurrency, fault-tolerance, etc. provided by BEAM and OTP. On top of that, the Elixir community has worked hard to provide the tooling and support developers typically expect these days[0]. This is not to say that Erlang should be radically altered or abandoned. I like the Erlang aesthetic and ethos, and I think the community knows the way it needs to evolve. It is just that I expect more people to come to BEAM and OTP through Elixir than Erlang, and I think it is that platform that should really be the focus. [0] to their credit, the Erlang community has definitely improved in this area.
momerath 10y ago
Unfortunately, I think you're correct. Anecdote: I used Erlang, over the course of several years, for very-high-level bot behavior and multi-player mission control, where performance was much less important than the things Erlang provides, but I still had issues with it. Nonetheless, I felt it had given me great leverage, and I thanked Joe Armstrong profusely, when I met him at a conference. Later at the con, I asked a panel whether there were active efforts at improving performance, perhaps with a JIT. They seemed to take it as an attack, and suggested that if I need performance, I should use another language. I mostly write in Clojure now.
- im_down_w_otp 10y ago
  (facepalm) that somewhat closely mirrors my experience. It's a bit baffling.
tormeh 10y ago
The slowness of Erlang was a conscious decision. You're supposed to offload all the computationally intensive things to native code. That may not be ideal for your use case and a PITA, but it's how the language and runtime was designed. I would guess Erlang would do (if it isn't already doing) well in HPC as the thing that coordinates all the Fortran code.
stcredzero 10y ago
I think the "more cores == faster" benefit is overstated. I've tried to create servers for massively multiplayer games in Clojure and Go. What I've found is that it "takes some doing" to be able to efficiently use many processors in parallel. You can't just slap on a pmap call in Clojure or do the equivalent cheesy channel trick in Golang and have good utilization of all your cores in parallel. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. For what value of "mostly scale as well?" For me, that means doing a little extra design/architecting to break out your Golang server process into multiple worker processes. it's 10 years later and we haven't seen massively parallel architectures take off. Because there are still barriers that are a bit too large. You either wind up having to do weird things to do what amounts to efficiently passing data between cores, or you end up using a slower language that's a bit far from the computing mainstream.
- deleted 10y ago
  [deleted]
- zazibar 10y ago
  Can you elaborate on how you implemented those servers? Would you still use Clojure or Go for them now? If not, what alternative do you suggest?
  - stcredzero 10y ago
    My first attempt was a server for a dungeon crawler that superficially looked like a rogue-like, but was actually 12 frames/sec realtime instead of turn based. It started out on Clojure, but I afterwards ported it to Go. In this server, the world was divided into 80x24 subgrids that did most of their processing in parallel. Processing would happen in two stages: 1) local data processing, where each entity got updated in a loop, sending out updates to clients and 2) collision resolution/movement. If I had to do it over again, I would relax the grid's invariant properties, and make each 80x24 subgrid completely independent, which would eliminate the collision resolution step. In my current server, I'm basically sharding my space-game into star systems, with free travel between star systems though "Hyperspace." There is a farm of "worker processes" that is coordinated by a master server. Any instance of a star system can be idempotently spawned when a client is attached to it. < https://www.emergencevector.com https://www.emergencevector.com > I think Go is a good option for writing a game server. You'd have to be pretty profligate to make the GC pause overrun one 16.66 ms tick, and if you miss a tick here and there, who cares?
gamache 10y ago
> ...It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so... > I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption. Comparing a bare-metal-but-with-GC language like Go to a miniature operating system like Erlang isn't quite fair -- in comparison, Go is missing an actor model, buffered CSP, crash supervision, hot upgrades, remote console/tracing/debugging, and built-in clustering (to name a few). Once other languages adopt "enough" of the BEAM featureset, they'll have similar performance.
- aaron-lebo 10y ago
  Hot upgrades and remote debugging are front in center in lisp and it has excellent performance. Even so, there's no general consensus among programmers about whether hot code reloading is preferred to static binaries. The actor model is a choice of implementation. Crash supervision stands out, but that's not necessary and is part of the Erlang culture - let it crash. It's not as important in other languages with a different underlying belief about errors. What features mean BEAM must necessarily be slow? Just to be clear: I really like Erlang. It obviously has different features than a lot of languages and most are aimed towards a certain goal: uptime. But if you can give just a little on that (and you might not have to at all), you can get most of those advantages with a different feature set. I wonder out of the Erlang experiment which are the objectively better features and which are just really cool?
rdtsc 10y ago
> I think the "more cores == faster" benefit is overstated. I find it is not. But of course the right tool for the right job. Multiplying matrices as a single batch process is probably not going to work very well. But I saw it works exceptionally for highly concurrent workload. This is not just "I ran a quick benchmarks on my laptop and then extrapolated by using an multiplier". Distributed systems work funny and they don't scale linearly. Sometimes a very fast, small sequential program will hit a wall and barf on itself when faced with multiple concurrent requests. The other secret sauce is not multi-core capability but process isolation. The reason is this: a large distributed system is only as useful / practical as it is fault tolerant. It doesn't matter if you run hand tunes assembly routine and process 1M requests per second, then it crashes every day and you have hours of debugging and downtime. The average throughput or uptime will be pretty bad. This is one key insight into what makes Erlang (and Elixir) a good tool for distributed systems - fault tolerance is built into the core of the platform.
tempz 10y ago
Erlang is made for IoT. Tens of millions of concurrent unrelated clients, embarrassingly parallelizable, scales linearly with number of cores, as these share nothing.

zitterbewegung 10y ago

The ways that massively parallel architectures have taken off are in GPU programming and they interact with them using the languages that they would have already done. C/C++ libraries that either use them directly or provide an interface to Python / Lua / Java etc...

davidw 10y ago

> fold the good parts of Erlang/BEAM into them A lot easier said than done, because there are a lot of really fundamental decisions about Erlang that make it what it is. You can't just slap on a scheduler for instance, or Erlang style processes. Go is probably the closest thing, and it's still lacking some things like the supervision tree. Erlang is certainly not 'the answer' in terms of computationally intensive anything, for the foreseeable future.

djKianoosh 10y ago
so many people (including me from time to time) misunderstand "computaionally expensive" that I think Elixir/erlang/beam will serve a lot of systems/apps well for a long time until they really see a need for "intense" computation
- davidw 10y ago
  Oh, absolutely - you can do tons of things with Erlang, just as people do with Ruby, Python, PHP, Tcl, etc... But that article is a bit hype-y.
  - djKianoosh 10y ago
    well, of course it is ;) consider the source.
lliamander 10y ago
> Go is probably the closest thing, and it's still lacking some things like the supervision tree. From what I've read, Go's runtime just isn't compatible with the kind fault-tolerance that Erlang provides (nor was it meant to). It lacks process monitoring, lacks process isolation, has a pause-the-world garbage collector, etc. Not a knock against Go, just pointing out that the design goals were different.

jcoffland 10y ago

Agreed. I don't think there is a big fuss about Erlang anymore. Still it's an interesting language from an acidemic perspective if not from a practical one.

lliamander 10y ago
Erlang is certainly inaccessible, but I don't think academic is really the right word for it. It was born out of very practical business needs, and the reason it is still around is mainly because it has done an excellent job of serving those use cases. What they haven't done (or to the extent that other languages have) is make the language and runtime accessible to the broader programming community. The result is that the people who (start to) use Erlang are the kind of people for whom no other tool will help them solve their problems. Although I have not used Elixir, it seems to me that it was designed as a language that is easy enough to adopt for the wider programming community, while still leveraging the concurrency, fault-tolerance, etc. provided by BEAM and OTP. On top of that, the Elixir community has worked hard to provide the tooling and support developers typically expect these days[0]. This is not to say that Erlang should be radically altered or abandoned. I like the Erlang aesthetic and ethos, and I think the community knows the way it needs to evolve. It is just that I expect more people to come to BEAM and OTP through Elixir than Erlang, and I think it is that platform that should really be the focus. [0] to their credit, the Erlang community has definitely improved in this area.

momerath 10y ago

Unfortunately, I think you're correct. Anecdote: I used Erlang, over the course of several years, for very-high-level bot behavior and multi-player mission control, where performance was much less important than the things Erlang provides, but I still had issues with it. Nonetheless, I felt it had given me great leverage, and I thanked Joe Armstrong profusely, when I met him at a conference. Later at the con, I asked a panel whether there were active efforts at improving performance, perhaps with a JIT. They seemed to take it as an attack, and suggested that if I need performance, I should use another language. I mostly write in Clojure now.

im_down_w_otp 10y ago
(facepalm) that somewhat closely mirrors my experience. It's a bit baffling.

tormeh 10y ago

The slowness of Erlang was a conscious decision. You're supposed to offload all the computationally intensive things to native code. That may not be ideal for your use case and a PITA, but it's how the language and runtime was designed. I would guess Erlang would do (if it isn't already doing) well in HPC as the thing that coordinates all the Fortran code.

stcredzero 10y ago

I think the "more cores == faster" benefit is overstated. I've tried to create servers for massively multiplayer games in Clojure and Go. What I've found is that it "takes some doing" to be able to efficiently use many processors in parallel. You can't just slap on a pmap call in Clojure or do the equivalent cheesy channel trick in Golang and have good utilization of all your cores in parallel. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. For what value of "mostly scale as well?" For me, that means doing a little extra design/architecting to break out your Golang server process into multiple worker processes. it's 10 years later and we haven't seen massively parallel architectures take off. Because there are still barriers that are a bit too large. You either wind up having to do weird things to do what amounts to efficiently passing data between cores, or you end up using a slower language that's a bit far from the computing mainstream.

deleted 10y ago
[deleted]
zazibar 10y ago
Can you elaborate on how you implemented those servers? Would you still use Clojure or Go for them now? If not, what alternative do you suggest?
- stcredzero 10y ago
  My first attempt was a server for a dungeon crawler that superficially looked like a rogue-like, but was actually 12 frames/sec realtime instead of turn based. It started out on Clojure, but I afterwards ported it to Go. In this server, the world was divided into 80x24 subgrids that did most of their processing in parallel. Processing would happen in two stages: 1) local data processing, where each entity got updated in a loop, sending out updates to clients and 2) collision resolution/movement. If I had to do it over again, I would relax the grid's invariant properties, and make each 80x24 subgrid completely independent, which would eliminate the collision resolution step. In my current server, I'm basically sharding my space-game into star systems, with free travel between star systems though "Hyperspace." There is a farm of "worker processes" that is coordinated by a master server. Any instance of a star system can be idempotently spawned when a client is attached to it. < https://www.emergencevector.com https://www.emergencevector.com > I think Go is a good option for writing a game server. You'd have to be pretty profligate to make the GC pause overrun one 16.66 ms tick, and if you miss a tick here and there, who cares?

gamache 10y ago

> ...It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so... > I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption. Comparing a bare-metal-but-with-GC language like Go to a miniature operating system like Erlang isn't quite fair -- in comparison, Go is missing an actor model, buffered CSP, crash supervision, hot upgrades, remote console/tracing/debugging, and built-in clustering (to name a few). Once other languages adopt "enough" of the BEAM featureset, they'll have similar performance.

aaron-lebo 10y ago
Hot upgrades and remote debugging are front in center in lisp and it has excellent performance. Even so, there's no general consensus among programmers about whether hot code reloading is preferred to static binaries. The actor model is a choice of implementation. Crash supervision stands out, but that's not necessary and is part of the Erlang culture - let it crash. It's not as important in other languages with a different underlying belief about errors. What features mean BEAM must necessarily be slow? Just to be clear: I really like Erlang. It obviously has different features than a lot of languages and most are aimed towards a certain goal: uptime. But if you can give just a little on that (and you might not have to at all), you can get most of those advantages with a different feature set. I wonder out of the Erlang experiment which are the objectively better features and which are just really cool?

rdtsc 10y ago

> I think the "more cores == faster" benefit is overstated. I find it is not. But of course the right tool for the right job. Multiplying matrices as a single batch process is probably not going to work very well. But I saw it works exceptionally for highly concurrent workload. This is not just "I ran a quick benchmarks on my laptop and then extrapolated by using an multiplier". Distributed systems work funny and they don't scale linearly. Sometimes a very fast, small sequential program will hit a wall and barf on itself when faced with multiple concurrent requests. The other secret sauce is not multi-core capability but process isolation. The reason is this: a large distributed system is only as useful / practical as it is fault tolerant. It doesn't matter if you run hand tunes assembly routine and process 1M requests per second, then it crashes every day and you have hours of debugging and downtime. The average throughput or uptime will be pretty bad. This is one key insight into what makes Erlang (and Elixir) a good tool for distributed systems - fault tolerance is built into the core of the platform.

tempz 10y ago

Erlang is made for IoT. Tens of millions of concurrent unrelated clients, embarrassingly parallelizable, scales linearly with number of cores, as these share nothing.