6 ms·

In the 200Xs, the BEAM VM had a clear performance advantage in dealing with high-concurrency network loads. It has never had a raw performance advantage in term

by jerf 5y ago

In the 200Xs, the BEAM VM had a clear performance advantage in dealing with high-concurrency network loads. It has never had a raw performance advantage in terms of the bytecode that it implemented, that is, the Erlang/Elixir layer (it was generally faster than Python/Ruby, but that wasn't saying much, especially back then, but it had clear performance disadvantages vs. C/C++/Java), but it had a superior internal runtime that could make up for that in performance benchmarks, as long as you didn't try to run too much BEAM bytecode. Much like how NumPy is very fast, as long as you don't try to run too much pure Python with it. However, since BEAM doesn't have access to unique CPU instructions that nobody else has or anything else, and since a lot of focus across a lot of languages has been put on that problem, that particular advantage has waned, and Erlang to my eyes has indeed been outright passed on this front by multiple languages. In the 200Xs, I did not see people talking much about NIFs as a solution for performance; that talk has started as an effort to keep up with things like Go and other languages that have taken advantage of BEAM's lessons and explorations of the space. Personally, while I think a lot of the hoopla surrounding Erlang/Elixir isn't wrong per se, I do think a lot of it is outdated. They'll say "We do X and nobody else does!" but while that may have been true 10-15 years ago, it isn't anymore. There's no performance reason to pick Erlang/Elixir over Go, for instance, and if you take the models of memory access back to Go there isn't a huge organizational reason either. What Erlang/Elixir force you to do, you can voluntarily do in other languages too. And I think that's become true across a lot of other of the putative "advantages"; it isn't that Erlang/Elixir aren't nice in some ways, but I do wonder how much of the recent push is stemming from people who are experiencing some of these capabilities for the first time and trusting the Erlang/Elixir storyline that they're unique, when in fact they are increasingly just table stakes for a new language nowadays rather than special characteristics unique to the BEAM family of languages.

lostcolony 5y ago
'What Erlang/Elixir force you to do, you can voluntarily do in other languages too'. Like immutability! :P Jokes aside, I agree for basic concurrency you can get pretty far with other modern languages. I think it's what brings people's attention to Erlang/Elixir, but I don't think it's the most important differentiator. It also isn't the one that Erlang's community (I can't speak to Elixir) really touts, except as one that is easily understood by those outside of it. The real benefit is fault tolerance. Everything about Erlang, the concurrency and distribution stories included, is built around fault tolerance. You need concurrency and distribution to be fault tolerant (can't have one bad process choking out others; can't have one bad machine taking the service down, etc). The immutability, the supervision tree, those also are about fault tolerance. I've written production systems in Go. It scaled better with way less tweaking than the JVM based stuff we'd written previously required. But it wasn't nearly as resilient to failure, or as predictable, as the Erlang stuff I've run in prod was.
- jerf 5y ago
  The funny thing is that precisely what got me into Go was replacing an Erlang system that was constantly falling over despite quite considerable efforts with a Go system that ran on a fraction of the resources, ran much more quickly, and by comparison was rock-solid. I just ported the essence of supervision trees in to Go and was off to the races. This is part of what I mean; Erlang doesn't have access to special CPU instructions that make supervision trees possible. They're just code. There's no reason you can write them in other languages. It's not even particularly hard, unless you insist on exactly matching all the accidental details of the way Erlang implements them instead of implementing their essential details in a manner idiomatic to the base language.
  - lostcolony 5y ago
    "It's not even particularly hard" Define hard here? Because there is a lot of bookkeeping involved, and, yes, to get some of the effects you have in Erlang that are necessary for reliability, you'd basically have to create your own runtime atop Go. I.e., yes, if you just want to "if this process fails, restart it", you can do that trivially in another language, but "if this process fails -kill these others and restart them from a known good state-" is devilishly hard, given that Go has no way to kill a running goroutine. You can send a message along a channel of "hey, you should stop", but if code execution in that goroutine never gets there, you have no guarantees. And while the CPU doesn't have special instructions, the VM -does-. Exit signals are guaranteed in the language spec; a colocated supervisor is guaranteed to be able to both detect a failed process, and to be able to kill others. Go offers no such tooling, let alone such guarantees. I'd be quite interested if you say you did that; I suspect it was, as mentioned, just "hey, if an error comes out of this response channel, restart the goroutine". Possibly also a "here's a channel we can send 'kill' commands on, downstream goroutines should check it occasionally to see if they should terminate". A lot of bookkeeping, no guarantees.
    - jerf 5y ago
      I think you're making a mistake a lot of Erlang/BEAM/etc. (let me call it just Erlang after this) advocates make, which is to conflate Erlang's solutions to problems with the only solutions to problems. Almost no software is written in a language that has the ability to actively kill running threads externally. This is not a catastrophic problem that causes the rest of us to routinely break down in tears; it is a thing that occasionally causes bugs, merely one on a long list of such things. On the scale of problems I have, this isn't even in my top 50. When systems are written with that understanding, it's only a minor roadbump. So the fact that I don't have Erlang's exact solution to that problem isn't even remotely worth me switching (back) to Erlang for. Is it a problem? Yes, absolutely. Is it a problem worth spending extremely valuable language design budget on? Heck no, and the fact that Erlang does is a negative to me, because what they gave up to get that capability is way more important to me. Does my solution exactly match Erlang? No. Of course not. But it gets me 90% of what I care about for 10% of the effort, and in the meantime I get the other things that directly impact my job on a minute-by-minute basis, like an even halfway decent type system (it's not like Go's is some sort of masterpiece here, but it's much better than BEAM's), which Erlang sacrificed as part of its original plan. I understand why they have the type system they have, and what they got out of it, and I'd rather have a decent static type system and solve those problems another way, which happens to be the conclusion pretty much everyone else has come to as well. Again, genius thinking in the 1990s, way ahead of everyone else, don't let my current assessment of Erlang diminish the fact I deeply respect what it did in its time... but not a solution I have much interest in in 2021.
querulous 5y ago
the elixir community is quite credulous. particularly when it comes to circa 200X era wisdom. people talk about the vm being the key to elixir/erlang and talk up things like lightweight green threads, message passing and the garbage collector but the truth is these are all of fairly low quality compared to other competing languages/implementations the real key for erlang and later elixir's success was pervasive async io. this was a genuine advantage during erlang's peak but languages, runtimes and libraries like go, node and nio have caught up and surpassed the erlang vm the truth is without that advantage almost everything in the erlang/elixir world is worse than more mainstream alternatives. there's some exceptions -- ecto is pretty good because it has one of the best written connection pooling implementations i've ever seen. i think both languages are pretty good and i still write the occasional thing in erlang for my own satisfaction but the world has moved on and elixir and especially erlang haven't innovated enough to have kept up
- divs1210 5y ago
  > these are all of fairly low quality compared to other competing languages/implementations Seriously? You say this without providing examples? I'm not aware of any other programming environment that has BEAM-like processes. Not Go, not Java, not any other system I know.
  - justsomeuser 5y ago
    What about functions as a service, like AWS lambda? If you use Node as an example, your code is JIT compiled to machine code, any single request can fail, and you can scale to any number of requests without thinking about the underlying OS or VM. Async/await will allow you to do a “blocking receive” like Erlangs processes.
    - pdimitar 5y ago
      Nah, BEAM is not suited well for that. It has a big startup time. BEAM is designed for long-running daemons, not "wake up, do small amount of work, get shut down". For something like this I'd choose Rust or OCaml due to their insanely fast cold startup time (if the program is a CLI tool). Erlang/BEAM is not there yet and it might not soon be.
hinkley 5y ago
> What Erlang/Elixir force you to do, you can voluntarily do in other languages too. This argument is older than dirt. I don't need Java, I can do all of this stuff in C (voluntarily). I don't need Y, I can do it in C++ voluntarily. You don't control your team. It doesn't matter what you are willing to volunteer if it doesn't work unless the whole team does it. If it only works if everyone has to do it, that's not voluntary now, is it? Every boundary that is by agreement only will constantly be pushed and pushed. Every time someone wants to leave early, or there's a production issue or a customer deadline, or they just don't want to. It's why size and speed are a constant fight at some places. Everything you fix is counteracted by ten other people who just don't care. Either the system has to enforce the rule or your coworkers become enforcers. That's a shit job to begin with, and doubly so for introverts, who will either do too little or too much in the face of boundary testers.

by jerf 5y ago

lostcolony 5y ago

'What Erlang/Elixir force you to do, you can voluntarily do in other languages too'. Like immutability! :P Jokes aside, I agree for basic concurrency you can get pretty far with other modern languages. I think it's what brings people's attention to Erlang/Elixir, but I don't think it's the most important differentiator. It also isn't the one that Erlang's community (I can't speak to Elixir) really touts, except as one that is easily understood by those outside of it. The real benefit is fault tolerance. Everything about Erlang, the concurrency and distribution stories included, is built around fault tolerance. You need concurrency and distribution to be fault tolerant (can't have one bad process choking out others; can't have one bad machine taking the service down, etc). The immutability, the supervision tree, those also are about fault tolerance. I've written production systems in Go. It scaled better with way less tweaking than the JVM based stuff we'd written previously required. But it wasn't nearly as resilient to failure, or as predictable, as the Erlang stuff I've run in prod was.

jerf 5y ago
The funny thing is that precisely what got me into Go was replacing an Erlang system that was constantly falling over despite quite considerable efforts with a Go system that ran on a fraction of the resources, ran much more quickly, and by comparison was rock-solid. I just ported the essence of supervision trees in to Go and was off to the races. This is part of what I mean; Erlang doesn't have access to special CPU instructions that make supervision trees possible. They're just code. There's no reason you can write them in other languages. It's not even particularly hard, unless you insist on exactly matching all the accidental details of the way Erlang implements them instead of implementing their essential details in a manner idiomatic to the base language.
- lostcolony 5y ago
  "It's not even particularly hard" Define hard here? Because there is a lot of bookkeeping involved, and, yes, to get some of the effects you have in Erlang that are necessary for reliability, you'd basically have to create your own runtime atop Go. I.e., yes, if you just want to "if this process fails, restart it", you can do that trivially in another language, but "if this process fails -kill these others and restart them from a known good state-" is devilishly hard, given that Go has no way to kill a running goroutine. You can send a message along a channel of "hey, you should stop", but if code execution in that goroutine never gets there, you have no guarantees. And while the CPU doesn't have special instructions, the VM -does-. Exit signals are guaranteed in the language spec; a colocated supervisor is guaranteed to be able to both detect a failed process, and to be able to kill others. Go offers no such tooling, let alone such guarantees. I'd be quite interested if you say you did that; I suspect it was, as mentioned, just "hey, if an error comes out of this response channel, restart the goroutine". Possibly also a "here's a channel we can send 'kill' commands on, downstream goroutines should check it occasionally to see if they should terminate". A lot of bookkeeping, no guarantees.
  - jerf 5y ago
    I think you're making a mistake a lot of Erlang/BEAM/etc. (let me call it just Erlang after this) advocates make, which is to conflate Erlang's solutions to problems with the only solutions to problems. Almost no software is written in a language that has the ability to actively kill running threads externally. This is not a catastrophic problem that causes the rest of us to routinely break down in tears; it is a thing that occasionally causes bugs, merely one on a long list of such things. On the scale of problems I have, this isn't even in my top 50. When systems are written with that understanding, it's only a minor roadbump. So the fact that I don't have Erlang's exact solution to that problem isn't even remotely worth me switching (back) to Erlang for. Is it a problem? Yes, absolutely. Is it a problem worth spending extremely valuable language design budget on? Heck no, and the fact that Erlang does is a negative to me, because what they gave up to get that capability is way more important to me. Does my solution exactly match Erlang? No. Of course not. But it gets me 90% of what I care about for 10% of the effort, and in the meantime I get the other things that directly impact my job on a minute-by-minute basis, like an even halfway decent type system (it's not like Go's is some sort of masterpiece here, but it's much better than BEAM's), which Erlang sacrificed as part of its original plan. I understand why they have the type system they have, and what they got out of it, and I'd rather have a decent static type system and solve those problems another way, which happens to be the conclusion pretty much everyone else has come to as well. Again, genius thinking in the 1990s, way ahead of everyone else, don't let my current assessment of Erlang diminish the fact I deeply respect what it did in its time... but not a solution I have much interest in in 2021.

querulous 5y ago

the elixir community is quite credulous. particularly when it comes to circa 200X era wisdom. people talk about the vm being the key to elixir/erlang and talk up things like lightweight green threads, message passing and the garbage collector but the truth is these are all of fairly low quality compared to other competing languages/implementations the real key for erlang and later elixir's success was pervasive async io. this was a genuine advantage during erlang's peak but languages, runtimes and libraries like go, node and nio have caught up and surpassed the erlang vm the truth is without that advantage almost everything in the erlang/elixir world is worse than more mainstream alternatives. there's some exceptions -- ecto is pretty good because it has one of the best written connection pooling implementations i've ever seen. i think both languages are pretty good and i still write the occasional thing in erlang for my own satisfaction but the world has moved on and elixir and especially erlang haven't innovated enough to have kept up

divs1210 5y ago
> these are all of fairly low quality compared to other competing languages/implementations Seriously? You say this without providing examples? I'm not aware of any other programming environment that has BEAM-like processes. Not Go, not Java, not any other system I know.
- justsomeuser 5y ago
  What about functions as a service, like AWS lambda? If you use Node as an example, your code is JIT compiled to machine code, any single request can fail, and you can scale to any number of requests without thinking about the underlying OS or VM. Async/await will allow you to do a “blocking receive” like Erlangs processes.
  - pdimitar 5y ago
    Nah, BEAM is not suited well for that. It has a big startup time. BEAM is designed for long-running daemons, not "wake up, do small amount of work, get shut down". For something like this I'd choose Rust or OCaml due to their insanely fast cold startup time (if the program is a CLI tool). Erlang/BEAM is not there yet and it might not soon be.

hinkley 5y ago

> What Erlang/Elixir force you to do, you can voluntarily do in other languages too. This argument is older than dirt. I don't need Java, I can do all of this stuff in C (voluntarily). I don't need Y, I can do it in C++ voluntarily. You don't control your team. It doesn't matter what you are willing to volunteer if it doesn't work unless the whole team does it. If it only works if everyone has to do it, that's not voluntary now, is it? Every boundary that is by agreement only will constantly be pushed and pushed. Every time someone wants to leave early, or there's a production issue or a customer deadline, or they just don't want to. It's why size and speed are a constant fight at some places. Everything you fix is counteracted by ten other people who just don't care. Either the system has to enforce the rule or your coworkers become enforcers. That's a shit job to begin with, and doubly so for introverts, who will either do too little or too much in the face of boundary testers.