4 ms·

Would it have been better if they went with Elixir? Write their code in a functional style. Get the benefits of the Erlang BEAM platform. Their system runs ov

Would it have been better if they went with Elixir? Write their code in a functional style. Get the benefits of the Erlang BEAM platform. Their system runs over the web, so time sensitivity isn’t as important, in comparison to video games, VR, or AR. Anyone ever done a performance comparison breakdown between something like Elixir vs. Rust?

steveklabnik 7y ago
Discord is a heavy Elixir user, and even uses it with Rust via NIF: https://blog.discordapp.com/using-rust-to-scale-elixir-for-11-million-concurrent-users-c6f19fc029d3 https://blog.discordapp.com/using-rust-to-scale-elixir-for-1...
jerf 7y ago
"Would it have been better if they went with Elixir?" No. It would have been unshippably bad. BEAM is generally fairly slow. It was fast at multitasking for a while, but that advantage has been claimed by several other runtimes in 2020. As a language, it is much slower than Rust. Plus, if you tried to implement a gigantic shared cache map in Erlang/Elixir, you'd have two major problems: One is that you'd need huge chunks of the map in single (BEAM) processes, and you'd get hit by the fact BEAM is not set up to GC well in that case. It wants lots of little processes, not a small number of processes holding tons of data. Second is that you'd be trading what in Rust is "accept some bytes, do some hashing, look some stuff up in memory" with generally efficient, low-copy operations, with "copy the network traffic into an Erlang binary, do some hashing, compute the PID that actually has the data, send a message to that PID with the request, wait for the reply message, and then send out the answer", with a whole lot of layers that expect to have time to make copies of lots of things. Adding this sort of coordination into these nominally fast lookups is going to slow this to a crawl. It's like when people try to benchmark Erlang/Elixir/Go's threading by creating processes/goroutines to receive two numbers and add them together "in parallel"; the IPC completely overshadows the tiny amount of work being done. (They mention tokio, but that's still going to add a lot less coordination overhead than Erlang messages.) Go is a significantly better language for this use case than Elixir/Erlang/BEAM is, let alone Rust. (This is not a "criticism" of Erlang/Elixir/BEAM. It's an engineering analysis. Erlang/Elixir/BEAM are still suitable for many tasks, just as people still use Python for many things despite the fact it would be a catastrophically bad choice for this particular task. This just isn't one of the tasks it would be suitable for.)
- sergiotapia 7y ago
  >It was fast at multitasking for a while, but that advantage has been claimed by several other runtimes in 2020. Such as?
  - jerf 7y ago
    Go, Rust is apparently getting there since it just smoked Go on a pretty core multitasking-heavy task here, and a lot of the old "stodgy" languages like Java while none of the cool kids were looking have gotten pretty good at large numbers of threads too. I'm also expecting some sort of sensible solution to this sort of concurrency challenge to simply be baseline expected functionality for the next generation of languages. Anyone sitting down in 2020 to write The Next Big Language who ignores the fact that by the time they're done, their CPUs are going to be 64-core and the GPUs will be several-hundred-thousand core is really going to be missing the boat. I actually encourage Erlang partisans to consider this a win in the general sense. Quite serious. If you consider languages as a multi-decade conversation, almost everything Erlang "said" in the late 1990s and early 2000s is in fact proving out to be true. However, while Erlang was a trailblazer, it isn't going to be that Next Big Language, nor anything like it. It got a lot of things right, but it's got too much wrong to be the NBL, and even if you did the minimal fixes, the result wouldn't be backwards compatible with Erlang anymore so it'd be a new language. A lot of Erlang partisans are making the error of thinking the next language needs to be just like Erlang, except perhaps more so. But that's not how progress gets made. The good ideas are ripped out at a much more granular level and recombined with all sorts of other ideas and the end result may be quite different than what you expect. Go, for instance, can very much be seen as a direct sequel to Erlang among other things, even though it may not seem to have "OTP" or "built-in multinode concurrency" or whatever other apparent bullet-list features Erlang has, because a lot of those bullet-list features are really just epiphenomena of the real underlying features. Rust has its own nods to Erlang too; the whole "ownership" thing comes from the experiences with both mutation and immutability in a threading environment. But rather than making "Erlang, but a bit moreso", you get something that takes the lessons, ponders on them for another 15 years, and produces something different. It isn't Erlang, because at the time that Erlang was being written (Joe Armstrong RIP), nobody had the experience to think of Rust and imagine it could make a practical language. (Nor am I entirely sure the computers of the time could have compiled it; even if we sat through the clock time I'm not sure Rust could be stuffed into a 1997 desktop computer's RAM.)

steveklabnik 7y ago

Discord is a heavy Elixir user, and even uses it with Rust via NIF: https://blog.discordapp.com/using-rust-to-scale-elixir-for-11-million-concurrent-users-c6f19fc029d3 https://blog.discordapp.com/using-rust-to-scale-elixir-for-1...

jerf 7y ago

"Would it have been better if they went with Elixir?" No. It would have been unshippably bad. BEAM is generally fairly slow. It was fast at multitasking for a while, but that advantage has been claimed by several other runtimes in 2020. As a language, it is much slower than Rust. Plus, if you tried to implement a gigantic shared cache map in Erlang/Elixir, you'd have two major problems: One is that you'd need huge chunks of the map in single (BEAM) processes, and you'd get hit by the fact BEAM is not set up to GC well in that case. It wants lots of little processes, not a small number of processes holding tons of data. Second is that you'd be trading what in Rust is "accept some bytes, do some hashing, look some stuff up in memory" with generally efficient, low-copy operations, with "copy the network traffic into an Erlang binary, do some hashing, compute the PID that actually has the data, send a message to that PID with the request, wait for the reply message, and then send out the answer", with a whole lot of layers that expect to have time to make copies of lots of things. Adding this sort of coordination into these nominally fast lookups is going to slow this to a crawl. It's like when people try to benchmark Erlang/Elixir/Go's threading by creating processes/goroutines to receive two numbers and add them together "in parallel"; the IPC completely overshadows the tiny amount of work being done. (They mention tokio, but that's still going to add a lot less coordination overhead than Erlang messages.) Go is a significantly better language for this use case than Elixir/Erlang/BEAM is, let alone Rust. (This is not a "criticism" of Erlang/Elixir/BEAM. It's an engineering analysis. Erlang/Elixir/BEAM are still suitable for many tasks, just as people still use Python for many things despite the fact it would be a catastrophically bad choice for this particular task. This just isn't one of the tasks it would be suitable for.)

sergiotapia 7y ago
>It was fast at multitasking for a while, but that advantage has been claimed by several other runtimes in 2020. Such as?
- jerf 7y ago
  Go, Rust is apparently getting there since it just smoked Go on a pretty core multitasking-heavy task here, and a lot of the old "stodgy" languages like Java while none of the cool kids were looking have gotten pretty good at large numbers of threads too. I'm also expecting some sort of sensible solution to this sort of concurrency challenge to simply be baseline expected functionality for the next generation of languages. Anyone sitting down in 2020 to write The Next Big Language who ignores the fact that by the time they're done, their CPUs are going to be 64-core and the GPUs will be several-hundred-thousand core is really going to be missing the boat. I actually encourage Erlang partisans to consider this a win in the general sense. Quite serious. If you consider languages as a multi-decade conversation, almost everything Erlang "said" in the late 1990s and early 2000s is in fact proving out to be true. However, while Erlang was a trailblazer, it isn't going to be that Next Big Language, nor anything like it. It got a lot of things right, but it's got too much wrong to be the NBL, and even if you did the minimal fixes, the result wouldn't be backwards compatible with Erlang anymore so it'd be a new language. A lot of Erlang partisans are making the error of thinking the next language needs to be just like Erlang, except perhaps more so. But that's not how progress gets made. The good ideas are ripped out at a much more granular level and recombined with all sorts of other ideas and the end result may be quite different than what you expect. Go, for instance, can very much be seen as a direct sequel to Erlang among other things, even though it may not seem to have "OTP" or "built-in multinode concurrency" or whatever other apparent bullet-list features Erlang has, because a lot of those bullet-list features are really just epiphenomena of the real underlying features. Rust has its own nods to Erlang too; the whole "ownership" thing comes from the experiences with both mutation and immutability in a threading environment. But rather than making "Erlang, but a bit moreso", you get something that takes the lessons, ponders on them for another 15 years, and produces something different. It isn't Erlang, because at the time that Erlang was being written (Joe Armstrong RIP), nobody had the experience to think of Rust and imagine it could make a practical language. (Nor am I entirely sure the computers of the time could have compiled it; even if we sat through the clock time I'm not sure Rust could be stuffed into a 1997 desktop computer's RAM.)