4 ms·
It is definitely way too simplistic to compare two languages based on factorial (also, fwiw, the native annotation is not doing anything in your example unless
by josevalim 7y ago
It is definitely way too simplistic to compare two languages based on factorial (also, fwiw, the native annotation is not doing anything in your example unless the Enum module and its dependencies are natively compiled too).*
Elixir (Erlang rather) does have some "secret sauce" that makes it performant on part of the web framework space. For example, the literal pool used by the compiler+VM alongside iolists+`writev`-based operations make it so most template renderings in Phoenix allocate as little memory as possible. This means that any literal string in your template is allocated once on boot and not per-rendering, and building the output buffer of the template engine does not concatenate strings either (which would generate even more garbage).
These two blog posts have a bit more detail:
* https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently/
* https://www.bignerdranch.com/blog/elixir-and-io-lists-part-2-io-lists-in-phoenix/
I believe Ruby started to gain some of these benefits with the frozen-string literal annotations. So you can opt-in and no longer allocate static strings on every render. But afaik the template engine still performs a reasonable amount of string concatenation at the static/dynamic boundary. It would also likely be possible to eliminate it by implementing something akin iolists/writev. Maybe it has already been done, I haven't kept up.
So in a sense, everything is doable in any of them, but in Erlang this is the default way to do it, so it pushes towards an efficient approach to work with I/O from day 1. Case in point: I didn't invent any of this, I just learned what was there. But you won't see those differences unless you are working on an application that is rendering medium to large-sized templates (which are most apps returning HTML) and benchmarks don't tend to exercise that.
Other benefits from Erlang that may shine in the web space is the per-process garbage collection. It may reduce the variance on the latency and for short-lived requests, you may not perform garbage collection at all. All the VM does is to reclaim the space once the request is over.
At the same time, there are benefits in Ruby and Python you won't find in Elixir. For example, if you need an algorithm that relies heavily in mutability, then Ruby and Python will definitely have an edge. Discord recently had an example of where they made an algorithm much faster by moving part of it to Rust to leverage mutability. But benchmarks don't tend to exercise that either.
In fact, benchmarks may highlight non-ideal behaviour. For fully IO-based exercises, I got faster results by running an acceptor per thread that reads+writes as fast as possible, rather than multiplexing requests on all cores. But in practice, most technologies that can leverage multi-core will prefer to multiplex because that will be beneficial as soon as you do any subsequent I/O or CPU work.
TL;DR - sure Elixir can be 10x faster in some workloads, sure Python can be 10x faster in others. But of course, it would be incorrect to use any of these results to say Elixir or Python is 10x faster than the other.
*Fun fact: I am working on some VM trickery that makes things like Fib/Fac 3-4x faster but at the moment the trickery fails to show any benefit on any slightly more complex code. If they were to accept it, your factorial benchmark would show something much faster for Elixir, but in practice very misleading. :)
- AlchemistCamp 7y ago> Other benefits from Erlang that may shine in the web space is the per-process garbage collection. It may reduce the variance on the latency and for short-lived requests, you may not perform garbage collection at all. This one was a big draw for me. It was a major reason I got interested in Elixir in the first place instead of going the Scala/Akka route.
- jashmatthews 7y agoThanks for the very informative reply! It's already the standard way in Ruby, even in Rails. Jeremy Evans did the template optimization you're describing using frozen strings 3+ years ago in Erubi and it was adopted by Rails just a few months later. There's no long STW GC phase during requests in modern CRuby either. It's nowhere near as elegant as BEAM's heap-per-process design but the GC issues seen in CRuby < 2.2 have been gone for almost 5 years now. GitHub retired OOBGC last year. Pause times are just 1-2ms and total time spent in GC is 0.5% on for the main application I work on. Are you talking about generating better bytecode based on the semantics of Elixir or a bytecode optimization pass?
- josevalim 7y ago> It's already the standard way in Ruby, even in Rails. Jeremy Evans did the template optimization you're describing using frozen strings 3+ years ago in Erubi and it was adopted by Rails just a few months later. Yes, I was aware of the frozen strings optimization. Thanks for confirming! The part I am not fully aware though is in regards to the output buffer, which is a separate optimization. Is the output of a template being stored in an array that is writev-ed to the socket? Or is a template still rendered to a large string? Or maybe something in the middle? If you don't mind me asking, what is OOBC? My Google-fu failed me. In any case, 0.5% is quite good, although that is also dependent on how much garbage the app generates. And as you know some libs/frameworks tend to be more mindful of that than others. :) > Are you talking about generating better bytecode based on the semantics of Elixir or a bytecode optimization pass? Do you mean the VM optimizations I was working on? I was mostly AOT compiling the bytecode of a function. That removes the VM dispatch overhead which further helps with CPU speculation. I see very good results on recursive functions but not much beyond that.