6 ms·

I'll pile on here with my experience. I took a fairly complex Python/Django app and rewrote it into Elixir/Phoenix (and ripped out nearly all the complex cachi

I'll pile on here with my experience. I took a fairly complex Python/Django app and rewrote it into Elixir/Phoenix (and ripped out nearly all the complex caching along the way). The Elixir/Phoenix app rewrite was a little more than 10x faster, much more consistent in response times, and needed far fewer resources to run.

jashmatthews 7y ago
There's no secret sauce in Elixir which isn't in Python to make it 10x faster: https://gist.github.com/jamatthews/7b41b5ce5058f3f4ed674b6ac6ee0e0d https://gist.github.com/jamatthews/7b41b5ce5058f3f4ed674b6ac... Elixir and BEAM are clearly enormously better at some tasks but general performance is actually very similar between Elixir, Ruby, and Python if you use similarly performant frameworks.
- udfalkso 7y ago
  Ok, but why did my app get 10 times faster? Why do so many other people report similar results? Perhaps factorial benchmarks don't translate well to the comparison of complex real world systems. I think, when it comes to web apps specifically, there are a few advantages that make a practical difference. These seem to be the major ones: - Compiled templates and more efficient string interpolation - Improved parallelism and lighter weight processes. This leads to better handling of many requests in parallel (esp. while most are just waiting on IO), better db connection pooling, less blocking, etc. A few other ideas are mentioned here by someone who seems to know more about this than I do: https://news.ycombinator.com/item?id=12014243 https://news.ycombinator.com/item?id=12014243
  - zbentley 7y ago
    > Ok, but why did my app get 10 times faster? You rewrote it. The language you rewrote it in probably has less to do with it than rebuilding all at once what initially was built over time/organically.
    - udfalkso 7y ago
      Yeah that’s probably the case for most rewrites. But in this case the django app was a full rewrite of a php app. It’s quite similar to the eventual elixir app. I feel it’s pretty much a fair fight, but you’ll have to take my word for it.
    - pdimitar 7y ago
      > You rewrote it. The language you rewrote it in probably has less to do with it than rebuilding all at once what initially was built over time/organically. That's a very unfair blanket statement. There are no ways to reduce the inherent flaws of mutable imperative languages no matter how much times you might rewrite the apps in the same language. You definitely can reduce bug surface but not by much; shared data is much more prone to bugs and human mistakes than a system with a runtime that puts immutable data and message-passing front and center. I'll give you that rewrites do improve state of any random app. But I think you are overestimating this factor with your pretty broad and somewhat dismissive statement.
- pdimitar 7y ago
  As another poster said, factorial calculations are hardly the dynamic languages' forte. The measured workflows in terms of web frameworks include a lot of other elements and none of them are CPU-operations-per-watt (namely pure muscle). Erlang's (and thus Elixir's) runtime shines particularly well in web apps' typical workflow, not on tasks where C++ likely is still king.
  - jashmatthews 7y ago
    How else do you explain a 23-230x response time decrease? Either it’s faster or it’s not. I’ve just shown you that it’s not faster. Benchmark Roda against Phoenix and you’ll see exactly the same is also true for the web workloads. There is no magic.
    - pdimitar 7y ago
      There's no magic of course. The difference in real world apps that many people in this thread claim are in favour of Phoenix comes from the responsiveness and robustness of Erlang's OTP itself. (I'll again reiterate that I rewrote a Rails app from scratch to Phoenix and thus accelerated it at least by a factor of 20x. I am just a working guy and have no vested interest in spreading misinformation.) A lot of web frameworks crumble or introduce abhorrent lags after a certain threshold of pressure. Elixir/Phoenix in comparison almost don't change their average response time until much later. Truthfully, after reviewing the TechEmpower repo for the last day and a half, I can only see their Fortunes benchmarks as being half-adequate. Most of their benchmarks are extremely unrealistic in terms of web apps usage. At this point I am not even sure what their goal is anymore.
- josevalim 7y ago
  It is definitely way too simplistic to compare two languages based on factorial (also, fwiw, the native annotation is not doing anything in your example unless the Enum module and its dependencies are natively compiled too).* Elixir (Erlang rather) does have some "secret sauce" that makes it performant on part of the web framework space. For example, the literal pool used by the compiler+VM alongside iolists+`writev`-based operations make it so most template renderings in Phoenix allocate as little memory as possible. This means that any literal string in your template is allocated once on boot and not per-rendering, and building the output buffer of the template engine does not concatenate strings either (which would generate even more garbage). These two blog posts have a bit more detail: * https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently/ * https://www.bignerdranch.com/blog/elixir-and-io-lists-part-2-io-lists-in-phoenix/ I believe Ruby started to gain some of these benefits with the frozen-string literal annotations. So you can opt-in and no longer allocate static strings on every render. But afaik the template engine still performs a reasonable amount of string concatenation at the static/dynamic boundary. It would also likely be possible to eliminate it by implementing something akin iolists/writev. Maybe it has already been done, I haven't kept up. So in a sense, everything is doable in any of them, but in Erlang this is the default way to do it, so it pushes towards an efficient approach to work with I/O from day 1. Case in point: I didn't invent any of this, I just learned what was there. But you won't see those differences unless you are working on an application that is rendering medium to large-sized templates (which are most apps returning HTML) and benchmarks don't tend to exercise that. Other benefits from Erlang that may shine in the web space is the per-process garbage collection. It may reduce the variance on the latency and for short-lived requests, you may not perform garbage collection at all. All the VM does is to reclaim the space once the request is over. At the same time, there are benefits in Ruby and Python you won't find in Elixir. For example, if you need an algorithm that relies heavily in mutability, then Ruby and Python will definitely have an edge. Discord recently had an example of where they made an algorithm much faster by moving part of it to Rust to leverage mutability. But benchmarks don't tend to exercise that either. In fact, benchmarks may highlight non-ideal behaviour. For fully IO-based exercises, I got faster results by running an acceptor per thread that reads+writes as fast as possible, rather than multiplexing requests on all cores. But in practice, most technologies that can leverage multi-core will prefer to multiplex because that will be beneficial as soon as you do any subsequent I/O or CPU work. TL;DR - sure Elixir can be 10x faster in some workloads, sure Python can be 10x faster in others. But of course, it would be incorrect to use any of these results to say Elixir or Python is 10x faster than the other. *Fun fact: I am working on some VM trickery that makes things like Fib/Fac 3-4x faster but at the moment the trickery fails to show any benefit on any slightly more complex code. If they were to accept it, your factorial benchmark would show something much faster for Elixir, but in practice very misleading. :)

jashmatthews 7y ago

There's no secret sauce in Elixir which isn't in Python to make it 10x faster: https://gist.github.com/jamatthews/7b41b5ce5058f3f4ed674b6ac6ee0e0d https://gist.github.com/jamatthews/7b41b5ce5058f3f4ed674b6ac... Elixir and BEAM are clearly enormously better at some tasks but general performance is actually very similar between Elixir, Ruby, and Python if you use similarly performant frameworks.

udfalkso 7y ago
Ok, but why did my app get 10 times faster? Why do so many other people report similar results? Perhaps factorial benchmarks don't translate well to the comparison of complex real world systems. I think, when it comes to web apps specifically, there are a few advantages that make a practical difference. These seem to be the major ones: - Compiled templates and more efficient string interpolation - Improved parallelism and lighter weight processes. This leads to better handling of many requests in parallel (esp. while most are just waiting on IO), better db connection pooling, less blocking, etc. A few other ideas are mentioned here by someone who seems to know more about this than I do: https://news.ycombinator.com/item?id=12014243 https://news.ycombinator.com/item?id=12014243
- zbentley 7y ago
  > Ok, but why did my app get 10 times faster? You rewrote it. The language you rewrote it in probably has less to do with it than rebuilding all at once what initially was built over time/organically.
  - udfalkso 7y ago
    Yeah that’s probably the case for most rewrites. But in this case the django app was a full rewrite of a php app. It’s quite similar to the eventual elixir app. I feel it’s pretty much a fair fight, but you’ll have to take my word for it.
  - pdimitar 7y ago
    > You rewrote it. The language you rewrote it in probably has less to do with it than rebuilding all at once what initially was built over time/organically. That's a very unfair blanket statement. There are no ways to reduce the inherent flaws of mutable imperative languages no matter how much times you might rewrite the apps in the same language. You definitely can reduce bug surface but not by much; shared data is much more prone to bugs and human mistakes than a system with a runtime that puts immutable data and message-passing front and center. I'll give you that rewrites do improve state of any random app. But I think you are overestimating this factor with your pretty broad and somewhat dismissive statement.
pdimitar 7y ago
As another poster said, factorial calculations are hardly the dynamic languages' forte. The measured workflows in terms of web frameworks include a lot of other elements and none of them are CPU-operations-per-watt (namely pure muscle). Erlang's (and thus Elixir's) runtime shines particularly well in web apps' typical workflow, not on tasks where C++ likely is still king.
- jashmatthews 7y ago
  How else do you explain a 23-230x response time decrease? Either it’s faster or it’s not. I’ve just shown you that it’s not faster. Benchmark Roda against Phoenix and you’ll see exactly the same is also true for the web workloads. There is no magic.
  - pdimitar 7y ago
    There's no magic of course. The difference in real world apps that many people in this thread claim are in favour of Phoenix comes from the responsiveness and robustness of Erlang's OTP itself. (I'll again reiterate that I rewrote a Rails app from scratch to Phoenix and thus accelerated it at least by a factor of 20x. I am just a working guy and have no vested interest in spreading misinformation.) A lot of web frameworks crumble or introduce abhorrent lags after a certain threshold of pressure. Elixir/Phoenix in comparison almost don't change their average response time until much later. Truthfully, after reviewing the TechEmpower repo for the last day and a half, I can only see their Fortunes benchmarks as being half-adequate. Most of their benchmarks are extremely unrealistic in terms of web apps usage. At this point I am not even sure what their goal is anymore.
josevalim 7y ago
It is definitely way too simplistic to compare two languages based on factorial (also, fwiw, the native annotation is not doing anything in your example unless the Enum module and its dependencies are natively compiled too).* Elixir (Erlang rather) does have some "secret sauce" that makes it performant on part of the web framework space. For example, the literal pool used by the compiler+VM alongside iolists+`writev`-based operations make it so most template renderings in Phoenix allocate as little memory as possible. This means that any literal string in your template is allocated once on boot and not per-rendering, and building the output buffer of the template engine does not concatenate strings either (which would generate even more garbage). These two blog posts have a bit more detail: * https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-output-efficiently/ * https://www.bignerdranch.com/blog/elixir-and-io-lists-part-2-io-lists-in-phoenix/ I believe Ruby started to gain some of these benefits with the frozen-string literal annotations. So you can opt-in and no longer allocate static strings on every render. But afaik the template engine still performs a reasonable amount of string concatenation at the static/dynamic boundary. It would also likely be possible to eliminate it by implementing something akin iolists/writev. Maybe it has already been done, I haven't kept up. So in a sense, everything is doable in any of them, but in Erlang this is the default way to do it, so it pushes towards an efficient approach to work with I/O from day 1. Case in point: I didn't invent any of this, I just learned what was there. But you won't see those differences unless you are working on an application that is rendering medium to large-sized templates (which are most apps returning HTML) and benchmarks don't tend to exercise that. Other benefits from Erlang that may shine in the web space is the per-process garbage collection. It may reduce the variance on the latency and for short-lived requests, you may not perform garbage collection at all. All the VM does is to reclaim the space once the request is over. At the same time, there are benefits in Ruby and Python you won't find in Elixir. For example, if you need an algorithm that relies heavily in mutability, then Ruby and Python will definitely have an edge. Discord recently had an example of where they made an algorithm much faster by moving part of it to Rust to leverage mutability. But benchmarks don't tend to exercise that either. In fact, benchmarks may highlight non-ideal behaviour. For fully IO-based exercises, I got faster results by running an acceptor per thread that reads+writes as fast as possible, rather than multiplexing requests on all cores. But in practice, most technologies that can leverage multi-core will prefer to multiplex because that will be beneficial as soon as you do any subsequent I/O or CPU work. TL;DR - sure Elixir can be 10x faster in some workloads, sure Python can be 10x faster in others. But of course, it would be incorrect to use any of these results to say Elixir or Python is 10x faster than the other. *Fun fact: I am working on some VM trickery that makes things like Fib/Fac 3-4x faster but at the moment the trickery fails to show any benefit on any slightly more complex code. If they were to accept it, your factorial benchmark would show something much faster for Elixir, but in practice very misleading. :)