4 ms·

Is Elixir really fast for web apps though? According to the web framework benchmark [1] the performance of Elixir is pretty bad - it's consistently slower than

Is Elixir really fast for web apps though? According to the web framework benchmark [1] the performance of Elixir is pretty bad - it's consistently slower than python and ruby frameworks. [1] https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=fortune https://www.techempower.com/benchmarks/#section=data-r12&hw=...

rdtsc 10y ago
It is hard to say how they measure and what they measure. According to their "multiple queries" benchmark, which I guess is the real world one? (Unless everyone expects all custumers to line up and send their requests one after another one). https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=query https://www.techempower.com/benchmarks/#section=data-r12&hw=... we should all be using Dart on the server coupled with MongoDB perhaps.
- Scarbutt 10y ago
  The Dart vm is pretty damn fast, way faster for any CPU bound stuff than the BEAM.
  - rdtsc 10y ago
    Faster than C++, Java, Go?
  - igouy 10y ago
    http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=dart&lang2=hipe http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan... as-in FAQ 1.4 "What sort of problems is Erlang not particularly suitable for? … The most common class of 'less suitable' problems is characterised by performance being a prime requirement and constant-factors having a large effect on performance. Typical examples are image processing, signal processing, sorting large volumes of data and low-level protocol termination."
sanswork 10y ago
Chris McCord said the following on reddit after the results came out: "We don't know what caused the errors and unfortunately we didn't have a chance to collaborate with them on a true run. A few months ago they added Phoenix in a preview, but it was a very poor implementation. They were testing JSON benchmarks through the :browser pipeline, complete with crsf token generation. They had a dev DB pool size of 10, where other frameworks were given of pool size of 100. And they also had heavy IO logging, where other frameworks did no logging. We sent a PR to address these issues, and I was hoping to see true results in the latest runs, but no preview was provided this time and we weren't able to work with them on the errors. tldr; these results are not representative of the framework." From my personal experience going from Rails/Sinatra to Phoenix it feels a lot faster but I haven't done any benchmarks so take that with a grain of salt.
asdf1234 10y ago
Erlang/Elixir is much, much faster than either Python or Ruby especially when it comes to workloads like your typical web app. In the Techempower benchmarks the Phoenix tests had a ton of errors and there was no preview run so whoever submitted them wasn't able to fix them. Look at the error column. I assume they'll be fixed in the next run.
- continuations 10y ago
  Error isn't why Elixir performed poorly in the web framework benchmark. Take a look at the "Single Query" test [1]. The entry for Elli has zero error and is still slower than many Python & Ruby frameworks. [1]: https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=db https://www.techempower.com/benchmarks/#section=data-r12&hw=...
  - clessg 10y ago
    Honestly that site's benchmarks have never been accurate for me in production. But that's how benchmarks work I guess - my bad for bringing up benchmarks! My experience with using Elixir/Phoenix for web applications has been extremely positive performance-wise however, and has surpassed or matched Go, Ruby, PHP, and Python consistently in all production scenarios. For the latter three, the difference has been order-of-magnitude. It depends on the problem domain as always. If you're doing a lot of naïve single-threaded number-crunching, have fun. But Elixir/Phoenix haven't failed me for web applications, even in very intensive situations. It's the first time I've barely had to do any tuning beyond external factors such as network and database queries (which, by the way, Phoenix's Ecto handles very gracefully and explicitly). This is my experience with every Elixir/Phoenix app I've worked on thus far. I apologize if I made it sound like some sort of universal truth.

rdtsc 10y ago

It is hard to say how they measure and what they measure. According to their "multiple queries" benchmark, which I guess is the real world one? (Unless everyone expects all custumers to line up and send their requests one after another one). https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=query https://www.techempower.com/benchmarks/#section=data-r12&hw=... we should all be using Dart on the server coupled with MongoDB perhaps.

Scarbutt 10y ago
The Dart vm is pretty damn fast, way faster for any CPU bound stuff than the BEAM.
- rdtsc 10y ago
  Faster than C++, Java, Go?
- igouy 10y ago
  http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=dart&lang2=hipe http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan... as-in FAQ 1.4 "What sort of problems is Erlang not particularly suitable for? … The most common class of 'less suitable' problems is characterised by performance being a prime requirement and constant-factors having a large effect on performance. Typical examples are image processing, signal processing, sorting large volumes of data and low-level protocol termination."

sanswork 10y ago

Chris McCord said the following on reddit after the results came out: "We don't know what caused the errors and unfortunately we didn't have a chance to collaborate with them on a true run. A few months ago they added Phoenix in a preview, but it was a very poor implementation. They were testing JSON benchmarks through the :browser pipeline, complete with crsf token generation. They had a dev DB pool size of 10, where other frameworks were given of pool size of 100. And they also had heavy IO logging, where other frameworks did no logging. We sent a PR to address these issues, and I was hoping to see true results in the latest runs, but no preview was provided this time and we weren't able to work with them on the errors. tldr; these results are not representative of the framework." From my personal experience going from Rails/Sinatra to Phoenix it feels a lot faster but I haven't done any benchmarks so take that with a grain of salt.

asdf1234 10y ago

Erlang/Elixir is much, much faster than either Python or Ruby especially when it comes to workloads like your typical web app. In the Techempower benchmarks the Phoenix tests had a ton of errors and there was no preview run so whoever submitted them wasn't able to fix them. Look at the error column. I assume they'll be fixed in the next run.

continuations 10y ago
Error isn't why Elixir performed poorly in the web framework benchmark. Take a look at the "Single Query" test [1]. The entry for Elli has zero error and is still slower than many Python & Ruby frameworks. [1]: https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=db https://www.techempower.com/benchmarks/#section=data-r12&hw=...
- clessg 10y ago
  Honestly that site's benchmarks have never been accurate for me in production. But that's how benchmarks work I guess - my bad for bringing up benchmarks! My experience with using Elixir/Phoenix for web applications has been extremely positive performance-wise however, and has surpassed or matched Go, Ruby, PHP, and Python consistently in all production scenarios. For the latter three, the difference has been order-of-magnitude. It depends on the problem domain as always. If you're doing a lot of naïve single-threaded number-crunching, have fun. But Elixir/Phoenix haven't failed me for web applications, even in very intensive situations. It's the first time I've barely had to do any tuning beyond external factors such as network and database queries (which, by the way, Phoenix's Ecto handles very gracefully and explicitly). This is my experience with every Elixir/Phoenix app I've worked on thus far. I apologize if I made it sound like some sort of universal truth.