3 ms·

Erlang vs. Stackless Python: a first benchmark

41 points by jaydub 18y ago

rw 18y ago
Article is from Summer 2007.
unrealwh 18y ago
not surprising that python is competitive...afaik haskell also crushes erlang in lightweight threading benchmarks (supposedly the core strength of erlang).
- jbjohns 18y ago
  Have a link for this? But lightweight threading isn't all that Erlang brings to the table. It comes with an entire library designed to create extremely fault tolerant, live-upgradable, extremely concurrent systems. Getting lightweight threading of the same speed or faster is good, but it's less than 1/3 of the equation.
- njharman 18y ago
  Fault-tolerance (uptime) is Erlang's core strength. Concurrency is just a pleasant and currently trendy side benefit.
- igouy 18y ago
  > afaik haskell also crushes erlang in lightweight threading For some definition of "crushes" http://shootout.alioth.debian.org/u32/benchmark.php?test=threadring&lang=all http://shootout.alioth.debian.org/u32/benchmark.php?test=thr... But with quad-core Erlang processes float between cores slowing things down for sequential programs http://shootout.alioth.debian.org/u32q/benchmark.php?test=threadring&lang=all http://shootout.alioth.debian.org/u32q/benchmark.php?test=th...
newt0311 18y ago
Nice article. Anybody want to try this with the new CPython multiprocess module?
tdavis 18y ago
I don't think Stackless is nearly as well-known as it should be. It has a solid value proposition and is blazing fast in my experience as well.
- inklesspen 18y ago
  Stackless is missing some important parts. I've spent a great deal of time working with it over the past few months, and there's no good SSL socket support. The built-in socket module will block the entire process (all tasklets), and while drop-in replacements exist, they either don't work with SSL, are buggy, or both. Stackless also lacks Erlang's transparent support for multi-process and multi-machine execution, though I talked to a guy who's working on adding that for his Master's degree.
  - tdavis 18y ago
    I'll be the first to admit that http socket issues exist and drop-in replacements are indeed often buggy. That being said, this wouldn't be a huge issue to overcome were Stackless given more TLC. Regardless, it has many other valuable uses.
amix 18y ago
For most things language performance is just a constant - the really hard issue is making something distributed and it's really here that Erlang shines. Being a Python programmer I know that creating a distributed application in Python is far from simple - while it's almost a trivial task in Erlang.
- graemep 18y ago
  I was wondering about this. Is it really comparing like with like? I do not know Erlang, but I thought it had a lot of overhead from being designed for distributed apps etc.
  - shutter 18y ago
    A better comparison would be with Multiprocessing, or Eventlet/Spawning.
bitdiddle 18y ago
It's certainly important to know where Erlang doesn't perform so well. Project Euler ( http://projecteuler.net/ http://projecteuler.net/ ) is also good for exploring the strengths of a given lanaguage. In terms of building robust and scalable distributed systems it cannot be touched. This stems for it's motivating use case of telephony. I'm not sure about Python but I know Haskell has been used largely in academic circles and hasn't had the same amount of attention paid to robustness. Joe Armstrong wrote a very good paper on the history of Erlang's development ( http://lambda-the-ultimate.org/node/2811 http://lambda-the-ultimate.org/node/2811 ).
silentbicycle 18y ago
Sounds like the thread-ring benchmark (http://shootout.alioth.debian.org/u32/benchmark.php?test=threadring&lang=all http://shootout.alioth.debian.org/u32/benchmark.php?test=thr...) at the shootout. It benchmarks an Erlang program that isn't the first the author ever wrote, so its performance is probably more representative.
- igouy 18y ago
  Yep - and both derive from this 10 year old measurement of context switching: "Performance Measurements of Threads in Java and Processes in Erlang" http://www.sics.se/~joe/ericsson/du98024.html http://www.sics.se/~joe/ericsson/du98024.html
chaostheory 18y ago
as was already pointed out, this isn't a very accurate benchmark; it wasn't tested on a multi-core/processor machine, nor more importantly was it tested on a cluster of machines.

41 points by jaydub 18y ago

rw 18y ago

Article is from Summer 2007.

not surprising that python is competitive...afaik haskell also crushes erlang in lightweight threading benchmarks (supposedly the core strength of erlang).

jbjohns 18y ago
Have a link for this? But lightweight threading isn't all that Erlang brings to the table. It comes with an entire library designed to create extremely fault tolerant, live-upgradable, extremely concurrent systems. Getting lightweight threading of the same speed or faster is good, but it's less than 1/3 of the equation.
njharman 18y ago
Fault-tolerance (uptime) is Erlang's core strength. Concurrency is just a pleasant and currently trendy side benefit.
igouy 18y ago
> afaik haskell also crushes erlang in lightweight threading For some definition of "crushes" http://shootout.alioth.debian.org/u32/benchmark.php?test=threadring&lang=all http://shootout.alioth.debian.org/u32/benchmark.php?test=thr... But with quad-core Erlang processes float between cores slowing things down for sequential programs http://shootout.alioth.debian.org/u32q/benchmark.php?test=threadring&lang=all http://shootout.alioth.debian.org/u32q/benchmark.php?test=th...

newt0311 18y ago

Nice article. Anybody want to try this with the new CPython multiprocess module?

tdavis 18y ago

I don't think Stackless is nearly as well-known as it should be. It has a solid value proposition and is blazing fast in my experience as well.

inklesspen 18y ago
Stackless is missing some important parts. I've spent a great deal of time working with it over the past few months, and there's no good SSL socket support. The built-in socket module will block the entire process (all tasklets), and while drop-in replacements exist, they either don't work with SSL, are buggy, or both. Stackless also lacks Erlang's transparent support for multi-process and multi-machine execution, though I talked to a guy who's working on adding that for his Master's degree.
- tdavis 18y ago
  I'll be the first to admit that http socket issues exist and drop-in replacements are indeed often buggy. That being said, this wouldn't be a huge issue to overcome were Stackless given more TLC. Regardless, it has many other valuable uses.

amix 18y ago

For most things language performance is just a constant - the really hard issue is making something distributed and it's really here that Erlang shines. Being a Python programmer I know that creating a distributed application in Python is far from simple - while it's almost a trivial task in Erlang.

graemep 18y ago
I was wondering about this. Is it really comparing like with like? I do not know Erlang, but I thought it had a lot of overhead from being designed for distributed apps etc.
- shutter 18y ago
  A better comparison would be with Multiprocessing, or Eventlet/Spawning.

bitdiddle 18y ago

It's certainly important to know where Erlang doesn't perform so well. Project Euler ( http://projecteuler.net/ http://projecteuler.net/ ) is also good for exploring the strengths of a given lanaguage. In terms of building robust and scalable distributed systems it cannot be touched. This stems for it's motivating use case of telephony. I'm not sure about Python but I know Haskell has been used largely in academic circles and hasn't had the same amount of attention paid to robustness. Joe Armstrong wrote a very good paper on the history of Erlang's development ( http://lambda-the-ultimate.org/node/2811 http://lambda-the-ultimate.org/node/2811 ).

silentbicycle 18y ago

Sounds like the thread-ring benchmark (http://shootout.alioth.debian.org/u32/benchmark.php?test=threadring&lang=all http://shootout.alioth.debian.org/u32/benchmark.php?test=thr...) at the shootout. It benchmarks an Erlang program that isn't the first the author ever wrote, so its performance is probably more representative.

igouy 18y ago
Yep - and both derive from this 10 year old measurement of context switching: "Performance Measurements of Threads in Java and Processes in Erlang" http://www.sics.se/~joe/ericsson/du98024.html http://www.sics.se/~joe/ericsson/du98024.html

chaostheory 18y ago

as was already pointed out, this isn't a very accurate benchmark; it wasn't tested on a multi-core/processor machine, nor more importantly was it tested on a cluster of machines.