4 ms·

Is there performance benchmark comparing Haskell and Erlang? I hear Erlang is also quite performant when handling a large number of requests, but no hard number

by akg 15y ago

Is there performance benchmark comparing Haskell and Erlang? I hear Erlang is also quite performant when handling a large number of requests, but no hard numbers. Are there similar web frameworks for Erlang as Yesod for Haskell? Also, what about Clojure? That runs on the JVM but how does it fare on the "correctness" claim of Haskell, i.e., "If it compiles it's close to what the programmer intended". I'm learning Haskell right now and really loving it. I wonder if I should stick with it for more production projects or make a switch to Erlang/Clojure. I know Erlang has quite a few success stories behind it, CouchDB being one of them.

apl 15y ago
> Also, what about Clojure? That runs on the JVM but how > does it fare on the "correctness" claim of Haskell, i.e., > "If it compiles it's close to what the programmer > intended". Clojure's a dynamic language, so fairly different in character and strengths. Save for a couple of statically typed Lisp dialects, provability/correctness/etc. is not really what that language family is all about. If you want JVM and reasonably advanced type systems (given that Haskell's claim to correctness essentially boils down to this), try Scala.
dons 15y ago
I think it would be uncontroversial to say that optimized GHC Haskell (statically typed, compiled, optimized, native code) is always faster than current Erlang implementations (dynamically typed, interpreted, lightly optimized). For example, the fastest Erlang programs on the benchmarks game are all slower than all the GHC Haskell programs, * http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=ghc&lang2=erlang http://shootout.alioth.debian.org/u64q/benchmark.php?test=al... Median slowdown relative to GHC is 7x. GHC is aiming for the high performance computing world of late, as well. The Erlang language just doesn't have the facilities for raw number crunching that's needed.
- fijal 15y ago
  It's always controversial to claim performance numbers without any actual evidence. Note that shootout numbers are by far not evidence about web-server performance.
  - jfoutz 15y ago
    Argument from authority is not much of an argument, but dons is intensely focused on ghc performance, here's an easy to google example. http://donsbot.wordpress.com/2010/02/21/smoking-fast-haskell-code-using-ghcs-new-llvm-codegen/ http://donsbot.wordpress.com/2010/02/21/smoking-fast-haskell... It's not to hard for you to go get benchmark numbers in haskell, check out http://www.serpentine.com/blog/2009/09/29/criterion-a-new-benchmarking-library-for-haskell/ http://www.serpentine.com/blog/2009/09/29/criterion-a-new-be... or read about how snap does its benchmarking. As for erlang, there's a lot of papers about reliability. I'm not saying the performance isn't there, it's just not a focus of the erlang community. You'll have a much tougher time figuring out instruction counts for a given function in erlang. Given that erlang is interpreted, without a major focus on performance, it seems unlikely erlang would be quicker. This is just - what i perceive as - the general consensus. I'd love for you to generate some numbers to prove me wrong.
- tete 15y ago
  Erlang is interpreted. Since this is not about a simple CGI application a benchmark in which you compare the running time of a compiled executable with the interpretation and running time a another program it doesn't make a lot of sense. Also this performance usually isn't the best way to argument about languages for web applications. Often your bottleneck is somewhere else and i you really have resource intensive stuff why not just interface C?
  - SomeOtherGuy 15y ago
    >Erlang is interpreted No it isn't, it is compiled to bytecode and executed on a virtual machine, just like most scripting languages are. >Since this is not about a simple CGI application a benchmark in which you compare the running time of a compiled executable with the interpretation and running time a another program it doesn't make a lot of sense. Read the site, this concern is explicitly addressed. The tests are long running, not short tests repeated a million times. So the byte code compilation step ends up too small to notice. That is why using a PHP bytecode caching plugin doesn't increase PHP's performance on the shootout, and why java isn't dead last from the massive startup time of the JVM. >Also this performance usually isn't the best way to argument about languages for web applications This is a myth. Very few web applications actually have one CPU intensive portion that can be moved out into a C module. Most web apps have pretty flat profiles, where overall language speedups make the biggest difference. When 90% of your time is spent in 10% of the code, a C module is a reasonable option. When 90% of your time is spent in 90% of your code, your only option is a faster language.
EvanMiller 15y ago
If you're looking for an Erlang web framework check out Chicago Boss. Brand new tutorial and everything: http://www.chicagoboss.org/ http://www.chicagoboss.org/ Of course, the project still needs some performance work, as highlighted in this post: http://labs.evolope.com/post/16054431816/a-comparison-of-web-servers-for-running-chicago-boss http://labs.evolope.com/post/16054431816/a-comparison-of-web... So CB is a long way from its theoretical maximum (largely because I haven't done any performance engineering on it yet). "Performance", by the way, depends largely on the task at hand. For complex template rendering/delivery, Erlang will probably beat anything out there because of Erlang's intelligent use of immutable data and I/O lists. Most other languages do a "memcpy" for every chunk of the template as it's being built into a final string. Erlang, on the other hand, keeps around pointers to the separate pieces of memory and then does a scattered write (writev) when it's time to send the result to the client.
- SomeOtherGuy 15y ago
  >For complex template rendering/delivery, Erlang will probably beat anything out there because of Erlang's intelligent use of immutable data and I/O lists. What templating engine are you talking about? I'd assume ErlyDTL for obvious reasons, but it is certainly not beating "anything out there". Are you just comparing it to scripting languages or what?
gtani 15y ago
Hmm, that google search leads to this ugly exchange, mid 2010: http://news.ycombinator.com/item?id=1448463 http://news.ycombinator.com/item?id=1448463 Anyway, benchmarks will always show GHC beating Erlang/BEAM, but here's a Simon (or one of his co-authors)saying that he's not into reinventing wheels, he's just going to copy good parts of erlang: http://blog.romanandreg.com/post/6797527470/erlang-and-haskell-sitting-on-a-tree http://blog.romanandreg.com/post/6797527470/erlang-and-haske... slide 41's pretty funny (by akka author): http://www.slideshare.net/jboner/akka-scala-summit-oscon-2010 http://www.slideshare.net/jboner/akka-scala-summit-oscon-201... I have a hazy recollection of a F# user or maybe a Softie saying they're doing the same thing with MailboxProcessors, but i can't google it. Who/why are all these people borrowing/copying/appropriating from OTP behaviors?

by akg 15y ago

apl 15y ago

> Also, what about Clojure? That runs on the JVM but how > does it fare on the "correctness" claim of Haskell, i.e., > "If it compiles it's close to what the programmer > intended". Clojure's a dynamic language, so fairly different in character and strengths. Save for a couple of statically typed Lisp dialects, provability/correctness/etc. is not really what that language family is all about. If you want JVM and reasonably advanced type systems (given that Haskell's claim to correctness essentially boils down to this), try Scala.

dons 15y ago

I think it would be uncontroversial to say that optimized GHC Haskell (statically typed, compiled, optimized, native code) is always faster than current Erlang implementations (dynamically typed, interpreted, lightly optimized). For example, the fastest Erlang programs on the benchmarks game are all slower than all the GHC Haskell programs, * http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=ghc&lang2=erlang http://shootout.alioth.debian.org/u64q/benchmark.php?test=al... Median slowdown relative to GHC is 7x. GHC is aiming for the high performance computing world of late, as well. The Erlang language just doesn't have the facilities for raw number crunching that's needed.

fijal 15y ago
It's always controversial to claim performance numbers without any actual evidence. Note that shootout numbers are by far not evidence about web-server performance.
- jfoutz 15y ago
  Argument from authority is not much of an argument, but dons is intensely focused on ghc performance, here's an easy to google example. http://donsbot.wordpress.com/2010/02/21/smoking-fast-haskell-code-using-ghcs-new-llvm-codegen/ http://donsbot.wordpress.com/2010/02/21/smoking-fast-haskell... It's not to hard for you to go get benchmark numbers in haskell, check out http://www.serpentine.com/blog/2009/09/29/criterion-a-new-benchmarking-library-for-haskell/ http://www.serpentine.com/blog/2009/09/29/criterion-a-new-be... or read about how snap does its benchmarking. As for erlang, there's a lot of papers about reliability. I'm not saying the performance isn't there, it's just not a focus of the erlang community. You'll have a much tougher time figuring out instruction counts for a given function in erlang. Given that erlang is interpreted, without a major focus on performance, it seems unlikely erlang would be quicker. This is just - what i perceive as - the general consensus. I'd love for you to generate some numbers to prove me wrong.
tete 15y ago
Erlang is interpreted. Since this is not about a simple CGI application a benchmark in which you compare the running time of a compiled executable with the interpretation and running time a another program it doesn't make a lot of sense. Also this performance usually isn't the best way to argument about languages for web applications. Often your bottleneck is somewhere else and i you really have resource intensive stuff why not just interface C?
- SomeOtherGuy 15y ago
  >Erlang is interpreted No it isn't, it is compiled to bytecode and executed on a virtual machine, just like most scripting languages are. >Since this is not about a simple CGI application a benchmark in which you compare the running time of a compiled executable with the interpretation and running time a another program it doesn't make a lot of sense. Read the site, this concern is explicitly addressed. The tests are long running, not short tests repeated a million times. So the byte code compilation step ends up too small to notice. That is why using a PHP bytecode caching plugin doesn't increase PHP's performance on the shootout, and why java isn't dead last from the massive startup time of the JVM. >Also this performance usually isn't the best way to argument about languages for web applications This is a myth. Very few web applications actually have one CPU intensive portion that can be moved out into a C module. Most web apps have pretty flat profiles, where overall language speedups make the biggest difference. When 90% of your time is spent in 10% of the code, a C module is a reasonable option. When 90% of your time is spent in 90% of your code, your only option is a faster language.

EvanMiller 15y ago

If you're looking for an Erlang web framework check out Chicago Boss. Brand new tutorial and everything: http://www.chicagoboss.org/ http://www.chicagoboss.org/ Of course, the project still needs some performance work, as highlighted in this post: http://labs.evolope.com/post/16054431816/a-comparison-of-web-servers-for-running-chicago-boss http://labs.evolope.com/post/16054431816/a-comparison-of-web... So CB is a long way from its theoretical maximum (largely because I haven't done any performance engineering on it yet). "Performance", by the way, depends largely on the task at hand. For complex template rendering/delivery, Erlang will probably beat anything out there because of Erlang's intelligent use of immutable data and I/O lists. Most other languages do a "memcpy" for every chunk of the template as it's being built into a final string. Erlang, on the other hand, keeps around pointers to the separate pieces of memory and then does a scattered write (writev) when it's time to send the result to the client.

SomeOtherGuy 15y ago
>For complex template rendering/delivery, Erlang will probably beat anything out there because of Erlang's intelligent use of immutable data and I/O lists. What templating engine are you talking about? I'd assume ErlyDTL for obvious reasons, but it is certainly not beating "anything out there". Are you just comparing it to scripting languages or what?

gtani 15y ago

Hmm, that google search leads to this ugly exchange, mid 2010: http://news.ycombinator.com/item?id=1448463 http://news.ycombinator.com/item?id=1448463 Anyway, benchmarks will always show GHC beating Erlang/BEAM, but here's a Simon (or one of his co-authors)saying that he's not into reinventing wheels, he's just going to copy good parts of erlang: http://blog.romanandreg.com/post/6797527470/erlang-and-haskell-sitting-on-a-tree http://blog.romanandreg.com/post/6797527470/erlang-and-haske... slide 41's pretty funny (by akka author): http://www.slideshare.net/jboner/akka-scala-summit-oscon-2010 http://www.slideshare.net/jboner/akka-scala-summit-oscon-201... I have a hazy recollection of a F# user or maybe a Softie saying they're doing the same thing with MailboxProcessors, but i can't google it. Who/why are all these people borrowing/copying/appropriating from OTP behaviors?