21 ms·

Benchmark: Java, Python, PHP, C, JavaScript, Swift, Ruby, Perl, Go, Lua, Rust

101 points by filipmandaric 9y ago

rhabarba 9y ago
No Lisp? :-(
- susi22 9y ago
  They used to have Lisp & Clojure. No idea why it's been removed. Some is still accessible: http://benchmarksgame.alioth.debian.org/u32/lisp.html http://benchmarksgame.alioth.debian.org/u32/lisp.html http://benchmarksgame.alioth.debian.org/u32/clojure.html http://benchmarksgame.alioth.debian.org/u32/clojure.html
  - agumonkey 9y ago
    As if Lisp needed more exposure..
  - igouy 9y ago
    >> No idea why it's been removed. << Lisp hasn't been removed. (Clojure has. So's Scala.)
- mark_l_watson 9y ago
  A CL Common Lisp and Racket are included.
- igouy 9y ago
  Go to the home page http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/ and click "Lisp".
  - rhabarba 9y ago
    Thanks.
z1mm32m4n 9y ago
I see that the point of this site is supposed to be that a change in implementation (keeping the language itself held constant) greater affect on performance than rewriting in another language. My problem with this site it makes poor support of that claim. The data is hidden behind loads of links, and none of the data is visualized at all. Even a few simple bar charts, keeping the hierarchy of the site otherwise unchanged would vastly aid comprehension. The points being made would come across much more easily. This sort of data is the poster child for visualization, yet this site relegates the data to tables of numbers.
- igouy 9y ago
  What on the website tells you that is the point? >> Even a few simple bar charts << http://benchmarksgame.alioth.debian.org/u64q/which-programs-are-fastest.html http://benchmarksgame.alioth.debian.org/u64q/which-programs-... >> would vastly aid comprehension. << Is it possible that you are just wrong? Is it possible that simple bar charts do not aid comprehension but encourage thoughtless instant conclusions?
rhlala 9y ago
I readed python was slow, but i had no idea it was relativly That slow!
FrozenVoid 9y ago
Years ago i was naive enough to believe these benchmarks. Now i know these are Potemkin-village level stunts and political maneuvering by language activists and staff of that site. If you really want actual benchmarking you have to do it yourself and read the .asm output for comparison. Another caveat is that some languages are genuinely faster in one hyperoptimized case(even faster than C) but fail horribly when things go more complicated and abstraction creep makes them much slower in most cases.
- LanceH 9y ago
  I don't find the .asm output to be meaningful at all. The realistic comparison for me is if I write a solution the easiest/most convenient way in each language, which language is naturally more efficient.
- olegkikin 9y ago
  You provided zero evidence for your claim. I'm not saying you're necessarily wrong, but so far you have only made an assertion. If you think some language is misrepresented in that benchmark, you can submit your own solution.
  - d0mine 9y ago
    On how the benchmarks should be interpreted http://benchmarksgame.alioth.debian.org/for-programming-language-researchers.html http://benchmarksgame.alioth.debian.org/for-programming-lang...
    - igouy 9y ago
      http://benchmarksgame.alioth.debian.org/dont-jump-to-conclusions.html http://benchmarksgame.alioth.debian.org/dont-jump-to-conclus...
  - tyingq 9y ago
    I had the same observation. I'm a Perl fan, but I can see some submissions were solely motivated to make Perl appear competitive. Like using threads in Perl, which isn't often done in real life. I'm not sure how much evidence is needed. It seems obvious that people who are fans of language X are going to submit code with the sole intention of bumping up performance at any cost. The site is still interesting, but probably useless for real world decision making.
joliu 9y ago
Ruby's fastest time for pidigits was 3.14, haha.
rilut 9y ago
No Scala anymore?
- igouy 9y ago
  No success getting programs updated for newer versions -- so, no Scala anymore, done.
RandyRanderson 9y ago
If you're a competent programmer, one should be able to make any real-world task take about the same time in any two languages. Therefore, any large-margin differences are benchmarking the programmer not the technology.
- clouddrover 9y ago
  > one should be able to make any real-world task take about the same time in any two languages. I don't think that's a practical claim. Interpreted or JIT compiled languages depend on the interpreter or VM. JavaScript has become much faster over time as JavaScript VMs have improved, independent of the details of any particular JavaScript program. Ruby has become faster over time for the same reason. I've seen Ruby programs I've written speed up significantly without any significant code changes when I moved from an older to a newer version of Ruby.
insulanian 9y ago
Wow, I expected Erlang to score better, considering the claims of Elixir community how fast it is... compared to Ruby. OTOH maybe that tells more about Ruby.
- innocentoldguy 9y ago
  This is why benchmarks really don't mean much. They are a myopic view of the complex nature of language selection. For example, I would never recommend Java for a bulletproof app that had to scale, because OTP is so much better at scalability, fault tolerance, live debugging, and hot code deployments. At the same time, I would never opt for Elixir for doing screen-scraping, because it is slow at it. Sometimes, developer productivity is more important than speed and scalability. It is important to know the strengths and weaknesses of various languages, and the most important aspects of your project, in order to pick the right tool for the job, I think. Performance benchmarks are but a small part of the selection process.
- rubyn00bie 9y ago
  Erlang makes no claims about being fast in these sorts of tests. They aren't using what it was designed to do. Also some languages like, Ruby, do really well on some of these tests because the C-code underneath has been written well (example regexes are just as fast under ruby as they are anywhere else because the underlying C-library kicks ass though I forget the name of it).
  - smitherfield 9y ago
    You're probably thinking of PCRE, although Ruby actually uses Onigmo: https://github.com/k-takata/Onigmo https://github.com/k-takata/Onigmo
- giancarlostoro 9y ago
  Erlang's claim to fame at least for me is parallelism. When you scale out to thousands upon thousands of clients. Think about the simple fact it was using all cores on a processor out of the box by design that a computer had available. There's also the whole zero downtime aspect. In Erlang you should be able to design software (again downtime from the software side, not much you can do about hardware failure or server maintenance updates to your OS) that if it needs an update it can update itself without killing off anyone connected to it. There's also it's functional aspects.
nodesocket 9y ago
The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory usage and performance on every test except regex-redux (edge case?). http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=go&lang2=node http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
- Terribledactyl 9y ago
  >regex-redux (edge case?) It looks like the JS version is using the built in runtime's regex engine. I don't know exactly why (maybe differences or missing features in go's regex) but the go version is actually using the c FFI to call pcre.h, and without profiling it I'd guess a huge hit is that alone.
  - weberc2 9y ago
    I think Go's regex lib is native Go, but simply naive. In any case, this particular benchmark isn't useful unless your application is regex as a service.
  - igouy 9y ago
    > I don't know exactly why … and without profiling it I'd guess a huge hit is that alone. Don't guess. Look at the measurements. Look at the source code. That Go PCRE program is faster than -- http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexredux&lang=go&id=1 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... Maybe faster still to use some of the techniques from this other Go PCRE program -- http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexredux&lang=go&id=9 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
- Safety1stClyde 9y ago
  > The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory usage and performance on every test except regex-redux (edge case?). Why would it be stunning that a programming language compiled to assembler can beat a broken-by-design interpreted language which doesn't even have integers or arrays? The stunning part is that the people behind node.js have managed to get that much performance.
Xoros 9y ago
So PHP is faster on most of those test than python or ruby ? I'm I missing something ? I thought everyone hating PHP ( also) because it was slow.
- ars 9y ago
  No PHP is hated because it's too easy to use. The means that a lot of rather bad (inexperienced?) programmers use it, which means there is a lot of bad PHP code out there. People see that bad code and assume the language is bad. The language has some warts and inconsistencies, and people are eager to point them out as justification for their hate. But for real programming most of them hardly matter.
  - thinkindie 9y ago
    this is something a bit naive - basically all dynamic languages have the same low barrier and can produce the same crappy code, see for example javascript or ruby (with RoR). Python too, considering the fact that it's used by a lot of people that their main job is not software engineering. The biggest difference between PHP and the other dynamic languages, though, is that PHP came out with pretty popular stuff like wordpress, joomla, magento or drupal that they all share crappy code coz they all have a large codebase started before PHP managed to fix some stuff starting from version 5.3
    - collyw 9y ago
      Python is as easy to write as PHP, but deploying it as a web app is a whole different story.
      - acdha 9y ago
        That comparison is complicated: it's definitely easy to drop a .php file on a server but once you start talking about frameworks, using a newer version than your OS distribution shipped, adding C extensions, etc. it's easy to find counter-examples, and the Go proponents are not wrong to observe that deploying a single file is easier than either. This is also becoming less of a distinguishing factor in the container era where the answer for a number of cases is “Extend a base Docker image”.
SwellJoe 9y ago
Hey, the benchmark game got a new website (at some point in the several years since I looked at it last)! Almost simplistic, but nice to read. As someone that works predominantly in a language long considered passé (Perl), I always kinda relish seeing that it is still faster than Python and Ruby for many problems. As Python and Ruby have gotten faster, I'd sort of assumed they were both notably and clearly faster than Perl, by now, but that's not the case though the difference is now quite small and probably debatable.
- rurban 9y ago
  Perl is only faster here in nbody because I tricked it. I used a tricky compile-time optimization for faster array index access. You wouldn't do that in the real world, and in the real world PHP, ruby and python are faster. cperl instead is only beaten by PHP. But the benchmark game is not open for faster alternate engines like luajit, nim, crystal, pypy, truffle-graal, ... or even slower ones like Perl6, which limits the impact of those better engines. Decades ago it was open and written in Perl. Then it was rewritten in Python and closed down. Go figure.
  - igouy 9y ago
    >> a tricky compile-time optimization for faster array index access. << And you even comment on that usage in your code. >> Decades ago it was open and written in Perl. Then it was rewritten in Python and closed down. Go figure. << At best your comment is disingenuous. You can download the Python measurement scripts, use them with luajit, nim, crystal, pypy, truffle-graal or Perl6 programs and publish the measurements. You don't. Go figure.
    - SwellJoe 9y ago
      Perhaps this is merely a reflection of the difficulty of finding where to download the measurement scripts? I found them (here: http://benchmarksgame.alioth.debian.org/play.html http://benchmarksgame.alioth.debian.org/play.html ) but, it took some clicking. Maybe a github link on the front page, or similar, would alleviate some of the frustration the previous poster had about it. I assumed the sources were available, somewhere, but until I went looking for it, I wouldn't have been able to say so with confidence. That's not an obligation, of course...volunteers should feel free to do whatever they want with their projects. But, if you wanted to keep the community involvement the game had historically, linking source in the lingua franca of the day (github or gitlab, or whatever, seems pretty standard today) would go a long way.
dpratt 9y ago
Any benchmark that includes a JVM language always makes me look for a disclaimer that they allowed for proper JIT/warming time by running the benchmark a few times in the same VM instance before collecting the numbers. I may have missed it in this article, but I didn't see it. Yes, startup time is a concern with the JVM, but that's not what it's built for - it's meant for long-runnning repeatable code paths.
- alayne 9y ago
  He doesn't do warmups for JITted languages.
  - peterashford 9y ago
    ...which means the measurements are meaningless. They will include the time to analyse the hot paths and then compile them.
    - igouy 9y ago
      They will include a few tenths of a second. Note the timings are seconds and tens-of-seconds.
    - deleted 9y ago
      [deleted]
- mpweiher 9y ago
  > that's not what it's built for There was nothing in the original design that said "only long running server processes". In fact, Java was originally aimed at set-top boxes, then web applets, then cross platform GUI apps. It was only after it failed in all of these that the server was discovered, despite the fact that "write once, run anywhere" is almost completely pointless when you control the hardware it runs on. The long startup times are a limitation of the implementation technology, not a principled design choice. See also: Jitterdämmerung ( http://blog.metaobject.com/2015/10/jitterdammerung.html http://blog.metaobject.com/2015/10/jitterdammerung.html )
- 9y ago
mike986 9y ago
they should have included luajit it's almost a dropin replacement for most projects
- trynumber9 9y ago
  It was included in the past [0]. It was removed to simplify the process and due to a "one implementation per language" rule. [0]: http://i.imgur.com/0xxOfVm.png http://i.imgur.com/0xxOfVm.png
deleted 9y ago
[deleted]
jackyb 9y ago
Does anybody know why Go binary-trees is so much slower than Java's?
- trishume 9y ago
  Binary-trees is allocation bottlenecked, and Java has bump pointer allocation where Go has slow allocation. The root cause being that Go's GC is non-moving so it can't do bump pointer allocation without fragmentation, so it has to do a whole bunch of computation to find a place for each node.
  - jackyb 9y ago
    I see, thanks!
frik 9y ago
With this new revision this great benchmark got less useful. Why can't I compare: Lua vs LuaGIT PHP5 vs PHP7 vs HHVM vs Hack Perl5 vs Perl6 GNU GCC vs Intel C vs CLANG This new rule "one implementation per language" is so stupid!!! @debian: change it back
- igouy 9y ago
  "If you're interested in something not shown on the benchmarks game website then please take the program source code and the measurement scripts and publish your own measurements."
bedros 9y ago
where's D-lang? it would be interesting to compare D with C, C++, and Rust
- wulfklaue 9y ago
  Good luck on seeing D on that site. There seems to be a active war between the sites maintainer and D. There used to be D benchmarks on the old site and people tries to release new code to the site, but for some reason the current maintainer has a issue with D. And he thinks its no unfair that D is not resented when a lot of people look at his site and think: "If your not here as a language, then the language must be bad". Good luck on changing his mindset because i have seen topics going back years regarding D not being there. If you want a more impartial benchmark set: https://github.com/kostya/benchmarks https://github.com/kostya/benchmarks And where is D on those tests ;)
  - igouy 9y ago
    > There seems to be a active war between the sites maintainer and D. No. > There used to be D benchmarks on the old site Back in early 2008. http://web.archive.org/web/20080730204710/http://shootout.alioth.debian.org:80/gp4/ http://web.archive.org/web/20080730204710/http://shootout.al... Look how many other language implementations were also shown back then and have not been shown since. > … for some reason the current maintainer has a issue with D No. > Good luck on changing his mindset… Make the measurements you want to see and publish them for everyone else to see.

101 points by filipmandaric 9y ago

rhabarba 9y ago

No Lisp? :-(

susi22 9y ago
They used to have Lisp & Clojure. No idea why it's been removed. Some is still accessible: http://benchmarksgame.alioth.debian.org/u32/lisp.html http://benchmarksgame.alioth.debian.org/u32/lisp.html http://benchmarksgame.alioth.debian.org/u32/clojure.html http://benchmarksgame.alioth.debian.org/u32/clojure.html
- agumonkey 9y ago
  As if Lisp needed more exposure..
- igouy 9y ago
  >> No idea why it's been removed. << Lisp hasn't been removed. (Clojure has. So's Scala.)
mark_l_watson 9y ago
A CL Common Lisp and Racket are included.
igouy 9y ago
Go to the home page http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/ and click "Lisp".
- rhabarba 9y ago
  Thanks.

z1mm32m4n 9y ago

I see that the point of this site is supposed to be that a change in implementation (keeping the language itself held constant) greater affect on performance than rewriting in another language. My problem with this site it makes poor support of that claim. The data is hidden behind loads of links, and none of the data is visualized at all. Even a few simple bar charts, keeping the hierarchy of the site otherwise unchanged would vastly aid comprehension. The points being made would come across much more easily. This sort of data is the poster child for visualization, yet this site relegates the data to tables of numbers.

igouy 9y ago
What on the website tells you that is the point? >> Even a few simple bar charts << http://benchmarksgame.alioth.debian.org/u64q/which-programs-are-fastest.html http://benchmarksgame.alioth.debian.org/u64q/which-programs-... >> would vastly aid comprehension. << Is it possible that you are just wrong? Is it possible that simple bar charts do not aid comprehension but encourage thoughtless instant conclusions?

rhlala 9y ago

I readed python was slow, but i had no idea it was relativly That slow!

FrozenVoid 9y ago

Years ago i was naive enough to believe these benchmarks. Now i know these are Potemkin-village level stunts and political maneuvering by language activists and staff of that site. If you really want actual benchmarking you have to do it yourself and read the .asm output for comparison. Another caveat is that some languages are genuinely faster in one hyperoptimized case(even faster than C) but fail horribly when things go more complicated and abstraction creep makes them much slower in most cases.

LanceH 9y ago
I don't find the .asm output to be meaningful at all. The realistic comparison for me is if I write a solution the easiest/most convenient way in each language, which language is naturally more efficient.
olegkikin 9y ago
You provided zero evidence for your claim. I'm not saying you're necessarily wrong, but so far you have only made an assertion. If you think some language is misrepresented in that benchmark, you can submit your own solution.
- d0mine 9y ago
  On how the benchmarks should be interpreted http://benchmarksgame.alioth.debian.org/for-programming-language-researchers.html http://benchmarksgame.alioth.debian.org/for-programming-lang...
  - igouy 9y ago
    http://benchmarksgame.alioth.debian.org/dont-jump-to-conclusions.html http://benchmarksgame.alioth.debian.org/dont-jump-to-conclus...
- tyingq 9y ago
  I had the same observation. I'm a Perl fan, but I can see some submissions were solely motivated to make Perl appear competitive. Like using threads in Perl, which isn't often done in real life. I'm not sure how much evidence is needed. It seems obvious that people who are fans of language X are going to submit code with the sole intention of bumping up performance at any cost. The site is still interesting, but probably useless for real world decision making.

joliu 9y ago

Ruby's fastest time for pidigits was 3.14, haha.

rilut 9y ago

No Scala anymore?

igouy 9y ago
No success getting programs updated for newer versions -- so, no Scala anymore, done.

RandyRanderson 9y ago

If you're a competent programmer, one should be able to make any real-world task take about the same time in any two languages. Therefore, any large-margin differences are benchmarking the programmer not the technology.

clouddrover 9y ago
> one should be able to make any real-world task take about the same time in any two languages. I don't think that's a practical claim. Interpreted or JIT compiled languages depend on the interpreter or VM. JavaScript has become much faster over time as JavaScript VMs have improved, independent of the details of any particular JavaScript program. Ruby has become faster over time for the same reason. I've seen Ruby programs I've written speed up significantly without any significant code changes when I moved from an older to a newer version of Ruby.

insulanian 9y ago

Wow, I expected Erlang to score better, considering the claims of Elixir community how fast it is... compared to Ruby. OTOH maybe that tells more about Ruby.

innocentoldguy 9y ago
This is why benchmarks really don't mean much. They are a myopic view of the complex nature of language selection. For example, I would never recommend Java for a bulletproof app that had to scale, because OTP is so much better at scalability, fault tolerance, live debugging, and hot code deployments. At the same time, I would never opt for Elixir for doing screen-scraping, because it is slow at it. Sometimes, developer productivity is more important than speed and scalability. It is important to know the strengths and weaknesses of various languages, and the most important aspects of your project, in order to pick the right tool for the job, I think. Performance benchmarks are but a small part of the selection process.
rubyn00bie 9y ago
Erlang makes no claims about being fast in these sorts of tests. They aren't using what it was designed to do. Also some languages like, Ruby, do really well on some of these tests because the C-code underneath has been written well (example regexes are just as fast under ruby as they are anywhere else because the underlying C-library kicks ass though I forget the name of it).
- smitherfield 9y ago
  You're probably thinking of PCRE, although Ruby actually uses Onigmo: https://github.com/k-takata/Onigmo https://github.com/k-takata/Onigmo
giancarlostoro 9y ago
Erlang's claim to fame at least for me is parallelism. When you scale out to thousands upon thousands of clients. Think about the simple fact it was using all cores on a processor out of the box by design that a computer had available. There's also the whole zero downtime aspect. In Erlang you should be able to design software (again downtime from the software side, not much you can do about hardware failure or server maintenance updates to your OS) that if it needs an update it can update itself without killing off anyone connected to it. There's also it's functional aspects.

nodesocket 9y ago

The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory usage and performance on every test except regex-redux (edge case?). http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=go&lang2=node http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...

Terribledactyl 9y ago
>regex-redux (edge case?) It looks like the JS version is using the built in runtime's regex engine. I don't know exactly why (maybe differences or missing features in go's regex) but the go version is actually using the c FFI to call pcre.h, and without profiling it I'd guess a huge hit is that alone.
- weberc2 9y ago
  I think Go's regex lib is native Go, but simply naive. In any case, this particular benchmark isn't useful unless your application is regex as a service.
- igouy 9y ago
  > I don't know exactly why … and without profiling it I'd guess a huge hit is that alone. Don't guess. Look at the measurements. Look at the source code. That Go PCRE program is faster than -- http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexredux&lang=go&id=1 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... Maybe faster still to use some of the techniques from this other Go PCRE program -- http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexredux&lang=go&id=9 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
Safety1stClyde 9y ago
> The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory usage and performance on every test except regex-redux (edge case?). Why would it be stunning that a programming language compiled to assembler can beat a broken-by-design interpreted language which doesn't even have integers or arrays? The stunning part is that the people behind node.js have managed to get that much performance.

Xoros 9y ago

So PHP is faster on most of those test than python or ruby ? I'm I missing something ? I thought everyone hating PHP ( also) because it was slow.

ars 9y ago
No PHP is hated because it's too easy to use. The means that a lot of rather bad (inexperienced?) programmers use it, which means there is a lot of bad PHP code out there. People see that bad code and assume the language is bad. The language has some warts and inconsistencies, and people are eager to point them out as justification for their hate. But for real programming most of them hardly matter.
- thinkindie 9y ago
  this is something a bit naive - basically all dynamic languages have the same low barrier and can produce the same crappy code, see for example javascript or ruby (with RoR). Python too, considering the fact that it's used by a lot of people that their main job is not software engineering. The biggest difference between PHP and the other dynamic languages, though, is that PHP came out with pretty popular stuff like wordpress, joomla, magento or drupal that they all share crappy code coz they all have a large codebase started before PHP managed to fix some stuff starting from version 5.3
  - collyw 9y ago
    Python is as easy to write as PHP, but deploying it as a web app is a whole different story.
    - acdha 9y ago
      That comparison is complicated: it's definitely easy to drop a .php file on a server but once you start talking about frameworks, using a newer version than your OS distribution shipped, adding C extensions, etc. it's easy to find counter-examples, and the Go proponents are not wrong to observe that deploying a single file is easier than either. This is also becoming less of a distinguishing factor in the container era where the answer for a number of cases is “Extend a base Docker image”.

SwellJoe 9y ago

Hey, the benchmark game got a new website (at some point in the several years since I looked at it last)! Almost simplistic, but nice to read. As someone that works predominantly in a language long considered passé (Perl), I always kinda relish seeing that it is still faster than Python and Ruby for many problems. As Python and Ruby have gotten faster, I'd sort of assumed they were both notably and clearly faster than Perl, by now, but that's not the case though the difference is now quite small and probably debatable.

rurban 9y ago
Perl is only faster here in nbody because I tricked it. I used a tricky compile-time optimization for faster array index access. You wouldn't do that in the real world, and in the real world PHP, ruby and python are faster. cperl instead is only beaten by PHP. But the benchmark game is not open for faster alternate engines like luajit, nim, crystal, pypy, truffle-graal, ... or even slower ones like Perl6, which limits the impact of those better engines. Decades ago it was open and written in Perl. Then it was rewritten in Python and closed down. Go figure.
- igouy 9y ago
  >> a tricky compile-time optimization for faster array index access. << And you even comment on that usage in your code. >> Decades ago it was open and written in Perl. Then it was rewritten in Python and closed down. Go figure. << At best your comment is disingenuous. You can download the Python measurement scripts, use them with luajit, nim, crystal, pypy, truffle-graal or Perl6 programs and publish the measurements. You don't. Go figure.
  - SwellJoe 9y ago
    Perhaps this is merely a reflection of the difficulty of finding where to download the measurement scripts? I found them (here: http://benchmarksgame.alioth.debian.org/play.html http://benchmarksgame.alioth.debian.org/play.html ) but, it took some clicking. Maybe a github link on the front page, or similar, would alleviate some of the frustration the previous poster had about it. I assumed the sources were available, somewhere, but until I went looking for it, I wouldn't have been able to say so with confidence. That's not an obligation, of course...volunteers should feel free to do whatever they want with their projects. But, if you wanted to keep the community involvement the game had historically, linking source in the lingua franca of the day (github or gitlab, or whatever, seems pretty standard today) would go a long way.

dpratt 9y ago

Any benchmark that includes a JVM language always makes me look for a disclaimer that they allowed for proper JIT/warming time by running the benchmark a few times in the same VM instance before collecting the numbers. I may have missed it in this article, but I didn't see it. Yes, startup time is a concern with the JVM, but that's not what it's built for - it's meant for long-runnning repeatable code paths.

alayne 9y ago
He doesn't do warmups for JITted languages.
- peterashford 9y ago
  ...which means the measurements are meaningless. They will include the time to analyse the hot paths and then compile them.
  - igouy 9y ago
    They will include a few tenths of a second. Note the timings are seconds and tens-of-seconds.
  - deleted 9y ago
    [deleted]
mpweiher 9y ago
> that's not what it's built for There was nothing in the original design that said "only long running server processes". In fact, Java was originally aimed at set-top boxes, then web applets, then cross platform GUI apps. It was only after it failed in all of these that the server was discovered, despite the fact that "write once, run anywhere" is almost completely pointless when you control the hardware it runs on. The long startup times are a limitation of the implementation technology, not a principled design choice. See also: Jitterdämmerung ( http://blog.metaobject.com/2015/10/jitterdammerung.html http://blog.metaobject.com/2015/10/jitterdammerung.html )
9y ago

mike986 9y ago

they should have included luajit it's almost a dropin replacement for most projects

trynumber9 9y ago
It was included in the past [0]. It was removed to simplify the process and due to a "one implementation per language" rule. [0]: http://i.imgur.com/0xxOfVm.png http://i.imgur.com/0xxOfVm.png

deleted 9y ago

[deleted]

jackyb 9y ago

Does anybody know why Go binary-trees is so much slower than Java's?

trishume 9y ago
Binary-trees is allocation bottlenecked, and Java has bump pointer allocation where Go has slow allocation. The root cause being that Go's GC is non-moving so it can't do bump pointer allocation without fragmentation, so it has to do a whole bunch of computation to find a place for each node.
- jackyb 9y ago
  I see, thanks!

frik 9y ago

With this new revision this great benchmark got less useful. Why can't I compare: Lua vs LuaGIT PHP5 vs PHP7 vs HHVM vs Hack Perl5 vs Perl6 GNU GCC vs Intel C vs CLANG This new rule "one implementation per language" is so stupid!!! @debian: change it back

igouy 9y ago
"If you're interested in something not shown on the benchmarks game website then please take the program source code and the measurement scripts and publish your own measurements."

bedros 9y ago

where's D-lang? it would be interesting to compare D with C, C++, and Rust

wulfklaue 9y ago
Good luck on seeing D on that site. There seems to be a active war between the sites maintainer and D. There used to be D benchmarks on the old site and people tries to release new code to the site, but for some reason the current maintainer has a issue with D. And he thinks its no unfair that D is not resented when a lot of people look at his site and think: "If your not here as a language, then the language must be bad". Good luck on changing his mindset because i have seen topics going back years regarding D not being there. If you want a more impartial benchmark set: https://github.com/kostya/benchmarks https://github.com/kostya/benchmarks And where is D on those tests ;)
- igouy 9y ago
  > There seems to be a active war between the sites maintainer and D. No. > There used to be D benchmarks on the old site Back in early 2008. http://web.archive.org/web/20080730204710/http://shootout.alioth.debian.org:80/gp4/ http://web.archive.org/web/20080730204710/http://shootout.al... Look how many other language implementations were also shown back then and have not been shown since. > … for some reason the current maintainer has a issue with D No. > Good luck on changing his mindset… Make the measurements you want to see and publish them for everyone else to see.