6 ms·

The Go Language is faster than the Computer Language Benchmarks Game thinks

39 points by gbin 14y ago

Uchikoma 14y ago
As long as someone in our industry writes "smoking scala, clojure, java and lisp" we have not progressed from being children, playing with toys in our sandbox with a mine-is-bigger-than-yours attitude. Sad.
- ryeguy 14y ago
  Oh please. It's a simple informal choice of words. To many people, programming languages are toys too, because programming is a hobby. It doesn't have to be all uptight super business speak all the time.
  - silentOpen 14y ago
    It doesn't have to be tribal, either.
    - silentOpen 14y ago
      Is it really so hard to understand that the language shootout promotes the same sort of (misinformed) tribalism-by-language that the blog post author both dislikes (otherwise, why concern yourself over relatively pointless microbenchmarks of single implementations?) and reinforces (apparently "smoking" other languages is good?). What happened to objective assessment?
      - igouy 14y ago
        Is it really so hard to understand that it hasn't been called the language shootout for over 5 years - and you are promoting "the same sort of (misinformed) tribalism-by-language" by talking about it in those terms? My guess is that your mistaken view of what the benchmarks game promotes reflects a lack of familiarity with what the website says about itself on the homepage and the Help page and the Conclusions page and ...
        silentOpen 14y ago
        Is it really so hard to understand that it hasn't been called the language shootout for over 5 years - and you are promoting "the same sort of (misinformed) tribalism-by-language" by talking about it in those terms? The "benchmarks game" is largely the same as when it was called the "language shootout" now with more caveats plastered about. Please explain to me which language tribe I am promoting by calling the "benchmarks game" by its original name? Is it the tribe of skeptics? Perhaps use an HTTP 301 to redirect <http://shootout.alioth.debian.org/&#62 http://shootout.alioth.debian.org/> to <http://microbenchmarks.alioth.debian.org/&#62 http://microbenchmarks.alioth.debian.org/> and expunge "shootout" from all names? My guess is that your mistaken view of what the benchmarks game promotes reflects a lack of familiarity with what the website says about itself on the homepage and the Help page and the Conclusions page and ... Instead of guessing what my "view" "reflects", consider what the "benchmarks game" has promoted in TFA: Title: "The Go Language is faster than the computer language benchmarks game thinks it is" Well, this isn't what the "benchmarks game" says it measures anywhere (in fact, the site explicitly decries this interpretation) and yet this is what it indirectly promotes in various language communities (and linkbait tastes good). Conclusion: "So Go should be between C and Clean, smoking scala, clojure, java and lisp." Again, this isn't what the site says it measures (it measures some aspects of some implementations, not languages) and yet this is how it is portrayed by third parties. I understand that it is difficult to give people a lot of 'easy' data and simultaneously educate them about the reasonable limits of inference based on those data. I understand that the benchmarks game has taken great pains to try to educate its readers and inform their conclusions. I don't know if it will ever be enough -- people (as we can see) like easy answers and aren't willing to 0.) do the work to get more complete data sets and 1.) temper their findings with objective reality instead of drawing misinformed conclusions. Do you see how the easily-digestible plots (indirectly) promote tribalism over rationality? Do you see how a plot is worth a thousand words?
        igouy 14y ago
        Talking about the benchmarks game as a shootout promotes "the same sort of (misinformed) tribalism-by-language" -- I didn't say you were promoting a particular language tribe. >>Perhaps use an HTTP 301 to redirect<< Perhaps that was understood 5 years ago, and there are obstacles to that approach. (Not that I think that's actually why people like to talk about the shootout.) >>promoted in TFA<< Here you are -- http://news.ycombinator.com/item?id=4544096 http://news.ycombinator.com/item?id=4544096 He's talking about this -- https://groups.google.com/d/topic/golang-nuts/KqA1Jlpu2nM/discussion https://groups.google.com/d/topic/golang-nuts/KqA1Jlpu2nM/di... He came up with it all by himself, the same way people make factorial performance comparisons. >>(and linkbait tastes good)<< This kind of traffic-less blog post isn't a benefit -- stackoverflow is by far the main source of link traffic, and much more traffic comes from direct search. >>Do you see how the easily-digestible plots (indirectly) promote tribalism over rationality?<< Do you see how you can intervene and explain what the plots actually say?
anuraj 14y ago
Go is unlikely to be faster than Java (JVM is very optimized) in the near future. So rerun your benchmarks!
- Tuna-Fish 14y ago
  In a sufficiently short test (where the jit cannot show it's chops) I'd expect go to win. The tests in the benchmark game are totally useless anyway. When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Given such a small code snippet, the CPU will never fail a (theoretically predictable) branch, will never spill L1, and, on SNB, will in fact never even have to decode an instruction after the uOP cache is primed.
  - batista 14y ago
    >When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Even if that's the case, one would assume that would hold for ALL languages, so you get a fair comparison at that...
    - lmm 14y ago
      In the same way that a bicycle race is a fair way to compare sprinters as long as you have all of them ride bicycles. It's a fair benchmark, but it's not benchmarking the case that's actually important.
      - batista 14y ago
        No, it's like comparing sprinters by having them run, and someone telling you: "hey, the problem is not that sprinter X is ten times slower. The problem is what you want to do in the real world. Do you want to go from A to B? Why not take the bus, etc". [I'm not saying this about the L1i cache case specially, which is a good point, but rather for the prevalent response to any benchmark in Go-land, a defensive attitude which I have not witnessed in any other language community. Usually Python, JS, Ruby, Rust etc guys get on to fixing such microbenchmark behavior or explain why it's as it is. Go guys just propose you forget about it and rewrite your code in another way]. I don't care about a specific real world case of getting from A to B, or how it can be done faster in another way. I care about measuring sprinters. The case that's important here is benchmarking itself. How fast each and every instruction of a similar type executes in a language. E.g if I do: for i in range(10): print i and for i:=0; i<10; i++ { fmt.Print(i); } I don't care if this kind of code is not representative of an actual program, I don't care if the code inside the loop might take more time in most cases, I don't care if I can write some particular program using some other structure. I only want to know why this takes X time in Python and c*X time in Go, and if the c factor can be improved. Suggestions about "real world programs" and "write this another way and then measure" in this regard are counter-productive, because they focus not on raw benchmarking the language but on specific cases.
jbellis 14y ago
A trivial benchmark that doesn't actually run long enough or do enough allocation to exercise the GC isn't going to be representative of real-world performance in any case.
- igouy 14y ago
  A trivial assertion without supporting evidence for the claims.
CJefferson 14y ago
The title is misleading, and the article is hard to read. The central complaint seems to be that the language shoot-out uses the 'go' compiler. This doesn't seem surprisingly, seeing as this is what the go website tells users to use. There is however a gcc frontend for go, complete as of July and still not in many distributions (this isn't a go problem, man distros haven't updated yet to gcc 4.7.1). So the real headline should be 'Computer Language Benchmarks uses google's go compiler, gccgo is faster'
- markokocic 14y ago
  That is one of the things I miss in Shootout. Before, there were allowed multiple implementations for each language, and users could see how big difference between Lua and Luajit is, or how Python2 compares to Python 3. Now that there is only one implementation for each language allowed, some important information is missing from the site.
  - masklinn 14y ago
    > Now that there is only one implementation for each language allowed Not that they respect that rule of course: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=yarv&lang2=jruby http://shootout.alioth.debian.org/u32/benchmark.php?test=all...
  - malkia 14y ago
    Wait... Is luajit is back in the benchmark shootout?
    - masklinn 14y ago
      no
  - deleted 14y ago
    [deleted]
  - igouy 14y ago
    "If you're interested in something not shown here then please take the program source code and the measurement scripts and publish your own measurements." http://shootout.alioth.debian.org/ http://shootout.alioth.debian.org/
SeanDav 14y ago
OT: I am a bit of a fan of dark backgrounds but you got to be careful with the colours for links etc. Dark grey on Darker Grey is not very visible. Additionally the article is not clearly laid out and if it didn't have "Go" in the title would not have had a chance at high front page spot on HN. Oh well such is life at the moment.
- gbin 14y ago
  Indeed, fixed the horrible link color. I also agree, Go is really hyped at the moment.
batista 14y ago
>*The Go Language is faster than the Computer Language Benchmarks Game thinks That's what Go advocates always say, but, for a static language with tight memory structures, it's not that fast at all. Now, he does compile with gcc go, which is faster, but using the go tool I frequently find for lots of use cases it's about 50-100% faster than Python, or 2-5 times slower than Java. Not every impressive -- it probably needs a better compiler with more optimizations and a better GC. When you point that out the the golang mailing list, it's mostly met with la-la-hands-in-the-ears denial, and suggestions to rewrite your program in a smarter way and micro-optimize it. But I don't want to find the optimal algorithm for something -- I want to compare speed, and for that same-ish algorithms in same-ish languages are fine. (Though, Russ Cox said the current HEAD has a lot of optimizations and is quite faster, so things might be improving there).
- ramchip 14y ago
  What do you mean by "100% faster"?
  - batista 14y ago
    Takes half the time to do the same task, how about it?
- tptacek 14y ago
  Not meaning this snarkily, but: if you write a program in Java and it's 2x faster than a Go program written in the same style, why not just write in Java?
  - batista 14y ago
    Well, and I mean this kinda snarkily, who said people don't? I, for one, found that for some kinds of work I do, Go doesn't give that much of a performance advantage, so I'm exploring other languages. That said, even if Java gives you 2x, that doesn't mean that you shouldn't complain about Go. E.g you might like Go's semantics and syntax, or it might have other benefits for you. That you like those things about Go, doesn't mean you don't get to say "shame about the performance".
markokrajnc 14y ago
I tested simple a simple Go program with a for loop counting from 1 to several millions and making simple integer calculations and the same with Java. Java was 4-5x faster than Go. It looks like HotSpot VM is preety much optimized and that Go still needs a lot of compiler optimizations... I know that this test isn´t representative and complete, but it is a good smoke test for first comparison. C and Go should be faster than Java in pure integer numerics (or at least as fast as Java)...
- supersillyus 14y ago
  Why should C and Go (AOT compiled) be faster than Java for pure integer numerics? For long-running simple tight loops of numerics, I'd assume HotSpot would be faster.
  - igouy 14y ago
    Don't assume -- measure.
- igouy 14y ago
  >>but it is a good smoke test for first comparison<< No it isn't. http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#benchmarking_simple http://www.oracle.com/technetwork/java/hotspotfaq-138619.htm... http://www.azulsystems.com/events/javaone_2002/microbenchmarks.pdf http://www.azulsystems.com/events/javaone_2002/microbenchmar... http://www.ibm.com/developerworks/java/library/j-jtp02225/index.html http://www.ibm.com/developerworks/java/library/j-jtp02225/in...
masklinn 14y ago
1. Languages are not fast (or slow), implementations are. The language semantics can help or hinder, but in and of itself a language doesn't run. 2. This is old news. The Shootout was debatable before, but 12~18 months ago it suddenly decided that only one implementation of each language (picked completely arbitrarily and according to no clear cut rules, and sometimes allowing two implementations e.g. Python gets CPython 3, Lua gets Lua, but Ruby gets both MRI 1.9 and JRuby, Erlang gets HiPE and Javascript gets V8) would be allowed after — as far as I understand — a spat with the pypy team.
- wheaties 14y ago
  JRuby really is a different language than Ruby. Not only is it built on a different ecosystem of libraries (anything on the JVM vs anything for Ruby) but there are language extensions that make it operate differently (Java MBeans being one of them.)
  - igouy 14y ago
    The JRuby measurements shown on the benchmarks game website are for plain Ruby programs without any language extensions, but using -Xcompile.invokedynamic=true
  - masklinn 14y ago
    > JRuby really is a different language than Ruby. No, JRuby is a JVM-based implementation of Ruby. > Not only is it built on a different ecosystem of libraries (anything on the JVM vs anything for Ruby) Irrelevant, you can make the same claim about any impementation of a language: GHC has extensions to Haskell, why isn't JHC benched as well? IronPython can use .Net libraries but not Python C-API libraries, why isn't it there? > there are language extensions Which are not used in the shootout and — again – are irrelevant anyway.
- igouy 14y ago
  "If you're interested in something not shown here then please take the program source code and the measurement scripts and publish your own measurements." http://shootout.alioth.debian.org/ http://shootout.alioth.debian.org/
Rickasaurus 14y ago
One datapoint is not meaningful here. Give it a go with the entire suite.
halostatue 14y ago
Personally, I'm surprised that anyone takes the shootout seriously. In my experience, benchmarks of this sort are about as useful and trustworthy as GPU benchmark results provided by a GPU vendor. After having spent a little while looking over the site, I've got to give igouy props for fixing many of the worst problems that existed on the site a few years back (some of the micro-benchmarks that used to exist were questionable, at best; some of the implementations were even worse). It is a far better presented, qualified, and curated site than it used to be. I still don't think it's a useful site and that people would be better off ignoring it, but with the changes made, I don't think it's a harmful site anymore.
- igouy 14y ago
  >>useful<< "#1. To show working programs written in less familiar programming languages" >>trustworthy<< Are you saying that you believe the measurements to be falsified?
  - halostatue 14y ago
    Benchmarking is, in general, useless without context. I believe that something like the CLBG fails "useful" in the sense that the curated programs are themselves of limited utility. Showing different benchmark implementations illustrates some things about different programming languages, but IMO these differences are of limited utility when learning languages. Do I believe that your measurements are falsified? No. But generalized benchmarks (that is, without meaningful context to the problem that you're trying to solve) are truthy, at best—sort of like statistics presented out of context (such as the 47% figure floating around in political circles). As I said, I think the way that the CLBG is presented now is much better than it was presented as the Shootout. I just don't think that it's useful toward real software development.
    - igouy 14y ago
      Saying "limited utility" says "useful" for something, and therefore not "useless". The measurements are The Truth about their very specific context -- they are not a Prophesy.

39 points by gbin 14y ago

As long as someone in our industry writes "smoking scala, clojure, java and lisp" we have not progressed from being children, playing with toys in our sandbox with a mine-is-bigger-than-yours attitude. Sad.

ryeguy 14y ago
Oh please. It's a simple informal choice of words. To many people, programming languages are toys too, because programming is a hobby. It doesn't have to be all uptight super business speak all the time.
- silentOpen 14y ago
  It doesn't have to be tribal, either.
  - silentOpen 14y ago
    Is it really so hard to understand that the language shootout promotes the same sort of (misinformed) tribalism-by-language that the blog post author both dislikes (otherwise, why concern yourself over relatively pointless microbenchmarks of single implementations?) and reinforces (apparently "smoking" other languages is good?). What happened to objective assessment?
    - igouy 14y ago
      Is it really so hard to understand that it hasn't been called the language shootout for over 5 years - and you are promoting "the same sort of (misinformed) tribalism-by-language" by talking about it in those terms? My guess is that your mistaken view of what the benchmarks game promotes reflects a lack of familiarity with what the website says about itself on the homepage and the Help page and the Conclusions page and ...
      - silentOpen 14y ago
        Is it really so hard to understand that it hasn't been called the language shootout for over 5 years - and you are promoting "the same sort of (misinformed) tribalism-by-language" by talking about it in those terms? The "benchmarks game" is largely the same as when it was called the "language shootout" now with more caveats plastered about. Please explain to me which language tribe I am promoting by calling the "benchmarks game" by its original name? Is it the tribe of skeptics? Perhaps use an HTTP 301 to redirect <http://shootout.alioth.debian.org/&#62 http://shootout.alioth.debian.org/> to <http://microbenchmarks.alioth.debian.org/&#62 http://microbenchmarks.alioth.debian.org/> and expunge "shootout" from all names? My guess is that your mistaken view of what the benchmarks game promotes reflects a lack of familiarity with what the website says about itself on the homepage and the Help page and the Conclusions page and ... Instead of guessing what my "view" "reflects", consider what the "benchmarks game" has promoted in TFA: Title: "The Go Language is faster than the computer language benchmarks game thinks it is" Well, this isn't what the "benchmarks game" says it measures anywhere (in fact, the site explicitly decries this interpretation) and yet this is what it indirectly promotes in various language communities (and linkbait tastes good). Conclusion: "So Go should be between C and Clean, smoking scala, clojure, java and lisp." Again, this isn't what the site says it measures (it measures some aspects of some implementations, not languages) and yet this is how it is portrayed by third parties. I understand that it is difficult to give people a lot of 'easy' data and simultaneously educate them about the reasonable limits of inference based on those data. I understand that the benchmarks game has taken great pains to try to educate its readers and inform their conclusions. I don't know if it will ever be enough -- people (as we can see) like easy answers and aren't willing to 0.) do the work to get more complete data sets and 1.) temper their findings with objective reality instead of drawing misinformed conclusions. Do you see how the easily-digestible plots (indirectly) promote tribalism over rationality? Do you see how a plot is worth a thousand words?
        igouy 14y ago
        Talking about the benchmarks game as a shootout promotes "the same sort of (misinformed) tribalism-by-language" -- I didn't say you were promoting a particular language tribe. >>Perhaps use an HTTP 301 to redirect<< Perhaps that was understood 5 years ago, and there are obstacles to that approach. (Not that I think that's actually why people like to talk about the shootout.) >>promoted in TFA<< Here you are -- http://news.ycombinator.com/item?id=4544096 http://news.ycombinator.com/item?id=4544096 He's talking about this -- https://groups.google.com/d/topic/golang-nuts/KqA1Jlpu2nM/discussion https://groups.google.com/d/topic/golang-nuts/KqA1Jlpu2nM/di... He came up with it all by himself, the same way people make factorial performance comparisons. >>(and linkbait tastes good)<< This kind of traffic-less blog post isn't a benefit -- stackoverflow is by far the main source of link traffic, and much more traffic comes from direct search. >>Do you see how the easily-digestible plots (indirectly) promote tribalism over rationality?<< Do you see how you can intervene and explain what the plots actually say?

anuraj 14y ago

Go is unlikely to be faster than Java (JVM is very optimized) in the near future. So rerun your benchmarks!

Tuna-Fish 14y ago
In a sufficiently short test (where the jit cannot show it's chops) I'd expect go to win. The tests in the benchmark game are totally useless anyway. When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Given such a small code snippet, the CPU will never fail a (theoretically predictable) branch, will never spill L1, and, on SNB, will in fact never even have to decode an instruction after the uOP cache is primed.
- batista 14y ago
  >When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Even if that's the case, one would assume that would hold for ALL languages, so you get a fair comparison at that...
  - lmm 14y ago
    In the same way that a bicycle race is a fair way to compare sprinters as long as you have all of them ride bicycles. It's a fair benchmark, but it's not benchmarking the case that's actually important.
    - batista 14y ago
      No, it's like comparing sprinters by having them run, and someone telling you: "hey, the problem is not that sprinter X is ten times slower. The problem is what you want to do in the real world. Do you want to go from A to B? Why not take the bus, etc". [I'm not saying this about the L1i cache case specially, which is a good point, but rather for the prevalent response to any benchmark in Go-land, a defensive attitude which I have not witnessed in any other language community. Usually Python, JS, Ruby, Rust etc guys get on to fixing such microbenchmark behavior or explain why it's as it is. Go guys just propose you forget about it and rewrite your code in another way]. I don't care about a specific real world case of getting from A to B, or how it can be done faster in another way. I care about measuring sprinters. The case that's important here is benchmarking itself. How fast each and every instruction of a similar type executes in a language. E.g if I do: for i in range(10): print i and for i:=0; i<10; i++ { fmt.Print(i); } I don't care if this kind of code is not representative of an actual program, I don't care if the code inside the loop might take more time in most cases, I don't care if I can write some particular program using some other structure. I only want to know why this takes X time in Python and c*X time in Go, and if the c factor can be improved. Suggestions about "real world programs" and "write this another way and then measure" in this regard are counter-productive, because they focus not on raw benchmarking the language but on specific cases.

jbellis 14y ago

A trivial benchmark that doesn't actually run long enough or do enough allocation to exercise the GC isn't going to be representative of real-world performance in any case.

igouy 14y ago
A trivial assertion without supporting evidence for the claims.

CJefferson 14y ago

The title is misleading, and the article is hard to read. The central complaint seems to be that the language shoot-out uses the 'go' compiler. This doesn't seem surprisingly, seeing as this is what the go website tells users to use. There is however a gcc frontend for go, complete as of July and still not in many distributions (this isn't a go problem, man distros haven't updated yet to gcc 4.7.1). So the real headline should be 'Computer Language Benchmarks uses google's go compiler, gccgo is faster'

markokocic 14y ago
That is one of the things I miss in Shootout. Before, there were allowed multiple implementations for each language, and users could see how big difference between Lua and Luajit is, or how Python2 compares to Python 3. Now that there is only one implementation for each language allowed, some important information is missing from the site.
- masklinn 14y ago
  > Now that there is only one implementation for each language allowed Not that they respect that rule of course: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=yarv&lang2=jruby http://shootout.alioth.debian.org/u32/benchmark.php?test=all...
- malkia 14y ago
  Wait... Is luajit is back in the benchmark shootout?
  - masklinn 14y ago
    no
- deleted 14y ago
  [deleted]
- igouy 14y ago
  "If you're interested in something not shown here then please take the program source code and the measurement scripts and publish your own measurements." http://shootout.alioth.debian.org/ http://shootout.alioth.debian.org/

SeanDav 14y ago

OT: I am a bit of a fan of dark backgrounds but you got to be careful with the colours for links etc. Dark grey on Darker Grey is not very visible. Additionally the article is not clearly laid out and if it didn't have "Go" in the title would not have had a chance at high front page spot on HN. Oh well such is life at the moment.

gbin 14y ago
Indeed, fixed the horrible link color. I also agree, Go is really hyped at the moment.

batista 14y ago

>*The Go Language is faster than the Computer Language Benchmarks Game thinks That's what Go advocates always say, but, for a static language with tight memory structures, it's not that fast at all. Now, he does compile with gcc go, which is faster, but using the go tool I frequently find for lots of use cases it's about 50-100% faster than Python, or 2-5 times slower than Java. Not every impressive -- it probably needs a better compiler with more optimizations and a better GC. When you point that out the the golang mailing list, it's mostly met with la-la-hands-in-the-ears denial, and suggestions to rewrite your program in a smarter way and micro-optimize it. But I don't want to find the optimal algorithm for something -- I want to compare speed, and for that same-ish algorithms in same-ish languages are fine. (Though, Russ Cox said the current HEAD has a lot of optimizations and is quite faster, so things might be improving there).

ramchip 14y ago
What do you mean by "100% faster"?
- batista 14y ago
  Takes half the time to do the same task, how about it?
tptacek 14y ago
Not meaning this snarkily, but: if you write a program in Java and it's 2x faster than a Go program written in the same style, why not just write in Java?
- batista 14y ago
  Well, and I mean this kinda snarkily, who said people don't? I, for one, found that for some kinds of work I do, Go doesn't give that much of a performance advantage, so I'm exploring other languages. That said, even if Java gives you 2x, that doesn't mean that you shouldn't complain about Go. E.g you might like Go's semantics and syntax, or it might have other benefits for you. That you like those things about Go, doesn't mean you don't get to say "shame about the performance".

markokrajnc 14y ago

I tested simple a simple Go program with a for loop counting from 1 to several millions and making simple integer calculations and the same with Java. Java was 4-5x faster than Go. It looks like HotSpot VM is preety much optimized and that Go still needs a lot of compiler optimizations... I know that this test isn´t representative and complete, but it is a good smoke test for first comparison. C and Go should be faster than Java in pure integer numerics (or at least as fast as Java)...

supersillyus 14y ago
Why should C and Go (AOT compiled) be faster than Java for pure integer numerics? For long-running simple tight loops of numerics, I'd assume HotSpot would be faster.
- igouy 14y ago
  Don't assume -- measure.
igouy 14y ago
>>but it is a good smoke test for first comparison<< No it isn't. http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#benchmarking_simple http://www.oracle.com/technetwork/java/hotspotfaq-138619.htm... http://www.azulsystems.com/events/javaone_2002/microbenchmarks.pdf http://www.azulsystems.com/events/javaone_2002/microbenchmar... http://www.ibm.com/developerworks/java/library/j-jtp02225/index.html http://www.ibm.com/developerworks/java/library/j-jtp02225/in...

masklinn 14y ago

1. Languages are not fast (or slow), implementations are. The language semantics can help or hinder, but in and of itself a language doesn't run. 2. This is old news. The Shootout was debatable before, but 12~18 months ago it suddenly decided that only one implementation of each language (picked completely arbitrarily and according to no clear cut rules, and sometimes allowing two implementations e.g. Python gets CPython 3, Lua gets Lua, but Ruby gets both MRI 1.9 and JRuby, Erlang gets HiPE and Javascript gets V8) would be allowed after — as far as I understand — a spat with the pypy team.

wheaties 14y ago
JRuby really is a different language than Ruby. Not only is it built on a different ecosystem of libraries (anything on the JVM vs anything for Ruby) but there are language extensions that make it operate differently (Java MBeans being one of them.)
- igouy 14y ago
  The JRuby measurements shown on the benchmarks game website are for plain Ruby programs without any language extensions, but using -Xcompile.invokedynamic=true
- masklinn 14y ago
  > JRuby really is a different language than Ruby. No, JRuby is a JVM-based implementation of Ruby. > Not only is it built on a different ecosystem of libraries (anything on the JVM vs anything for Ruby) Irrelevant, you can make the same claim about any impementation of a language: GHC has extensions to Haskell, why isn't JHC benched as well? IronPython can use .Net libraries but not Python C-API libraries, why isn't it there? > there are language extensions Which are not used in the shootout and — again – are irrelevant anyway.
igouy 14y ago
"If you're interested in something not shown here then please take the program source code and the measurement scripts and publish your own measurements." http://shootout.alioth.debian.org/ http://shootout.alioth.debian.org/

Rickasaurus 14y ago

One datapoint is not meaningful here. Give it a go with the entire suite.

halostatue 14y ago

Personally, I'm surprised that anyone takes the shootout seriously. In my experience, benchmarks of this sort are about as useful and trustworthy as GPU benchmark results provided by a GPU vendor. After having spent a little while looking over the site, I've got to give igouy props for fixing many of the worst problems that existed on the site a few years back (some of the micro-benchmarks that used to exist were questionable, at best; some of the implementations were even worse). It is a far better presented, qualified, and curated site than it used to be. I still don't think it's a useful site and that people would be better off ignoring it, but with the changes made, I don't think it's a harmful site anymore.

igouy 14y ago
>>useful<< "#1. To show working programs written in less familiar programming languages" >>trustworthy<< Are you saying that you believe the measurements to be falsified?
- halostatue 14y ago
  Benchmarking is, in general, useless without context. I believe that something like the CLBG fails "useful" in the sense that the curated programs are themselves of limited utility. Showing different benchmark implementations illustrates some things about different programming languages, but IMO these differences are of limited utility when learning languages. Do I believe that your measurements are falsified? No. But generalized benchmarks (that is, without meaningful context to the problem that you're trying to solve) are truthy, at best—sort of like statistics presented out of context (such as the 47% figure floating around in political circles). As I said, I think the way that the CLBG is presented now is much better than it was presented as the Shootout. I just don't think that it's useful toward real software development.
  - igouy 14y ago
    Saying "limited utility" says "useful" for something, and therefore not "useless". The measurements are The Truth about their very specific context -- they are not a Prophesy.