4 ms·

> … what the site shows most prominently… Program after Program after Program. https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.h

by igouy 8y ago

> … what the site shows most prominently… Program after Program after Program. https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/measurements/java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-java-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Of course, it's really not possible to show any computer program which is not written in a programming language. > … basically say that these comparisons mean nothing … Obviously not: please don't put your words in others' mouths. > … a lot of data that could be used for some really cool stuff… Take the data and do that really cool stuff. https://salsa.debian.org/benchmarksgame-team/benchmarksgame https://salsa.debian.org/benchmarksgame-team/benchmarksgame > … but there's no information about the market aspects. What do you mean by "market aspects" ? > …it may be true for some specific benchmark entries and maybe even all current ones… So it would be wrong to try to dismiss those measurements, based on nothing more than extrapolation about how some other toy benchmark program might perform. > … cool stuff the game could do … You're allowed your own personal opinion about what's cool and what isn't (and no doubt others will have their own personal opinion).

pron 8y ago
> Obviously not It's certainly not obvious to me what they comparisons mean (is it obvious to you?) and the authors of the site don't even state, let alone support, a hypothesis. They just say what the comparisons don't mean. So unless it's a work of art, I think observers can be forgiven for thinking it doesn't mean anything. > What do you mean by "market aspects"? I mean a hypothesis about "free market competition" driving people to write really fast programs. > You're allowed your own personal opinion about what's cool and what isn't True, but I meant something that could be used in a more rigorous/meaningful way (e.g. comparisons where only one variable changes). Meaning is perhaps also subjective, but I don't even know the subjective opinion of the meaning behind the game, because the site doesn't say. > https://salsa.debian.org/benchmarksgame-team/benchmarksgame https://salsa.debian.org/benchmarksgame-team/benchmarksgame Cool. Anyway, now that I've taken a slightly closer look, I realize I spoke too soon when I said that the SBCL vs. Java benchmark is "better than nothing." If, say, Java uses parallelism while SBCL doesn't, then if Java is 4x faster that may not be a "better than nothing" indication.
- igouy 8y ago
  > not obvious to me Program X measured less than Program Y. > … comparisons where only one variable changes… Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK). > I spoke too soon You chose to summarize data: "Java outperforms SBCL by a very wide margin -- 1.25-4x faster". Do you mean that you made mistakes when you made your summary.
  - pron 8y ago
    > Program X measured less than Program Y. That's like asking what does a program do and being told, "it does what the code says." Technically true, but not what we mean when we ask. Also, I don't think that's really what the site's authors thought. After all, when I click on the Haskell page, I don't see a comparison of Haskell programs, but a comparison of the fastest Haskell program with the fastest Java program. It's not obvious to me what that means, or even that it means anything at all. > Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK). That's exactly right, but because that's rather tedious, I would present just meaningful comparisons rather than non-meaningful ones, so no inter-language comparisons. I'm not saying they're always meaningless, but they can't be made with the data currently available on that site. Unfortunately, we cannot manufacture (non-fictitious) meaning out of data as we like. > Do you mean that you made mistakes when you made your summary. Yes. By "better than nothing" I overestimated the significance of results, at least in this case. After a closer look I realized that different programs use different algorithms, and so the comparison tells me nothing about the relative speed of those languages.
    - igouy 8y ago
      > That's like… No it is not. Program X may not have measured less than Program Y. > … meaningful comparisons… Earlier you complained there was "code that's very unidiomatic" but now you say "writing all the Lisp programs as if they were being written in Java" would be exactly right? 1) You seem to be contradicting your own argument. 2) Your desire for "rigorous" comparison is in-conflict with your desire for inter-language comparison. > … different programs use different algorithms… For some programs, for some definition of "algorithm". > … the comparison tells me nothing about the relative speed of those languages. What is the speed of a programming language?

by igouy 8y ago

pron 8y ago

> Obviously not It's certainly not obvious to me what they comparisons mean (is it obvious to you?) and the authors of the site don't even state, let alone support, a hypothesis. They just say what the comparisons don't mean. So unless it's a work of art, I think observers can be forgiven for thinking it doesn't mean anything. > What do you mean by "market aspects"? I mean a hypothesis about "free market competition" driving people to write really fast programs. > You're allowed your own personal opinion about what's cool and what isn't True, but I meant something that could be used in a more rigorous/meaningful way (e.g. comparisons where only one variable changes). Meaning is perhaps also subjective, but I don't even know the subjective opinion of the meaning behind the game, because the site doesn't say. > https://salsa.debian.org/benchmarksgame-team/benchmarksgame https://salsa.debian.org/benchmarksgame-team/benchmarksgame Cool. Anyway, now that I've taken a slightly closer look, I realize I spoke too soon when I said that the SBCL vs. Java benchmark is "better than nothing." If, say, Java uses parallelism while SBCL doesn't, then if Java is 4x faster that may not be a "better than nothing" indication.

igouy 8y ago
> not obvious to me Program X measured less than Program Y. > … comparisons where only one variable changes… Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK). > I spoke too soon You chose to summarize data: "Java outperforms SBCL by a very wide margin -- 1.25-4x faster". Do you mean that you made mistakes when you made your summary.
- pron 8y ago
  > Program X measured less than Program Y. That's like asking what does a program do and being told, "it does what the code says." Technically true, but not what we mean when we ask. Also, I don't think that's really what the site's authors thought. After all, when I click on the Haskell page, I don't see a comparison of Haskell programs, but a comparison of the fastest Haskell program with the fastest Java program. It's not obvious to me what that means, or even that it means anything at all. > Seems like your idea of a comparison, between for-example Lisp and Java programs, would require writing all the Lisp programs as if they were being written in Java. (We used up our "one variable change" with SBCL instead-of OpenJDK). That's exactly right, but because that's rather tedious, I would present just meaningful comparisons rather than non-meaningful ones, so no inter-language comparisons. I'm not saying they're always meaningless, but they can't be made with the data currently available on that site. Unfortunately, we cannot manufacture (non-fictitious) meaning out of data as we like. > Do you mean that you made mistakes when you made your summary. Yes. By "better than nothing" I overestimated the significance of results, at least in this case. After a closer look I realized that different programs use different algorithms, and so the comparison tells me nothing about the relative speed of those languages.
  - igouy 8y ago
    > That's like… No it is not. Program X may not have measured less than Program Y. > … meaningful comparisons… Earlier you complained there was "code that's very unidiomatic" but now you say "writing all the Lisp programs as if they were being written in Java" would be exactly right? 1) You seem to be contradicting your own argument. 2) Your desire for "rigorous" comparison is in-conflict with your desire for inter-language comparison. > … different programs use different algorithms… For some programs, for some definition of "algorithm". > … the comparison tells me nothing about the relative speed of those languages. What is the speed of a programming language?