3 ms·

Personally, I'm surprised that anyone takes the shootout seriously. In my experience, benchmarks of this sort are about as useful and trustworthy as GPU benchma

Personally, I'm surprised that anyone takes the shootout seriously. In my experience, benchmarks of this sort are about as useful and trustworthy as GPU benchmark results provided by a GPU vendor. After having spent a little while looking over the site, I've got to give igouy props for fixing many of the worst problems that existed on the site a few years back (some of the micro-benchmarks that used to exist were questionable, at best; some of the implementations were even worse). It is a far better presented, qualified, and curated site than it used to be. I still don't think it's a useful site and that people would be better off ignoring it, but with the changes made, I don't think it's a harmful site anymore.

igouy 14y ago
>>useful<< "#1. To show working programs written in less familiar programming languages" >>trustworthy<< Are you saying that you believe the measurements to be falsified?
- halostatue 14y ago
  Benchmarking is, in general, useless without context. I believe that something like the CLBG fails "useful" in the sense that the curated programs are themselves of limited utility. Showing different benchmark implementations illustrates some things about different programming languages, but IMO these differences are of limited utility when learning languages. Do I believe that your measurements are falsified? No. But generalized benchmarks (that is, without meaningful context to the problem that you're trying to solve) are truthy, at best—sort of like statistics presented out of context (such as the 47% figure floating around in political circles). As I said, I think the way that the CLBG is presented now is much better than it was presented as the Shootout. I just don't think that it's useful toward real software development.
  - igouy 14y ago
    Saying "limited utility" says "useful" for something, and therefore not "useless". The measurements are The Truth about their very specific context -- they are not a Prophesy.
  - igouy 14y ago
    I'm sorry about the reply I made a few days ago -- it doesn't seem to address what you wrote. >>limited utility<< I don't think there's anything on the website that suggests the benchmarks game provides some sort of perfect, definitive and ultimate statement about anything at all. On the contrary -- "Here you'll find provisional facts about the performance of programs written in ≈24 different programming languages for a dozen simple tasks." >>useful toward real software development<< That would be depend on how well informed the "real" software developers are, and there seem to be plenty of programmers with strange ideas about languages they haven't used.

igouy 14y ago

>>useful<< "#1. To show working programs written in less familiar programming languages" >>trustworthy<< Are you saying that you believe the measurements to be falsified?

halostatue 14y ago
Benchmarking is, in general, useless without context. I believe that something like the CLBG fails "useful" in the sense that the curated programs are themselves of limited utility. Showing different benchmark implementations illustrates some things about different programming languages, but IMO these differences are of limited utility when learning languages. Do I believe that your measurements are falsified? No. But generalized benchmarks (that is, without meaningful context to the problem that you're trying to solve) are truthy, at best—sort of like statistics presented out of context (such as the 47% figure floating around in political circles). As I said, I think the way that the CLBG is presented now is much better than it was presented as the Shootout. I just don't think that it's useful toward real software development.
- igouy 14y ago
  Saying "limited utility" says "useful" for something, and therefore not "useless". The measurements are The Truth about their very specific context -- they are not a Prophesy.
- igouy 14y ago
  I'm sorry about the reply I made a few days ago -- it doesn't seem to address what you wrote. >>limited utility<< I don't think there's anything on the website that suggests the benchmarks game provides some sort of perfect, definitive and ultimate statement about anything at all. On the contrary -- "Here you'll find provisional facts about the performance of programs written in ≈24 different programming languages for a dozen simple tasks." >>useful toward real software development<< That would be depend on how well informed the "real" software developers are, and there seem to be plenty of programmers with strange ideas about languages they haven't used.