3 ms·

Although you are of course correct, I would caution against a complete dismissal of micro benchmarks. Firstly because they are a sort of marketing: I've peruse

Although you are of course correct, I would caution against a complete dismissal of micro benchmarks. Firstly because they are a sort of marketing: I've perused the shootout before and noticed an implementation get a higher score than I was expecting. This has sparked an interest in digging deeper into that implementation/language. Secondly because of the "can't even" problem i.e. why should I invest time investigating a new language if it can't even do simple benchmarks well? I appreciate that this isn't a particularly analytical view but I expect that it's not uncommon.

kibwen 11y ago
It's a mistake to consider the benchmarks game a "can't even" problem, the level of effort given to each language's implementations is all over the map. This also isn't an utter dismissal of the concept of microbenchmarks, it's an utter dismissal of the concept of microbenchmarks as used to compare language implementations. Something like the TechEmpower web framework benchmarks is at least marginally better because the tasks permit a wider variation of implementations and at least pretend to do some kind of useful tasks, instead of pretending to be an arbiter of implementation quality (which, as much as the benchmarks game disclaims it, is still what people take it as).
- igouy 11y ago
  >>the level of effort given to each language's implementations is all over the map<< Evidently the level of effort to contribute a working Rust spectral-norm program is too high for anyone ;-) >>pretending to be an arbiter of implementation quality (which, as much as the benchmarks game disclaims it, is still what people take it as).<< The benchmarks game is "just 10 tiny examples."
  - kibwen 11y ago
    > Evidently the level of effort to contribute a working > Rust spectral-norm program is too high for anyone ;-) Or they just don't care and refuse to let the benchmarks game hold their free time hostage. :P
    - igouy 11y ago
      That could be a respectable position; but not when "they" trash talk instead. That just looks bad.

kibwen 11y ago

It's a mistake to consider the benchmarks game a "can't even" problem, the level of effort given to each language's implementations is all over the map. This also isn't an utter dismissal of the concept of microbenchmarks, it's an utter dismissal of the concept of microbenchmarks as used to compare language implementations. Something like the TechEmpower web framework benchmarks is at least marginally better because the tasks permit a wider variation of implementations and at least pretend to do some kind of useful tasks, instead of pretending to be an arbiter of implementation quality (which, as much as the benchmarks game disclaims it, is still what people take it as).

igouy 11y ago
>>the level of effort given to each language's implementations is all over the map<< Evidently the level of effort to contribute a working Rust spectral-norm program is too high for anyone ;-) >>pretending to be an arbiter of implementation quality (which, as much as the benchmarks game disclaims it, is still what people take it as).<< The benchmarks game is "just 10 tiny examples."
- kibwen 11y ago
  > Evidently the level of effort to contribute a working > Rust spectral-norm program is too high for anyone ;-) Or they just don't care and refuse to let the benchmarks game hold their free time hostage. :P
  - igouy 11y ago
    That could be a respectable position; but not when "they" trash talk instead. That just looks bad.