4 ms·
> The onus is on the study's author to make it comprehensive and thorough, not the critics. It's always possible for the critics to go out there and do somethi
by davidw 14y ago
> The onus is on the study's author to make it comprehensive and thorough, not the critics.
It's always possible for the critics to go out there and do something better, rather than limiting themselves to pointing out flaws.
- politician 14y agoDoing both is also possible...
- calibraxis 14y agoThe problem is that it takes effort to make good comparative benchmarks — for a platform you're not totally immersed in, you'd ideally speak with the community about your methods. Furthermore, good benchmarks require real investigation explaining the differences; otherwise you could be measuring something silly. And you should graph some of the stats by time. Maybe in an ideal world, quality benchmarks would be commonplace. But in this world, doing good benchmarks in response to everyone's bad benchmarks is a heavy burden.
- pyre 14y agoSo what you're really saying is that no one is 'allowed' to make a critical remark unless they are willing to stand up and do better themselves?
- cantankerous 14y agoWriting good benchmarks is hard. Spotting bad ones is easier. I personally am glad that it's this way. Otherwise, we'd be taking bunk benchmarks at face value because we can't tell for ourselves.
- SkyMarshal 14y agoProblem of induction - for theories that cannot be logically or mathematically proven, and can only be confirmed by empirical evidence, no amount of confirming evidence can prove the theory true, but one single refuting observation can disprove it. So maybe the study's author only cared about webbit, ws4py, and Snap, and that was the implicit constraint of his test. If so, that's fine, and the submitter mangled the author's intent with an overly-editorialized, senationalized title. But if the author was actually using webbit, ws4py, and Snap as proxies for all of Java, Python, and Haskell, then no one should be surprised that it gets quickly shot down. There's value in critics debunking BS quickly, even if they don't provide a better alternative. Absence of bad knowledge is better than presence of it.