21 ms·
Given that their code is on github and many people criticized the unidiomatic nature of their code, wouldn't it make sense to set up a competition to improve th
by sveme 12y ago
Given that their code is on github and many people criticized the unidiomatic nature of their code, wouldn't it make sense to set up a competition to improve the code? Everyone can submit pull requests with their improved codes and the authors should simply rerun the codes on a standard machine to get a benchmark. Obviously, a couple of rules should apply:
1. when you improve the code, do not move any elements outside the benchmarked language, i.e., don't use mex files in Matlab, c++ in R etc.
2. maybe create two or three different versions of your code - single processor, multiple cpus, gpu and run it on a single machine with a predefined set up.
This will remove at least some of the criticism thrown at the economists and has an additional benefit: researchers trying to find the best-suited language for their problem can not only use speed but also expressiveness, aesthetics, code length, boilerplate and ease of use to guide their decision.
- igouy 12y agohttp://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/