4 ms·
Sure; but lots of papers don't publish their raw data or their code. Or they do publish their code but its full of obvious problems. Abstract: "Algorithm X (ou
by josephg 5y ago
Sure; but lots of papers don't publish their raw data or their code.
Or they do publish their code but its full of obvious problems. Abstract: "Algorithm X (ours) outperforms algorithm Y". Then you go read the code, and they have a sloppy implementation of algorithm Y which is missing all the obvious optimizations. Actually, they often have sloppy implementations of both algorithms X and Y, because they're a student and they've never really optimized anything before.
So the conclusion of the paper is that some sloppily written code that you can't inspect outperforms some other sloppily written code that you also can't inspect. Wow.
- cuteboy19 5y agoEven I am guilty of this. There is basically no incentive for anyone to properly cross verify all these papers. At best the reviewers would check for any egregious errors in calculations.
- stnikolauswagne 5y agoSomewhat fun annectode about that: As my thesis in uni I was parallelizing some genetic analysis algorithm that a team published a paper about. I ended up with clean code with good speedup but my results were completely off from what was could be found in the initial paper. After weeks of pulling my hair out I looked into the actual code the paper was based on and found out that at some point they hardcoded a constant in such a way that there would always be a division by zero somewhere down the line which messed with their results. Been very sceptical of „peer-reviewed“ CS stuff ever since.