3 ms·
I think this is a fair point, but in my experience, having a concrete replica that people can start from (and compare to the paper) can make a year's difference
by dekhn 6y ago
I think this is a fair point, but in my experience, having a concrete replica that people can start from (and compare to the paper) can make a year's difference in speeding up progress.
Many times, I've read a paper, thought something was great, and then implemented the paper and failed to reproduce the author's results. In the cases where I've been able to compare my implementation to a reference on github, I often find the paper doesn't match the code, or a subtle data processing step was left out. Having a replica (a commit hash and a pointed to versioned input data) can often make a huge difference in time.
- dllthomas 6y agoYeah, I'm certainly not saying it isn't advisable or even important. I'm just saying it's not the same thing as replication.