3 ms·
how many papers important results are simply bugs? Numerical code is already bug prone due to subtle and hard to test errors, research code that's not code rev
by lumost 8y ago
how many papers important results are simply bugs?
Numerical code is already bug prone due to subtle and hard to test errors, research code that's not code reviewed or necessarily even tested for correctness can easily generate important results erroneously.
It's also counter-productive not to publish the underlying source code for these papers, as it adds a barrier to other researchers applying the algorithm in new situations. I'd be interested in seeing if those 6% of papers which include the code get more citations than the population of papers which do not include code.
- MAXPOOL 8y ago> important results are simply bugs Probably none. If the paper is important and collects citations, the algorithm is in use. Computer science != working code. Code is required when you produce something where the scientific importance is less clear. There is need to provide more evidence. Many papers are just "Hey I made some some tweaks and it works in this particular case." Those papers should have working code.
- lumost 8y agoMost papers leverage a results table which compares the newly proposed approach with existing approaches. This section is baselines the results of the new approach with prior work and helps determine whether a new result is actually important or yet another way to achieve the same results as previous work. e.g. the tables on page 7 of this paper https://www.semanticscholar.org/paper/Automatic-Acquisition-of-Lexical-Formality-Brooke-Wang/823a397b29bf596e2734d3dff7ab5abec2f60ac9 https://www.semanticscholar.org/paper/Automatic-Acquisition-... These tables are generated using real implementations that may or may not be correct, and should be subject to review when the paper is published.