4 ms·
I doubt (very profoundly) that giving the source will result in better peer reviews. I strongly agree with you on this point. The peer reviewer obviously won't
by Maro 16y ago
I doubt (very profoundly) that giving the source will result in better peer reviews.
I strongly agree with you on this point. The peer reviewer obviously won't have the time to check the code. But the very act of releasing the code will force the original author to be more thorough. In this sense, it will result in better, less buggy code, better results and better science.
Also, going one step beyond the peer reviewer, I think an interested party has the right to check the results for software and data bugs, given that we're in agreement that the peer reviewer probably won't have the time?
Another anecdote. We found a really interesting signal in a popular astro dataset, which would have put us on the frontpage of a few magazines for sure. After many sleepless nights, it turned out that it was a bug in the processing pipeline software. It took me an unreasonable amount of time to find the bug, because the software wasn't released, so I had to go hunting around for clues in the appropriate papers. (This was a very large survey, so there were tons of papers.) If we had higher software engineering standards, maybe this bug would not have made it into the production pipeline and pollute the dataset.