3 ms·
> although pointless and impractical I disagree. Consider computational biology papers – you should be able to show me code that takes the raw data and turns i
by buchanae 8y ago
> although pointless and impractical
I disagree. Consider computational biology papers – you should be able to show me code that takes the raw data and turns it into your results. Peer review is a sort of global, public code review in this case. There is a lot of value (education, validation, propagation, sharing, etc), and it's completely practical. Sadly, we (as a field) are nowhere near that, IMO.
- Gatsky 8y agoI have not said that reproducing a result is pointless or impractical. If you are a computational biologist, you will surely know that there are endless bench marking papers comparing published methods for sequence alignment, variant calling, differential expression, genome reconstruction, phylogeny reconstruction, cell segmentation etc etc. You will also know that if a methodology for producing a result is obscure, it will not be repeated or cited much, and a re-implementation of the method that is accessible will easily over take it. Published data sets are also endlessly re-analysed. If someone publishes a paper that is code heavy without any code, then they get chewed out on twitter or at conferences (rightly so). Top tier journals have also started requiring publication of custom code. My argument is that a 'reproducibility project' of the kind described above is pointless and impractical. I do not see evidence that this project has taught us anything.