3 ms·
Of course, but the flip side is that same confluence of attributes has also exacerbated issues of reproducibility. Just as science and the methods/mediums by wh
by BadInformatics 6y ago
Of course, but the flip side is that same confluence of attributes has also exacerbated issues of reproducibility. Just as science and the methods/mediums by which we conduct/disseminate it have changed, so too should the standard of what is considered acceptable to reproduce. This is especially relevant given how much broader the societal and policy implications have become.
More concretely, it is 100% fair (and I might argue necessary) to demand more of our institutions and work to improve their failures. I'm sure many researchers have encountered publications of the form "we applied <proprietary model (TM)> (not explained) to <proprietary data> (partially explained) after <two sentence description of preprocessing> and obtained SOTA results!" in a reputable venue. Sure, this might be even less reproducible 200 years ago than now, but the authors would also be less likely to be competing with you for limited funding! Debating about the traditional definition of reproducibility has its place, but we should also be doing as much as possible to give reviewers and replicators a leg up. This is often flies in the face of many incentives the research community faces, but shifting blame to institutions by default (not saying you're doing this, but I've seen many who do) is taking the easy road out and does little to help the imbalanced ratio of discussion:progress.