3 ms·

Here's the thing though: you're trying to solve a problem here which doesn't exist. In physical science, no one commits academic fraud by manipulating a differ

• __MatrixMan__ 2y ago
  I'm not really trying to address fraud. Most of the time when I try to recreate a computational result from a paper, things go poorly. I want to fix that. Recently I found one where the authors must've mislabeled something because the data for mutant A actually corresponded with the plot for mutant B. Other times it'll take days of tinkering just to figure out which versions of the dependencies are necessary to make it work at all. None of that sort of sleuthing should've required a human in the loop at all. I should be able to hold one thing constant (be it the data or the pipeline), change the other, and rebuild the paper to determine whether the conclusions are invariant to the change I made. Human patience for applying skepticism to complex things is scarce. I want to automate as much of replication as possible so that what skepticism is available is applied more effectively. It would just be a nicer world to live in.