4 ms·
This isn't intended as snarky, but I don't understand what "it's the same as everything" is supposed to mean. What is "it"? What is "everything"? Why are they t
by qchris 5y ago
This isn't intended as snarky, but I don't understand what "it's the same as everything" is supposed to mean. What is "it"? What is "everything"? Why are they the same?
I'd also argue that your reduction of this problem sort of misses the point. One of the big problems with the way that studies are done is not that replication efforts aren't conclusive (it's very difficult to prove something doesn't exist), it's that a) non-replicable studies are generally considered as valuable as replicable ones, and b) as a result, it's extremely difficult to replicate many studies to begin with, because there's no incentive to take the time to make it possible. Even if the end result of a replication paper is "we couldn't produce the same results", the people working on it can say "this author's experiments were exceedingly difficult to even try to reproduce," or conversely "we didn't get the same results, but their data collection methods and analysis code were well-documented and accessible." That has a lot of value!
If you tried doing triple replication for every paper, I agree that maybe wouldn't be the best use of resources. But the current state of affairs is so bad that a well-organized drive to create single-attempt replication on a fraction of publicly-funded projects has the potential to be a significant driver of change.
- a-dub 5y ago"the same as everything" is an observation that often times verification/correctness/accuracy efforts are tossed aside in favor of new development and this is a truism across many fields. in science you see this as funding being committed to shiny new nature and science cover stories, with replication being left as an afterthought. in software you see this as heavy commitments to new features that drive revenue, with security/compliance/architecture and qa remaining underfunded and less respected. (until, of course, the problems that result from underfunding them make themselves apparent).