3 ms·

> I don't think we do. I think we need to foster a culture of honesty and rigor. Of good science. Which is decidedly different from fostering a culture of "repe

> I don't think we do. I think we need to foster a culture of honesty and rigor. Of good science. Which is decidedly different from fostering a culture of "repetition" for its own sake. No one is proposing repetition for its own sake. The point of repetition is to create rigor, and you can't do rigorous science without repetition. > Paying for the cost of mountains upon mountains of lab techs and materials that it would require to replicate every study published in a major journal just isn't a good use of ever-dwindling science dollars. Replicate where it's not far off the critical path. Replicate where the study is going to have a profound effect on the direction of research in several labs. But don't just replicate because "science!" I could see a valid argument for only doing science that will be worth replicating, because if you don't bother to replicate you aren't really proving anything.

throwaway729 10y ago
I could see a valid argument for only doing science that will be worth replicating, because if you don't bother to replicate you aren't really proving anything. Exactly. A lot of the science I've done should not be replicated. If someone told me they wanted to replicate it, I would urge them not to. Not because I have something to hide. But because some other lab did something strictly superior that should be replicated instead. Or because the experiment asked the wrong questions. Or because the experiment itself could be pretty easily re-designed to avoid some pretty major threats. The problem is that is that hindsight really is 20/20. It's kind of impossible to ONLY do good science. So it's important to have the facility to recognize when science (including your own) isn't good -- or is good but not as good as something else -- and is therefore not worth replicating. I guess the two key insights are: 1. Not all science is worth replicating (either because it's too expensive or for some other reason). 2. Replication doesn't necessarily reduce doubt (particularly in the case of poorly designed experiments, or when the experiment asks the wrong questions).
- adrianm 10y ago
  This is a really good post which contributes to the conversation. Why make it on a throwaway account? We need more of this here!

throwaway729 10y ago

I could see a valid argument for only doing science that will be worth replicating, because if you don't bother to replicate you aren't really proving anything. Exactly. A lot of the science I've done should not be replicated. If someone told me they wanted to replicate it, I would urge them not to. Not because I have something to hide. But because some other lab did something strictly superior that should be replicated instead. Or because the experiment asked the wrong questions. Or because the experiment itself could be pretty easily re-designed to avoid some pretty major threats. The problem is that is that hindsight really is 20/20. It's kind of impossible to ONLY do good science. So it's important to have the facility to recognize when science (including your own) isn't good -- or is good but not as good as something else -- and is therefore not worth replicating. I guess the two key insights are: 1. Not all science is worth replicating (either because it's too expensive or for some other reason). 2. Replication doesn't necessarily reduce doubt (particularly in the case of poorly designed experiments, or when the experiment asks the wrong questions).

adrianm 10y ago
This is a really good post which contributes to the conversation. Why make it on a throwaway account? We need more of this here!