2 ms·

> Your #1 is false. A single unreplicated study is much less meaningful than several successful replications, but it does have some meaning ... Not to a scient

by lutusp 14y ago

> Your #1 is false. A single unreplicated study is much less meaningful than several successful replications, but it does have some meaning ... Not to a scientist. Without replication, a single study is conjecture, interesting but weightless. Are you not aware that an unreplicated study is a standing joke among scientists? Why do you think the gag science journal is named the "Journal of Irreproducibe Results"? Scientists find the name funny because (a) it's an inside joke and (b) it's a truth about science that member of the public just don't get. http://www.jir.com/ http://www.jir.com/ > That doesn't mean that a single study always has to provide its own replication Then it's a good thing I never said that. But a single study has no standing until and unless a disinterested, independent laboratory succeeds in replicating it. This issue is so critical to the future of psychology that, after a series of scandals about replication and credibility, a well-known figure in the field has made an emergency call to create replication circles among psychologists to (a) replicate each other's studies, and thereby try to (b) restore public confidence in the field. He says without a replication program, "I see a train wreck looming." http://www.nature.com/polopoly_fs/7.6716.1349271308!/suppinfoFile/Kahneman%20Letter.pdf http://www.nature.com/polopoly_fs/7.6716.1349271308!/suppinf... (PDF) Quote: "For all these reasons, right or wrong, your field is now the poster child for doubts about the integrity of psychological research. Your problem is not with the few people who have actively challenged the validity of some priming results. It is with the much larger population of colleagues who in the past accepted your surprising results as facts when they were published. ... My reason for writing this letter is that I see a train wreck looming." To see the importance of replication, all you need to do is read the scientific literature.

jleader 14y ago
To address your final sentence first, I don't think anyone here has tried to argue against the importance of replication. And yes, I'm familiar with the JIR. I was introduced to it about 35 years ago by my high school's astronomy teacher, Mr. Norm Sperling, now the editor and publisher. I wouldn't take any major actions based on a single study, other than trying to replicate or counter it, but that doesn't mean it has "no meaning". You have legitimate criticisms of the study, but you wrap them in overly-broad generalizations about what is and isn't science. You remind me of an old friend who had such a strict definition of "science" that he claimed that in 4 years as an undergraduate at Caltech, he'd only met one "scientist". So is Prof. Kahneman a scientist?

by lutusp 14y ago

jleader 14y ago

To address your final sentence first, I don't think anyone here has tried to argue against the importance of replication. And yes, I'm familiar with the JIR. I was introduced to it about 35 years ago by my high school's astronomy teacher, Mr. Norm Sperling, now the editor and publisher. I wouldn't take any major actions based on a single study, other than trying to replicate or counter it, but that doesn't mean it has "no meaning". You have legitimate criticisms of the study, but you wrap them in overly-broad generalizations about what is and isn't science. You remind me of an old friend who had such a strict definition of "science" that he claimed that in 4 years as an undergraduate at Caltech, he'd only met one "scientist". So is Prof. Kahneman a scientist?