4 ms·

The Reddit thread on this article is pretty good: https://www.reddit.com/r/science/comments/3imphg/scientists_replicated_100_recent_psychology/ https://www.redd

The Reddit thread on this article is pretty good: https://www.reddit.com/r/science/comments/3imphg/scientists_replicated_100_recent_psychology/ https://www.reddit.com/r/science/comments/3imphg/scientists_... There's a comment from one of the paper's co-authors that will probably need repeating here (https://www.reddit.com/r/science/comments/3imphg/scientists_replicated_100_recent_psychology/cuhtyv6 https://www.reddit.com/r/science/comments/3imphg/scientists_...): "To those wanting to dismiss psychological science as "cult science" based on these findings, note how ironic your response is. You're discrediting the very people whose data you are using to back up your claim. This massive, groundbreaking project was conducted on psychological science by psychological scientists. In my view, psychological scientists are among the most dedicated and rigorous scientists there are. No other field has had the courage to instantiate a project like this. And I am sure that many of you would be shocked to find out how low the reproducibility rates are in other fields. Problems of non-reproducibility, publication bias, data faking, lack of transparency, and the like plague every scientific field. The people you are labeling as "cult" scientists are leading the movement to improve science of all types in a much needed way." Also, the current top comment quotes the following from the paper: "Any temptation to interpret these results as a defeat for psychology, or science more generally, must contend with the fact that this project demonstrates science behaving as it should. Hypotheses abound that the present culture in science may be negatively affecting the reproducibility of findings. An ideological response would discount the arguments, discredit the sources, and proceed merrily along. The scientific process is not ideological. Science does not always provide comfort for what we wish to be; it confronts us with what is" And another co-author chimed in (https://www.reddit.com/r/science/comments/3imphg/scientists_replicated_100_recent_psychology/cui2nir https://www.reddit.com/r/science/comments/3imphg/scientists_...): "The real interesting part is seeing how other disciplines hold up in terms of reproducibility. A new project has been started: Reproducibility Project: Cancer Biology, they will try to replicate 50 studies. I am very curious how this will turn out, I highly encourage other disciplines to also start a reproducibility project to test how consistent their findings actually will be. I don't see these results as discouraging, instead, I see it as a big step in developing scientific methods. Now we know which methods and standards might be wrong, we can try to fix it (for example by developing guidlines)."

leereeves 11y ago
Worth repeating: > Problems of non-reproducibility, publication bias, data faking, lack of transparency, and the like plague every scientific field.
- thaumaturgy 11y ago
  Yes. There's a good article on this from a few years back that began to popularize a term for this, called the "Decline Effect": http://www.newyorker.com/magazine/2010/12/13/the-truth-wears-off http://www.newyorker.com/magazine/2010/12/13/the-truth-wears... (oblig.: author is disgraced science journalist Jonah Lehrer, but this is one of his pieces that has stood up to all scrutiny). The most straightforward explanation, mentioned in the article and in some of the /r/science thread comments, is that null results are much harder to publish than positive results. This is not limited to the field of psychology. All that said, I'm really excited by this paper and by the gradually increasing interest in this problem.
- paulmd 11y ago
  It's a particular problem with the "soft" sciences, though. For example, psychology is tough because you can't read people's minds, you have to go with how it affects their behavior in some measurable way or how they feel about it, and there's a tremendous number of confounding variables. Economics is another tough one, because experiments are not isolated from exogenous conditions and thus cannot really have a control. The whole thing is also highly interrelated with psychology as well. I'd throw poli-sci in that category too. At least in physics somebody else can set up the same experiment and run it again. Sure, the same tendencies to publication bias, etc apply, but usually within a few years to a decade such things are discovered, corrected, and the problem identified. It's much harder to identify what resulted in the change of outcome for a soft-science experiment.
  - thaumaturgy 11y ago
    This is addressed a bit in the Reddit thread, where the comments from practicing scientists seem to mostly agree with you. Mathematicians are claiming they aren't affected at all, physicists are claiming that disproving a major theory in physics would get you a Nobel instead of scorn, and the bioscientists saying it's a bigger problem in their field. That said, I think it's dangerous to give any branch of science an automatic pass on this. The fundamental causes of the Decline Effect aren't limited to the "soft" sciences, they just currently seem to have a more susceptible culture.

leereeves 11y ago

Worth repeating: > Problems of non-reproducibility, publication bias, data faking, lack of transparency, and the like plague every scientific field.

thaumaturgy 11y ago
Yes. There's a good article on this from a few years back that began to popularize a term for this, called the "Decline Effect": http://www.newyorker.com/magazine/2010/12/13/the-truth-wears-off http://www.newyorker.com/magazine/2010/12/13/the-truth-wears... (oblig.: author is disgraced science journalist Jonah Lehrer, but this is one of his pieces that has stood up to all scrutiny). The most straightforward explanation, mentioned in the article and in some of the /r/science thread comments, is that null results are much harder to publish than positive results. This is not limited to the field of psychology. All that said, I'm really excited by this paper and by the gradually increasing interest in this problem.
paulmd 11y ago
It's a particular problem with the "soft" sciences, though. For example, psychology is tough because you can't read people's minds, you have to go with how it affects their behavior in some measurable way or how they feel about it, and there's a tremendous number of confounding variables. Economics is another tough one, because experiments are not isolated from exogenous conditions and thus cannot really have a control. The whole thing is also highly interrelated with psychology as well. I'd throw poli-sci in that category too. At least in physics somebody else can set up the same experiment and run it again. Sure, the same tendencies to publication bias, etc apply, but usually within a few years to a decade such things are discovered, corrected, and the problem identified. It's much harder to identify what resulted in the change of outcome for a soft-science experiment.
- thaumaturgy 11y ago
  This is addressed a bit in the Reddit thread, where the comments from practicing scientists seem to mostly agree with you. Mathematicians are claiming they aren't affected at all, physicists are claiming that disproving a major theory in physics would get you a Nobel instead of scorn, and the bioscientists saying it's a bigger problem in their field. That said, I think it's dangerous to give any branch of science an automatic pass on this. The fundamental causes of the Decline Effect aren't limited to the "soft" sciences, they just currently seem to have a more susceptible culture.