3 ms·

The repercussions are pretty depressing for social sciences. Especially Psych, which relies on a lot of "30 undergraduates were recruited" to demonstrate effect

The repercussions are pretty depressing for social sciences. Especially Psych, which relies on a lot of "30 undergraduates were recruited" to demonstrate effects. Particularly damaging is that this particular theory (thinking hard leads to drops in blood glucose levels) was widely promoted in the general public; I recall watching a TED talk about it, as well as a pop-sci book. There was an econtalk episode late last year with Brian Nosek about the Reproducability Project (http://www.econtalk.org/archives/2015/11/brian_nosek_on.html http://www.econtalk.org/archives/2015/11/brian_nosek_on.html). They went the further step of analyzing the wider field to see how bad the biases are. Previous attempts to call out replication (priming theory) did not go well, leading to a lot of hostility. Nosek's process is a bit more friendly, by inviting the original authors's critique early, before data collection. I figure it's a bit harder to blast the auditors for considering a replication study without appearing as though you doubt the effect is real yourself. They still got a bit of pushback, but nowhere near as acrimonious.

dwaltrip 10y ago
It sounds like some of these researchers need their own ego depleted, for the sake of science. Critical, well-intentioned analyses of a study by other groups should be a welcome thing, as it will lead to more robust results. It seems like the incentives have gotten way out of whack.
- jldugger 10y ago
  Bear in mind it's not just ego, but livelihood. People's grant lines and tenure cases rely in part on contributing to the development of the field, and having the basis of your research retracted usually undermines your tenure review and puts a hamper on continued grant funding. So it's not super surprising that people are defensive, but I'm always amazed at how publicly aggressive things can get.

dwaltrip 10y ago

It sounds like some of these researchers need their own ego depleted, for the sake of science. Critical, well-intentioned analyses of a study by other groups should be a welcome thing, as it will lead to more robust results. It seems like the incentives have gotten way out of whack.

jldugger 10y ago
Bear in mind it's not just ego, but livelihood. People's grant lines and tenure cases rely in part on contributing to the development of the field, and having the basis of your research retracted usually undermines your tenure review and puts a hamper on continued grant funding. So it's not super surprising that people are defensive, but I'm always amazed at how publicly aggressive things can get.