3 ms·

Psychologists very rarely replicate prior work -- after all, it's already been done. Why repeat what we already know? Richard P. Feynman dedicates a long sectio

by lutusp 12y ago

Psychologists very rarely replicate prior work -- after all, it's already been done. Why repeat what we already know? Richard P. Feynman dedicates a long section of his essay "Cargo Cult Science" to this deplorable part of psychological research: http://neurotheory.columbia.edu/~ken/cargo_cult.html http://neurotheory.columbia.edu/~ken/cargo_cult.html Quote: "She [a psychology student] was curious as to whether, if she changed the circumstances to Y, they would still do A. So her proposal was to do the experiment under circumstances Y and see if they still did A. "I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person--to do it under condition X to see if she could also get result A, and then change to Y and see if A changed. Then she would know the the real difference was the thing she thought she had under control. "She was very delighted with this new idea, and went to her professor. And his reply was, no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1947 or so, and it seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happened." I can tell you that this attitude hasn't changed since the time of Feynman's essay (1974), indeed it seems to have gotten worse. By the way, here's the "punch line" from Feynman's famous essay: "I think the educational and psychological studies I mentioned are examples of what I would like to call cargo cult science. In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head to headphones and bars of bamboo sticking out like antennas--he's the controller--and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land."

ExpiredLink 12y ago
> Psychologists very rarely replicate prior work This is simply not the case - no matter what Feynman said. Just skim e.g. through the Journal of Personality and Social Psychology. Like in any other science you attract attention when you refute a well-known experiment.
- lutusp 12y ago
  >> Psychologists very rarely replicate prior work > This is simply not the case - no matter what Feynman said. So Nobel Prizewinner Kahnemann was wrong? http://www.nature.com/news/nobel-laureate-challenges-psychologists-to-clean-up-their-act-1.11535 http://www.nature.com/news/nobel-laureate-challenges-psychol... Title: "Nobel laureate challenges psychologists to clean up their act" Quote: "Nobel prize-winner Daniel Kahneman has issued a strongly worded call to one group of psychologists to restore the credibility of their field by creating a replication ring to check each others’ results." Quote: "For all these reasons, right or wrong, your field is now the poster child for doubts about the integrity of psychological research,” Kahneman writes. “I believe that you should collectively do something about this mess.”" The problem is not the existence of perfunctory efforts to re-perform studies, The problem is the absence of serious replications. It's a very serious crisis. But don't take my word for it -- read the literature yourself. > Like in any other science you attract attention when you refute a well-known experiment. Yes, as though that happens, as though an existing psychological theory is falsified by a new one. What you need to realize is that when one study refutes another, no one notices because neither study addresses a falsifiable theory.

by lutusp 12y ago

ExpiredLink 12y ago

> Psychologists very rarely replicate prior work This is simply not the case - no matter what Feynman said. Just skim e.g. through the Journal of Personality and Social Psychology. Like in any other science you attract attention when you refute a well-known experiment.

lutusp 12y ago
>> Psychologists very rarely replicate prior work > This is simply not the case - no matter what Feynman said. So Nobel Prizewinner Kahnemann was wrong? http://www.nature.com/news/nobel-laureate-challenges-psychologists-to-clean-up-their-act-1.11535 http://www.nature.com/news/nobel-laureate-challenges-psychol... Title: "Nobel laureate challenges psychologists to clean up their act" Quote: "Nobel prize-winner Daniel Kahneman has issued a strongly worded call to one group of psychologists to restore the credibility of their field by creating a replication ring to check each others’ results." Quote: "For all these reasons, right or wrong, your field is now the poster child for doubts about the integrity of psychological research,” Kahneman writes. “I believe that you should collectively do something about this mess.”" The problem is not the existence of perfunctory efforts to re-perform studies, The problem is the absence of serious replications. It's a very serious crisis. But don't take my word for it -- read the literature yourself. > Like in any other science you attract attention when you refute a well-known experiment. Yes, as though that happens, as though an existing psychological theory is falsified by a new one. What you need to realize is that when one study refutes another, no one notices because neither study addresses a falsifiable theory.