5 ms·

Questioning the Hawthorne effect (2009)

19 points by deverton 12y ago

eridius 12y ago
Have there been no experiments on the Hawthorne effect since then?
- lutusp 12y ago
  Psychologists very rarely replicate prior work -- after all, it's already been done. Why repeat what we already know? Richard P. Feynman dedicates a long section of his essay "Cargo Cult Science" to this deplorable part of psychological research: http://neurotheory.columbia.edu/~ken/cargo_cult.html http://neurotheory.columbia.edu/~ken/cargo_cult.html Quote: "She [a psychology student] was curious as to whether, if she changed the circumstances to Y, they would still do A. So her proposal was to do the experiment under circumstances Y and see if they still did A. "I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person--to do it under condition X to see if she could also get result A, and then change to Y and see if A changed. Then she would know the the real difference was the thing she thought she had under control. "She was very delighted with this new idea, and went to her professor. And his reply was, no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1947 or so, and it seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happened." I can tell you that this attitude hasn't changed since the time of Feynman's essay (1974), indeed it seems to have gotten worse. By the way, here's the "punch line" from Feynman's famous essay: "I think the educational and psychological studies I mentioned are examples of what I would like to call cargo cult science. In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head to headphones and bars of bamboo sticking out like antennas--he's the controller--and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land."
  - ExpiredLink 12y ago
    > Psychologists very rarely replicate prior work This is simply not the case - no matter what Feynman said. Just skim e.g. through the Journal of Personality and Social Psychology. Like in any other science you attract attention when you refute a well-known experiment.
    - lutusp 12y ago
      >> Psychologists very rarely replicate prior work > This is simply not the case - no matter what Feynman said. So Nobel Prizewinner Kahnemann was wrong? http://www.nature.com/news/nobel-laureate-challenges-psychologists-to-clean-up-their-act-1.11535 http://www.nature.com/news/nobel-laureate-challenges-psychol... Title: "Nobel laureate challenges psychologists to clean up their act" Quote: "Nobel prize-winner Daniel Kahneman has issued a strongly worded call to one group of psychologists to restore the credibility of their field by creating a replication ring to check each others’ results." Quote: "For all these reasons, right or wrong, your field is now the poster child for doubts about the integrity of psychological research,” Kahneman writes. “I believe that you should collectively do something about this mess.”" The problem is not the existence of perfunctory efforts to re-perform studies, The problem is the absence of serious replications. It's a very serious crisis. But don't take my word for it -- read the literature yourself. > Like in any other science you attract attention when you refute a well-known experiment. Yes, as though that happens, as though an existing psychological theory is falsified by a new one. What you need to realize is that when one study refutes another, no one notices because neither study addresses a falsifiable theory.
- bildung 12y ago
  I'm not from the US, but in my local experience both educational science and psychology are chronically underfunded compared to the natural sciences. It's pretty hard to receive grants for replicating works of others. Combine that with the recently posted publication bias in the social sciences (https://news.ycombinator.com/item?id=8241076 https://news.ycombinator.com/item?id=8241076) and you have the present situation.
- e2e8 12y ago
  I took a quick look at pubmed and found two studies. Both support the existence of an effect. http://www.ncbi.nlm.nih.gov/pubmed/25002555 http://www.ncbi.nlm.nih.gov/pubmed/25002555 http://www.ncbi.nlm.nih.gov/pubmed/25141012 http://www.ncbi.nlm.nih.gov/pubmed/25141012 And a systematic review that says that an effect exists but little is know about how it might affect results. http://www.ncbi.nlm.nih.gov/pubmed/24275499 http://www.ncbi.nlm.nih.gov/pubmed/24275499
ExpiredLink 12y ago
So what is the theory behind the Hawthorne effect? It is relatively easy to replicate those experiments. But effects like this remain 'unfounded' when they are not embedded into a larger theory. The 'larger theories' are the main problem of psychology and social sciences in general. In contrast to e.g. physics they don't have the luxury to produce theories in and for a context-free imaginative ideal world.

19 points by deverton 12y ago

eridius 12y ago

Have there been no experiments on the Hawthorne effect since then?

lutusp 12y ago
Psychologists very rarely replicate prior work -- after all, it's already been done. Why repeat what we already know? Richard P. Feynman dedicates a long section of his essay "Cargo Cult Science" to this deplorable part of psychological research: http://neurotheory.columbia.edu/~ken/cargo_cult.html http://neurotheory.columbia.edu/~ken/cargo_cult.html Quote: "She [a psychology student] was curious as to whether, if she changed the circumstances to Y, they would still do A. So her proposal was to do the experiment under circumstances Y and see if they still did A. "I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person--to do it under condition X to see if she could also get result A, and then change to Y and see if A changed. Then she would know the the real difference was the thing she thought she had under control. "She was very delighted with this new idea, and went to her professor. And his reply was, no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1947 or so, and it seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happened." I can tell you that this attitude hasn't changed since the time of Feynman's essay (1974), indeed it seems to have gotten worse. By the way, here's the "punch line" from Feynman's famous essay: "I think the educational and psychological studies I mentioned are examples of what I would like to call cargo cult science. In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head to headphones and bars of bamboo sticking out like antennas--he's the controller--and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land."
- ExpiredLink 12y ago
  > Psychologists very rarely replicate prior work This is simply not the case - no matter what Feynman said. Just skim e.g. through the Journal of Personality and Social Psychology. Like in any other science you attract attention when you refute a well-known experiment.
  - lutusp 12y ago
    >> Psychologists very rarely replicate prior work > This is simply not the case - no matter what Feynman said. So Nobel Prizewinner Kahnemann was wrong? http://www.nature.com/news/nobel-laureate-challenges-psychologists-to-clean-up-their-act-1.11535 http://www.nature.com/news/nobel-laureate-challenges-psychol... Title: "Nobel laureate challenges psychologists to clean up their act" Quote: "Nobel prize-winner Daniel Kahneman has issued a strongly worded call to one group of psychologists to restore the credibility of their field by creating a replication ring to check each others’ results." Quote: "For all these reasons, right or wrong, your field is now the poster child for doubts about the integrity of psychological research,” Kahneman writes. “I believe that you should collectively do something about this mess.”" The problem is not the existence of perfunctory efforts to re-perform studies, The problem is the absence of serious replications. It's a very serious crisis. But don't take my word for it -- read the literature yourself. > Like in any other science you attract attention when you refute a well-known experiment. Yes, as though that happens, as though an existing psychological theory is falsified by a new one. What you need to realize is that when one study refutes another, no one notices because neither study addresses a falsifiable theory.
bildung 12y ago
I'm not from the US, but in my local experience both educational science and psychology are chronically underfunded compared to the natural sciences. It's pretty hard to receive grants for replicating works of others. Combine that with the recently posted publication bias in the social sciences (https://news.ycombinator.com/item?id=8241076 https://news.ycombinator.com/item?id=8241076) and you have the present situation.
e2e8 12y ago
I took a quick look at pubmed and found two studies. Both support the existence of an effect. http://www.ncbi.nlm.nih.gov/pubmed/25002555 http://www.ncbi.nlm.nih.gov/pubmed/25002555 http://www.ncbi.nlm.nih.gov/pubmed/25141012 http://www.ncbi.nlm.nih.gov/pubmed/25141012 And a systematic review that says that an effect exists but little is know about how it might affect results. http://www.ncbi.nlm.nih.gov/pubmed/24275499 http://www.ncbi.nlm.nih.gov/pubmed/24275499

ExpiredLink 12y ago

So what is the theory behind the Hawthorne effect? It is relatively easy to replicate those experiments. But effects like this remain 'unfounded' when they are not embedded into a larger theory. The 'larger theories' are the main problem of psychology and social sciences in general. In contrast to e.g. physics they don't have the luxury to produce theories in and for a context-free imaginative ideal world.