6 ms·

I think I'm at the point now where i will consider the whole of psychology a castle of dishonest and incompetent studies until proven otherwise. The usual benef

by harry8 8y ago

I think I'm at the point now where i will consider the whole of psychology a castle of dishonest and incompetent studies until proven otherwise. The usual benefit given to scientists of the assumption that is not Alchemy had to go for psychology. It's too important and the abuse is too widespread. It really seems pervasive in the discipline at this point.

Karunamon 8y ago
There's also the reproduction crisis[1], if you weren't aware of it already. When your field can only have less than half of its top papers reproduced in full, something is very wrong with your field. It's to the point where one can safely assume that any psych paper (but especially social psych) is garbage until such time as it's been replicated a time or two. [1]: https://www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.18248 https://www.nature.com/news/over-half-of-psychology-studies-...
- Tycho 8y ago
  And consider this: in psychology we actually have the possibility of running and repeating controlled experiments. Think how bad things must be in other fields where that’s not the case.
- feanaro 8y ago
  > It's to the point where one can safely assume that any psych paper (but especially social psych) is garbage until such time as it's been replicated a time or two. Good, because it is probable that what we should assume this for any science[1]. [1]: http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 http://journals.plos.org/plosmedicine/article?id=10.1371/jou...
  - neuromantik8086 8y ago
    Everyone assumes that science is filled with neat, well-manicured lab notebooks, pristine methodology and fastidious characters. In reality, the current scientific culture is one of rampant distractions, where pretty much everyone is running around like a beheaded chicken. Woe unto those that might point out that a prominent researcher with tenure isn't using sound methods. That, as well as a generally toxic culture that makes Wall Street look good, was part of why I left. There have been some efforts towards creating a better culture (in particular, I've had my eye on Popper [1] and the "SciOps" movement and Software Carpentry [2], as well as various open access efforts). At the end of the day though, scarcity in terms of funding plus lack of job security plus medieval power dynamics are holding any substantial cultural shift back. [1] http://falsifiable.us/ http://falsifiable.us/ [2] https://software-carpentry.org/ https://software-carpentry.org/
- c3534l 8y ago
  Not only is the replication crisis not solely about psychology, the numbers are far worse than half for fields like chemistry, biology, and physics[0]. [0] https://en.wikipedia.org/wiki/Replication_crisis#Overall https://en.wikipedia.org/wiki/Replication_crisis#Overall
  - ummonk 8y ago
    That's not what the numbers there are saying though. They're saying that over half of scientists have failed to replicate a single study (either their own or someone else's). It says nothing about the replicability of most studies.
  - TangoTrotFox 8y ago
    This is completely incorrect, and that Wiki page has been written to be highly misleading to lead people to the same mistaken comparison as they're comparing entirely different numbers that look directly comparable. What that says is that e.g. 69% of scientists working in physics and engineering had failed to replicate some experiment at least once in their career. In other words if they performed 1000 replication attempts of other scientists' experiments and failed to reproduce the same results once, they would be counted in that survey. By contrast in psychology, one hallmark experiment tried to replicate the findings of 100 key studies in psychology from highly regarded psychology journals. And it was unable to replicate the findings of 64% of the studies. What's that number for physics? No idea other than that it's going to be much lower. Make sense? Wiki is, almost as a rule, just completely awful for contemporary or potentially controversial topics and this is a textbook example of that.
- rusk 8y ago
  The reproduction crisis affects all social sciences, and medicine to a certain degree. In fact anything where it's difficult to isolate and control for the phenomena are facing this. Climate science, Economics and food science are all suffering similar issues. I think this speaks more to the nature of academia, and the commercialisation of publication than the validity of the field in and of itself.
- UncleMeat 8y ago
  At least psych is addressing the reproduction crisis. Other fields are just ignoring it (cough, ML).
peterashford 8y ago
The Milgram experiments have been replicated many times in several different cultures. I wouldn't discount everything.
- dalbasal 8y ago
  But... it's taught alongside and on par with unproven theories, unreplicable expirements, and known to be fraudulent science. I mean, just because it's in a psychology textbook doesn't necessarily mean that's it's BS... But that's not a decent standard. Even astrology clears that bar.
- rusk 8y ago
  > The Milgram experiments have been replicated many times More recent critical analysis has poked a lot of holes in Milgram [0]. Important work no doubt but not quite the smoking gun it has been proclaimed to be. [0] https://www.theatlantic.com/health/archive/2015/01/rethinking-one-of-psychologys-most-infamous-experiments/384913/ https://www.theatlantic.com/health/archive/2015/01/rethinkin...
  - neilobremski 8y ago
    I started reading this link and this sentence popped out at me: > It’s a phenomenon that’s been used to explain atrocities from the Holocaust to the Vietnam War’s My Lai massacre to the abuse of prisoners at Abu Ghraib. This sounds an awful like the sentence in the Lifespan of a Lie: > It has been invoked to explain the massacre at My Lai during the Vietnam War, the Armenian genocide, and the horrors of the Holocaust. We seem to be using a lot of these experiments to explain the same things ...
    - rusk 8y ago
      Well the backdrop to the Milgram experiment was actually WW2 and trying to understand what happened. It was a key motivation. It seems to me that a lot of the seminal social psychology studies happen around this time ...

by harry8 8y ago

Karunamon 8y ago

There's also the reproduction crisis[1], if you weren't aware of it already. When your field can only have less than half of its top papers reproduced in full, something is very wrong with your field. It's to the point where one can safely assume that any psych paper (but especially social psych) is garbage until such time as it's been replicated a time or two. [1]: https://www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.18248 https://www.nature.com/news/over-half-of-psychology-studies-...

Tycho 8y ago
And consider this: in psychology we actually have the possibility of running and repeating controlled experiments. Think how bad things must be in other fields where that’s not the case.
feanaro 8y ago
> It's to the point where one can safely assume that any psych paper (but especially social psych) is garbage until such time as it's been replicated a time or two. Good, because it is probable that what we should assume this for any science[1]. [1]: http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 http://journals.plos.org/plosmedicine/article?id=10.1371/jou...
- neuromantik8086 8y ago
  Everyone assumes that science is filled with neat, well-manicured lab notebooks, pristine methodology and fastidious characters. In reality, the current scientific culture is one of rampant distractions, where pretty much everyone is running around like a beheaded chicken. Woe unto those that might point out that a prominent researcher with tenure isn't using sound methods. That, as well as a generally toxic culture that makes Wall Street look good, was part of why I left. There have been some efforts towards creating a better culture (in particular, I've had my eye on Popper [1] and the "SciOps" movement and Software Carpentry [2], as well as various open access efforts). At the end of the day though, scarcity in terms of funding plus lack of job security plus medieval power dynamics are holding any substantial cultural shift back. [1] http://falsifiable.us/ http://falsifiable.us/ [2] https://software-carpentry.org/ https://software-carpentry.org/
c3534l 8y ago
Not only is the replication crisis not solely about psychology, the numbers are far worse than half for fields like chemistry, biology, and physics[0]. [0] https://en.wikipedia.org/wiki/Replication_crisis#Overall https://en.wikipedia.org/wiki/Replication_crisis#Overall
- ummonk 8y ago
  That's not what the numbers there are saying though. They're saying that over half of scientists have failed to replicate a single study (either their own or someone else's). It says nothing about the replicability of most studies.
- TangoTrotFox 8y ago
  This is completely incorrect, and that Wiki page has been written to be highly misleading to lead people to the same mistaken comparison as they're comparing entirely different numbers that look directly comparable. What that says is that e.g. 69% of scientists working in physics and engineering had failed to replicate some experiment at least once in their career. In other words if they performed 1000 replication attempts of other scientists' experiments and failed to reproduce the same results once, they would be counted in that survey. By contrast in psychology, one hallmark experiment tried to replicate the findings of 100 key studies in psychology from highly regarded psychology journals. And it was unable to replicate the findings of 64% of the studies. What's that number for physics? No idea other than that it's going to be much lower. Make sense? Wiki is, almost as a rule, just completely awful for contemporary or potentially controversial topics and this is a textbook example of that.
rusk 8y ago
The reproduction crisis affects all social sciences, and medicine to a certain degree. In fact anything where it's difficult to isolate and control for the phenomena are facing this. Climate science, Economics and food science are all suffering similar issues. I think this speaks more to the nature of academia, and the commercialisation of publication than the validity of the field in and of itself.
UncleMeat 8y ago
At least psych is addressing the reproduction crisis. Other fields are just ignoring it (cough, ML).

peterashford 8y ago

The Milgram experiments have been replicated many times in several different cultures. I wouldn't discount everything.

dalbasal 8y ago
But... it's taught alongside and on par with unproven theories, unreplicable expirements, and known to be fraudulent science. I mean, just because it's in a psychology textbook doesn't necessarily mean that's it's BS... But that's not a decent standard. Even astrology clears that bar.
rusk 8y ago
> The Milgram experiments have been replicated many times More recent critical analysis has poked a lot of holes in Milgram [0]. Important work no doubt but not quite the smoking gun it has been proclaimed to be. [0] https://www.theatlantic.com/health/archive/2015/01/rethinking-one-of-psychologys-most-infamous-experiments/384913/ https://www.theatlantic.com/health/archive/2015/01/rethinkin...
- neilobremski 8y ago
  I started reading this link and this sentence popped out at me: > It’s a phenomenon that’s been used to explain atrocities from the Holocaust to the Vietnam War’s My Lai massacre to the abuse of prisoners at Abu Ghraib. This sounds an awful like the sentence in the Lifespan of a Lie: > It has been invoked to explain the massacre at My Lai during the Vietnam War, the Armenian genocide, and the horrors of the Holocaust. We seem to be using a lot of these experiments to explain the same things ...
  - rusk 8y ago
    Well the backdrop to the Milgram experiment was actually WW2 and trying to understand what happened. It was a key motivation. It seems to me that a lot of the seminal social psychology studies happen around this time ...