4 ms·

That psychology has some serious problems as a science is not news. As the author says, there's lots of cargo cult science in it, and the publish or perish syst

That psychology has some serious problems as a science is not news. As the author says, there's lots of cargo cult science in it, and the publish or perish system doesn't help in the quest of raising quality of research. On the other hand, I'm baffled by the how the author reaches the conclusion that 90% of psychologic research is trash; they cherry pick some flawed, weak research and conclude from that all the body of research has to be thrown out of the window. That's simply dishonest and wrong; there's a solid body of research in experimental psychology that has been replicated (think psychology of memory, or psychology of perception, or psychology of learning). In doing so, the author reminds me of another anti-psychology evangelist, Paul Lutus, who also had a keen interest in debunking psychotherapy and psychiatry (you can Google him, he's all over Internet forums). As the next post of the author will be about psychiatric diagnosis, I guess he'll follow a similar direction. It's clever, really: discredit psychology from the root so that you can discredit your diagnosis.

0thgen 3y ago
But to be fair, multiple studies have suggested (for benchmark psychology experiments) a replication rate of less that 50% (https://royalsocietypublishing.org/doi/10.1098/rsos.231240 https://royalsocietypublishing.org/doi/10.1098/rsos.231240). I think the quality of psychology research is actually a bimodal distribution. In one mode, you have solid, replicable, serious science. In the other mode (like the priming studies), 90% trash might actually be a generous estimate
- remoquete 3y ago
  We can't experiment with humans, so social sciences will always have to deal with low quality, statistically noisy results.
  - p4ul 3y ago
    I would modify this slightly. We do experiment with humans; we just cannot do _certain kinds_ of experiments with humans because of ethical considerations. We constantly do clinic trials, which are essentially experiments with a high degree of controls and randomization. I think probably the major challenge for the social sciences is the nature of the phenomena under study. Psychologists, sociologists, and economists aren't studying physical quantities that can easily be measured. Instead, they typically study abstract and intangible variables. Measuring these accurately is very difficult.
    - 0thgen 3y ago
      Very well said. And this is partly why psychology researchers have to be so careful in (1) the claims they make, and (2) deciding what counts as a scientific contribution or career point.
- wolverine876 3y ago
  In at least the original landmark replication study, the studies actually replicated very well, but (~60% of the time) the signal wasn't as strong in the replicated study as in the original reported results. That tells us a lot both ways. Also, while they are valuable, the error could be in the replication study.
nonrandomstring 3y ago
There's a scale of disagreement and responses I see in people. Discomfort You can agree, and accept the evidence of something. And yet still say in good faith that you don't "go along" with that. My favourite idiom is when French people say "Sure, But that's not my philosophy". Personal philosophy is considered of equal importance to "scientific"/observed truth and expressing discomfort is okay. An example might be "AI being able to write poems as well as people". Even if a study shows that nobody can tell the machine from the poet, one can still say "I am not okay with that fact". The discontent makes no effort to challenge, except maybe personal withdrawal or denial. Scepticism This is healthy. Indeed it's part of the scientific method. We should always tolerate and maybe encourage sceptical thinking. The sceptic debates and disputes. But the sceptic is still open to being won over by arguments. The sceptic is expressive of her disagreement. Cynicism When we see again and again that a group or an idea is wrong, or persistently deceptive, the focus shifts from the ideas to the identity of the group. Nothing said by <psychologists or economists or politicians> or whatever can be trusted. A cynic can be healed, but often only by overwhelming new evidence or a deep personal shift in world view. Opposition The opponent has already decided in their mind they cannot or will not accept some ideas. The good opponent sets up an alternative hypothesis and engages in research to find new truths that will hopefully displace the old ones. The negative opponent spends time only on discrediting others. Iconoclasm and Vendettas This is the height of unreason. Being stuck in a black and white world, split over good and evil in which the opposition must be completely eradicated. I see this more and more in the world now. People, like the "internet campaigners" you mention make a career out of aggressive iconoclasm. They engage in "wars on..." and talk of "zero tolerance". --- Not sure I want to attach moral value judgement to any of these "levels", I'm sure I can be capable of all. Some things are clearly rotten in the world and people are tepidly accepting of them, whereas other things absolutely need consigning to history. But there's a lot of ugliness in the last two categories even if they are morally justified.
- kelseyfrog 3y ago
  There's definitely opposition and psychology iconoclasts floating around here. Others would label this as "trapped priors." I like to ask, "what evidence would change your mind?" to suss this out. Usually I get no response and the conversation dies. The other response is a standard of evidence unattainable - another way of justifying a trapped priors. If the evidence required outside of the realm of possibility, it's effectively the same thing.

0thgen 3y ago

But to be fair, multiple studies have suggested (for benchmark psychology experiments) a replication rate of less that 50% (https://royalsocietypublishing.org/doi/10.1098/rsos.231240 https://royalsocietypublishing.org/doi/10.1098/rsos.231240). I think the quality of psychology research is actually a bimodal distribution. In one mode, you have solid, replicable, serious science. In the other mode (like the priming studies), 90% trash might actually be a generous estimate

remoquete 3y ago
We can't experiment with humans, so social sciences will always have to deal with low quality, statistically noisy results.
- p4ul 3y ago
  I would modify this slightly. We do experiment with humans; we just cannot do _certain kinds_ of experiments with humans because of ethical considerations. We constantly do clinic trials, which are essentially experiments with a high degree of controls and randomization. I think probably the major challenge for the social sciences is the nature of the phenomena under study. Psychologists, sociologists, and economists aren't studying physical quantities that can easily be measured. Instead, they typically study abstract and intangible variables. Measuring these accurately is very difficult.
  - 0thgen 3y ago
    Very well said. And this is partly why psychology researchers have to be so careful in (1) the claims they make, and (2) deciding what counts as a scientific contribution or career point.
wolverine876 3y ago
In at least the original landmark replication study, the studies actually replicated very well, but (~60% of the time) the signal wasn't as strong in the replicated study as in the original reported results. That tells us a lot both ways. Also, while they are valuable, the error could be in the replication study.

nonrandomstring 3y ago

There's a scale of disagreement and responses I see in people. Discomfort You can agree, and accept the evidence of something. And yet still say in good faith that you don't "go along" with that. My favourite idiom is when French people say "Sure, But that's not my philosophy". Personal philosophy is considered of equal importance to "scientific"/observed truth and expressing discomfort is okay. An example might be "AI being able to write poems as well as people". Even if a study shows that nobody can tell the machine from the poet, one can still say "I am not okay with that fact". The discontent makes no effort to challenge, except maybe personal withdrawal or denial. Scepticism This is healthy. Indeed it's part of the scientific method. We should always tolerate and maybe encourage sceptical thinking. The sceptic debates and disputes. But the sceptic is still open to being won over by arguments. The sceptic is expressive of her disagreement. Cynicism When we see again and again that a group or an idea is wrong, or persistently deceptive, the focus shifts from the ideas to the identity of the group. Nothing said by <psychologists or economists or politicians> or whatever can be trusted. A cynic can be healed, but often only by overwhelming new evidence or a deep personal shift in world view. Opposition The opponent has already decided in their mind they cannot or will not accept some ideas. The good opponent sets up an alternative hypothesis and engages in research to find new truths that will hopefully displace the old ones. The negative opponent spends time only on discrediting others. Iconoclasm and Vendettas This is the height of unreason. Being stuck in a black and white world, split over good and evil in which the opposition must be completely eradicated. I see this more and more in the world now. People, like the "internet campaigners" you mention make a career out of aggressive iconoclasm. They engage in "wars on..." and talk of "zero tolerance". --- Not sure I want to attach moral value judgement to any of these "levels", I'm sure I can be capable of all. Some things are clearly rotten in the world and people are tepidly accepting of them, whereas other things absolutely need consigning to history. But there's a lot of ugliness in the last two categories even if they are morally justified.

kelseyfrog 3y ago
There's definitely opposition and psychology iconoclasts floating around here. Others would label this as "trapped priors." I like to ask, "what evidence would change your mind?" to suss this out. Usually I get no response and the conversation dies. The other response is a standard of evidence unattainable - another way of justifying a trapped priors. If the evidence required outside of the realm of possibility, it's effectively the same thing.