3 ms·

Yes, I agree with the posts arguments. One HUGE thing it's missing, though, is the deliberate hacking of results to reach statistical significance. I'm willing

Yes, I agree with the posts arguments. One HUGE thing it's missing, though, is the deliberate hacking of results to reach statistical significance. I'm willing to bet that the results of a majority of psychology studies are not reproducible. In another lifetime, I worked as a research assistant at a very large, well-funded, Ivy League psychology lab. Talk about p-hacking. Our PI would go so far as to deny potential candidates entry into our study, as well as the therapy, simply because the PI thought these candidates wouldn't help the therapy our PI developed look good in our study. Note, these candidates did meet all our OFFICIAL study criteria for entry into the study.

kitsune_ 2y ago
This is basically just scientific fraud, no?
- hannob 2y ago
  It's also how most empirical science operates. If someone collects data and the study outcome is not preregistered, you can assume p-hacking. It would be implausible not to. And in most fields, preregistration is not common. (And even if there's preregistration, regularly people just switch their outcomes, and nobody cares.) And to play the devil's advocate: psychology is probably doing better these days than most other fields, because it's been the posterchild example of the replication crisis.
- nervousvarun 2y ago
  I personally lean to yes, but that's more about what people do with the results than the results themselves. Here's an infamous example: https://en.wikipedia.org/wiki/Milgram_experiment#Validity https://en.wikipedia.org/wiki/Milgram_experiment#Validity Honestly after reading that it seems impossible to really conclude anything...as it's just full of conflicting results...is that innately fraud? No but certainly careers/$ have been made from biased/agenda-driven interpretations which seems fraudulent.
- AdamN 2y ago
  There probably should be room in some of the social sciences for flexibility like this as long as it's called out right at the top as part of the experiment design so that the reader knows this is exploratory initial research being done for directional purposes - and that's it. Unfortunately as the movement from History, Philosophy, and the other liberal arts disciplines became 'sciencified', the ability to deliberate on something rigorous but still with enough room to explore has been sacrificed in favor of trying to be more like the physical sciences.
acchow 2y ago
"I'm willing to bet that the results of a majority of psychology studies are not reproducible" Indeed > Study replication rates were 23% for the Journal of Personality and Social Psychology, 48% for Journal of Experimental Psychology: Learning, Memory, and Cognition, and 38% for Psychological Science. Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%). https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
- left-struck 2y ago
  That is appalling, imho you might as well call an area of study, that has less than 50% reproducibility for studies published in “credible” journals, a pseudoscience.
  - hirvi74 2y ago
    I do not even think 50% is that good either and should be lumped in with pseudoscience as well.
Miraltar 2y ago
The author talked about it in length in an other blog post. https://www.experimental-history.com/p/im-so-sorry-for-psychologys-loss?utm_source=publication-search https://www.experimental-history.com/p/im-so-sorry-for-psych...

kitsune_ 2y ago

This is basically just scientific fraud, no?

hannob 2y ago
It's also how most empirical science operates. If someone collects data and the study outcome is not preregistered, you can assume p-hacking. It would be implausible not to. And in most fields, preregistration is not common. (And even if there's preregistration, regularly people just switch their outcomes, and nobody cares.) And to play the devil's advocate: psychology is probably doing better these days than most other fields, because it's been the posterchild example of the replication crisis.
nervousvarun 2y ago
I personally lean to yes, but that's more about what people do with the results than the results themselves. Here's an infamous example: https://en.wikipedia.org/wiki/Milgram_experiment#Validity https://en.wikipedia.org/wiki/Milgram_experiment#Validity Honestly after reading that it seems impossible to really conclude anything...as it's just full of conflicting results...is that innately fraud? No but certainly careers/$ have been made from biased/agenda-driven interpretations which seems fraudulent.
AdamN 2y ago
There probably should be room in some of the social sciences for flexibility like this as long as it's called out right at the top as part of the experiment design so that the reader knows this is exploratory initial research being done for directional purposes - and that's it. Unfortunately as the movement from History, Philosophy, and the other liberal arts disciplines became 'sciencified', the ability to deliberate on something rigorous but still with enough room to explore has been sacrificed in favor of trying to be more like the physical sciences.

acchow 2y ago

"I'm willing to bet that the results of a majority of psychology studies are not reproducible" Indeed > Study replication rates were 23% for the Journal of Personality and Social Psychology, 48% for Journal of Experimental Psychology: Learning, Memory, and Cognition, and 38% for Psychological Science. Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%). https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis

left-struck 2y ago
That is appalling, imho you might as well call an area of study, that has less than 50% reproducibility for studies published in “credible” journals, a pseudoscience.
- hirvi74 2y ago
  I do not even think 50% is that good either and should be lumped in with pseudoscience as well.

Miraltar 2y ago

The author talked about it in length in an other blog post. https://www.experimental-history.com/p/im-so-sorry-for-psychologys-loss?utm_source=publication-search https://www.experimental-history.com/p/im-so-sorry-for-psych...