5 ms·
Almost no findings in psychology survive 20 years, almost all of them are determined to be fraudulent manipulation of data and findings or at the least fail to
by PaulKeeble 3y ago
Almost no findings in psychology survive 20 years, almost all of them are determined to be fraudulent manipulation of data and findings or at the least fail to be replicated. Psychology is not producing scientific findings as a field, it's completely ignorable and responsible for a large chunk of the science replication crisis.
- klysm 3y agoAn epistemological crisis rooted in a shitty incentive system and powered by bad statistics
- booleandilemma 3y agoIt sure does give you some cool factoids to bring up at parties though. Did you hear the one about the prison experiment?
- blululu 3y agoThe interpretation is never so simple. I'm not sure what that guy at Stanford was trying to prove but his research conclusively proved that tenured psych professors at leading universities can be wildly unethical. This discovery helped support the creation of institutional review boards across the country - one of the first major restrictions on academic freedom in the history of science.
- sandworm101 3y agoOr they proved that if you setup some people to work/live in a fake prison, they end up acting basically like all the other people who already work/live in realworld prisons. It's like having people sit in a MacDonald's for ten hours each day, then writing a paper about how subjects ate more fast food during the trial period. Breaking news: Starting a new job in Hawaii increases the likelihood that you will take up surfing.
- vgalin 3y agoIf this is an established fact, where could one learn more about this?
- taeric 3y agoIt is, at least, fairly well accepted as a thing. Name a "fact" that has been accepted as a learning, dive into it, find that it hasn't held up. :( Some are purely on the popular science side of failure. The 10k of practice was basically thrown out, but the original research still seems good. They never claimed "if you do 10k of work, you will be good." The "marshmallow test," though, seems completely tossed? Maybe there was something there? Anchoring and other items? Not sure how well those have survived. :( https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis looks to be a good article, though I haven't finished reading it.
- scythe 3y agoSome of Kahneman and Tversky's work was thrown out, but the majority of it replicates: https://www.nature.com/articles/s41562-020-0886-x https://www.nature.com/articles/s41562-020-0886-x
- PaulKeeble 3y agoThe Dunning Kruger data is also just a statistical manipulation.
- mike_hearn 3y agoWorse: the original Dunning Kruger experimental design doesn't even make sense. https://news.ycombinator.com/item?id=31119836 https://news.ycombinator.com/item?id=31119836
- endominus 3y agoPer https://www.gleech.org/psych https://www.gleech.org/psych: - Stanford Prison Experiment; Not an experiment, abuse was scripted, experimenter constantly intervened, reactions of participants were faked, and there was not even a scientific hypothesis they were testing. Definitely read the paper[0] debunking it. - Milgram Experiment: (the one where people were ordered to shock actors) No good evidence for it. Researchers did not follow script, implausible levels of agreement between different experiments. Killer line is, “only half of the people who undertook the experiment fully believed it was real and of those, 66% disobeyed the experimenter.” - Robber's Cave: (the one where two groups of kids immediately formed tribal hatred between one another) The conflict was orchestrated by experimenters and the experiment was actually repeated because the first time the kids absolutely refused to turn on one another. More information at [1]. - At best, weak evidence for implicit bias testing and stereotype threat. - Weak evidence of "facial feedback" (smiling causes a good mood and frowning causes a bad mood) - Good evidence against "ego depletion" (the idea that willpower is limited in a muscle-like fashion) - Mixed evidence for Dunning-Kruger effect - Questionable evidence for "hungry judge" effect (the idea that judicial sentences are massively more merciful in the morning and after a lunch recess due to "ego depletion" - this is also thoroughly debunked here [2]) - The 10,000 hours of practice leading to expertise idea has been disowned by its proponents - No good evidence that tailoring teaching to students’ preferred learning styles has any effect on objective measures of attainment. - No good evidence that brains contain one mind per hemisphere. i.e. the left-brain, right-brain split that people talk about, especially after the link between the hemispheres is severed. - No good evidence for left/right hemisphere dominance correlating with personality differences. Per https://danluu.com/dunning-kruger/ https://danluu.com/dunning-kruger/: - Most people talking about Dunning-Kruger have no idea what it actually means. The actual purported bias is much weaker than people claim - basically, that everyone either overestimates their ability, but that estimation is still positively correlated with actual ability, or estimated ability has basically no correlation with actual ability and everyone is just guessing. - Increasing your wealth does in fact make you happier at a predictable rate. There is no "plateau" of wealth or income - what appears to be a plateau is misleading displays of data. In effect, increasing income by a proportional rate will increase reported happiness by a fixed rate. For example, say you make $10, and your happiness is 50. Then, your income increases to $20 and your happiness increases to 60. Then, doubling your happiness is necessary to increase your happiness by 10. It's a logarithmic function; plotted on a standard axis, it looks like a plateau, but plotted on a logarithmic scale and it's a constantly increasing line. - Hedonic Adaptation (aka the hedonic treadmill) is a myth. Bad life events (divorce, disability, death of a loved one) all have negative long-term effects on happiness. Vice versa for positive events. [0]: https://www.gwern.net/docs/psychology/2019-letexier.pdf https://www.gwern.net/docs/psychology/2019-letexier.pdf [1]: https://www.theguardian.com/science/2018/apr/16/a-real-life-lord-of-the-flies-the-troubling-legacy-of-the-robbers-cave-experiment https://www.theguardian.com/science/2018/apr/16/a-real-life-... [2]: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html http://daniellakens.blogspot.com/2017/07/impossibly-hungry-j...
- posterboy 3y agoOr Psychology is difficult and having passed the the bar is not a quality controlled guarantee, same as for example nautical engineering isn't. That's not enough reason to say that boats don't float.
- NoMoreNicksLeft 3y agoWhat is psychology used for, and whatever those things are... are they still floating (so to speak)? Some of it is used in policy. So when some convicted murderer is out on furlough and kills again, am I wrong to wonder if that was some of the top-notch applied psychology at work?
- posterboy 3y agoThat's shifting the goal post and it suffers from survivor bias. The individual decision is met by few people. The policy is instated by very different people and informed by many factors outside the control of psychology, where false positives don't invalidate the whole system if the alternative is a slippery slope with draconic punishments – a paradox because the most severe punishment could amount to manslaughter if not murder.
- pyrale 3y agoThere are differences, however, between research using poor methodology that end up being worthless because it can't be replicated, and research where some data points were deleted and others were created in order to show a better result. The work here isn't fraudulent because of a mistake or because the author has shaky statistics knowledge, but because it was fabricated.
- UncleMeat 3y agoPsychology is also one of the only fields funding replication. My experience in CS is that the replicability of experimental results is embarrassingly bad, but this isn't making headlines in the same way so people don't consider it to be a problem. This is data fraud in psych. Data fraud has also happened in plenty of other fields. When it occurs in these fields it is seen as a one-off. When it occurs in psych, it is because the entire field is useless garbage. That's not the lesson to take away here.
- stcroixx 3y agoWell, CS doesn't involve pharmaceuticals so the harm caused by fraud isn't as serious.
- UncleMeat 3y agoWe could maybe make the conversation about the social penetration of ideas coming out of various disciplines and how invalid research can harm policy or medical practice, but that'd be a completely different discussion than "psychology is utter garbage as a field."
- manicennui 3y agoBecause no one writes software that can lead to death.
- friendly_wizard 3y agoTell that to air traffic controllers
- philwelch 3y agoTherac-25.
- ajuc 3y agoYou can easily kill people with software. It has happened.
- 3y ago
- hx8 3y agoThis is such an exaggerated claim. * Plenty of studies inherently cannot survive 20 years. Send out a survey and ask people "on a scale of 1 to 10 how happy are you". These results will not match the results in 21 years simply because the population you are studying is different. This example is simplified, but explains why many results change over time. * Plenty of studies have survived 20 years, and those that study psychology are really clear about which knowledge is foundational and which is on more questionable ground. The Big Five personalty traits is about 40 years old and has been found to be consistent across cultures. A ton of psychological research around psychological responses has survived the test of time. * Psychologist often relay on more questionable data than other scientists because it's much cheaper and easier for psychologist to collect questionable data than other sciences. This allows for a wider exploration and makes research much more accessible. Anyone reading a paper with this type of data will be naturally skeptical. I am specifically referring to data collected from self-reported questionnaires, often collecting a sample that generalizes poorly over interesting populations.
- tgv 3y agoAs I've written more than once here: it's are harder to experiment upon than physics. Society/individual behavior is also way more complex than e.g. rheology or organic chemistry, and doesn't submit itself to lab research. But that's not the only problem. The problem is that people try it anyway. They have some hypothesis within a theoretical framework which is embedded in other frameworks, none of which is proven. How could they be? But, they set up an experiment anyway, and (usually after a couple of attempts) they find something that can reject H0. Then they publish an article stating that their theoretical framework is a fact. There is so much wrong in this process, even ignoring the manipulation of the stimuli and conditions, and the statistical procedures, yet the theory has a good chance to make school and become the subject of research in dozens of psych departments, each contributing articles to the literature about it. After a while, an essentially flawed theory enters the handbooks. Then, if you want to make a name for yourself as a researcher, and you need publications, you can simply attack older theories. They'll crumble like cookies, you get your publications, and the cycle restarts. And as critical as I am of psychology, other social sciences have even lower empirical standards. Educational sciences, sociology, linguistics all have very little to show for a century of research. And that's ignoring disciplines like political sciences or history.
- BeetleB 3y ago> and responsible for a large chunk of the science replication crisis. Former semiconductor researcher here. It is quite common for researchers not to believe published papers - even in prestigious journals from well known researchers. No one bothers replicating, and they don't put enough information in the paper to replicate (competition - they don't want others to get the secret sauce). This was true for both computational and experimental papers. Oh, and I did know one person personally who falsified data (and was caught). He just transferred to another prestigious school and got his PhD there instead. It was quite demoralizing and helped me decide not to pursue academia.
- rcxdude 3y agoI think a lot of the communication around science publishing has overemphasized the power of peer review. More people should realise it's the table stakes for science, not the conclusion.
- somsak2 3y agoAny sources for these claims?