3 ms·
What’s frustrating about this situation is how obviously wrong the experiments were as described in the book, and yet how unshakably the findings were defended
by camjohnson26 6y ago
What’s frustrating about this situation is how obviously wrong the experiments were as described in the book, and yet how unshakably the findings were defended by Kahneman and others. I read the book before the crisis and stopped reading after the priming chapter, it was clearly fantasy, especially the experiment about reading words related to old age and then walking more slowly down a hallway. Now that the crisis is public knowledge it’s hard to believe these findings were ever accepted by the mainstream, but they were.
You just have to think for yourself and weigh the evidence, be humble and open to other perspectives. Mindlessly following perceived experts won’t save you.
- 0xddd 6y agoIt seems to me that people put too much faith in statistics despite not having a fundamental understanding of the methods. I'm sure in plenty of cases these frivolous results get published due to academic fraud and the pressure of publish or perish. But in other cases, it seems like researchers really think "p<.05" translates to "your alternative hypothesis is irrefutably true" and will believe their flimsy conclusions because, as far as they're concerned, numbers don't lie. (If I remember correctly, when Brian Wansink was exposed for p-hacking, it was partly because he wrote a blog post openly describing his sketchy research methods, where he seemed totally unaware that he was churning out illegitimate results.)
- anitil 6y agoI've been trying to find this blog post, any chance you can remember more of it to help my googling?
- 0xddd 6y agoI managed to dig it up: https://web.archive.org/web/20170312041524/http:/www.brianwansink.com/phd-advice/the-grad-student-who-never-said-no https://web.archive.org/web/20170312041524/http:/www.brianwa.... It really is incredible to read Wansink's clueless non-rebuttals to the comments that point out his academic fraud.
- lowdose 6y agoStaggering replies from him on the negative comments on his methods and thinking, what a character. What a gem to read, absurt funny title "The Grad Student Who Never Said "No". And the fact that at least 54 times people took a snapshot of the webpage. We should print this on toilet paper and sell it to universities.
- anitil 6y agoThankyou!
- wyattpeak 6y agoOn what, at the time, did you base the assumption that people walking slowly after reading words related to old age was clearly fantasy? It doesn't sound likely to me, but neither do a bunch of other cognitive results (the gorilla running through the basketball court comes to mind). I'm all for questioning and rerunning tests, but dismissing them because they seem clearly fanciful without data just brings us back to pre-science. Science which is only allowed to discover plausible-sounding things isn't science at all.
- aabhay 6y agoThe burden of proof is on science, not me. I’m not going to follow every conjecture and hypothesis floated by a scientist. If something sounds fanciful and violates my common sense, I need a robust experiment. Accepting results without that just brings us back to pre-science.
- jbay808 6y agoSounds like you're properly applying priors! There's an interesting chapter in Jaynes' Probability Theory: the Logic of Science where he talks about how much evidence it would take to convince him that ESP is real. At the time that book was written, there were experiments supporting it that failed to replicate. Basically you're totally allowed to read a scientific study, say "okay, that's an interesting result", and then still not believe the conclusion if you have enough reason to doubt the premise a-priori.