8 ms·
Daryl Bem and the Replication Crisis
- eth0up 9y agoPlease also see and consider: https://en.wikipedia.org/wiki/Lawrence_LeShan https://en.wikipedia.org/wiki/Lawrence_LeShan A particularly excellent work by Dr LeShan is: The Medium, the Mystic, and the Physicist: Toward a General Theory of the Paranormal (1974) - The appendix is excellent, if the unabridged version.
- sedeki 9y agoAnyone care to give a tl;dr version?
- frgtpsswrdlame 9y agoIt's about the replication crisis and it's well written so I'd consider reading the whole thing when/if you have the time. Here's a few of the most important parts. Having served for a time as an associate editor of JPSP, Bem knew his methods would be up to snuff. With about 100 subjects in each experiment, his sample sizes were large. He’d used only the most conventional statistical analyses. He’d double- and triple-checked to make sure there were no glitches in the randomization of his stimuli. Even with all that extra care, Bem would not have dared to send in such a controversial finding had he not been able to replicate the results in his lab, and replicate them again, and then replicate them five more times. His finished paper lists nine separate ministudies of ESP. Eight of those returned the same effect. ... But for most observers, at least the mainstream ones, the paper posed a very difficult dilemma. It was both methodologically sound and logically insane. Daryl Bem had seemed to prove that time can flow in two directions—that ESP is real. If you bought into those results, you’d be admitting that much of what you understood about the universe was wrong. If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk. ... These dodgy methods were clearly rife in academic science. A 2011 survey of more than 2,000 university psychologists had found that more than half of those researchers admitted using them. But how badly could they really screw things up? By running 15,000 simulations, Simmons, Nelson, and Simonsohn showed that a researcher could almost double her false-positive rate (often treated as if it were 5 percent) with just a single, seemingly innocuous manipulation. And if a researcher combined several questionable (but common) research practices—fiddling with the sample size and choosing among dependent variables after the fact, for instance—the false-positive rate might soar to more than 60 percent. ... That’s more or less Bem’s position. “The critics said that I put psychologists in an uncomfortable position and that they’d have to revise their views of the physical world or their views on research practice,” he told me. “I think both are true. I still believe in psi, but I also think that methods in the field need to be cleaned up.”
- toyg 9y ago> If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk. That's... not really difficult to admit.
- frgtpsswrdlame 9y agoIt is if you've dedicated your life to academic psychology.
- junkculture 9y agoWell written? I stopped reading at "mansplaining guinea pigs." Anyone who resorts to such tropes can't be taken seriously.
- throwaway91111 9y agoBasically, a gentleman appears to have shown high quality scientific studies showing some effect implying the existence of ESP. So either ESP is a real thing, or our metric for scientific reliability is not effective at weeding out fraudulent claims. However.... it's really quite an enjoyable article to read if you have the time.
- lightedman 9y agoBasically the guy has very sound and rigorous methodology, and has achieved higher-than-random-chance results (to the tune of a mere 53% correct guessing based on variables in one experiment involving hidden erotic pictures.)
- sammydavis 9y agoI suppose claiming ESP is real is like claiming cold fusion is real. You better have some damn good proof because there have been a lot of shoddy claims that didn't hold up in practice. So has this been replicated by outsiders based on his paper? I personally am extremely skeptical. If we did discover something like ESP is possible, it would be the biggest discovery so far this century. It would open up possible consideration of religion, and all those people who claim to astral travel, could any of that stuff be real? It would be cool and scary. But naw, it's got to be bullshit :-)
- tptacek 9y agoNo, he has a methodology that is facially sound but fundamentally unserious: his team explores a gigantic space of possible experiments that would demonstrate an effect and reports only on those that support his claim. As a result, his results don't reliably replicate. He's using heavily biased statistics, just on a higher order.
- m00s3 9y agoHas anyone called James Randi yet?
- eth0up 9y agoBefore invoking that guy, have a glance at this first: http://www.human.cornell.edu/hd/upload/CIWS-module_ESP.pdf http://www.human.cornell.edu/hd/upload/CIWS-module_ESP.pdf The above is something I quickly found - one of many examples. It doesn't have any hallmarks of quackery that stand out to me. I've always wished Mr Randi would graduate to confronting more formidable scientists in the field, rather than picking on spoon benders[1] and other lunatics. He is a powerful thinker and has some great work behind him, but he's missed some important opportunities to challenge himself. 1. Uri Geller
- smt88 9y agoThe title is clickbait. Bem didn't prove ESP is real. He (unknowingly) manipulated his data.
- jfoutz 9y agoOr perhaps Bem believed there were problems with methodology and used ESP to demonstrate the problem. The title seems warranted given the nature of the attack on methodology.
- smt88 9y agoPerhaps, but not in this case. He is a true believer.
- asn0 9y agoInteresting, can you elaborate or do you have a reference?
- smt88 9y agoThis is an entertaining explanation: http://www.npr.org/sections/money/2016/01/15/463237871/episode-677-the-experiment-experiment http://www.npr.org/sections/money/2016/01/15/463237871/episo...
- abandonliberty 9y agoDaryl did the science perfectly at a standard beyond what most studies do. Well beyond the accepted rigor and burden of proof, he demonstrated that ESP is real. Therefore, the accepted burden of proof is broken. While you accuse the title of clickbaiting, you seem to agree entirely with it.
- tptacek 9y agoThe article doesn't believe he did either: in fact, the whole premise of the story is the reality of ESP is a nonsense result. This is probably a case where the editors rewrote the title; it's mostly a story about how lack of rigor is sabotaging the social sciences.
- notliketherest 9y agoStopped reading at the line: "A few students—all of them white guys, Wu remembers—would hang around to ask about the research and to probe for flaws in its design. Wu still didn’t believe in ESP, but she found herself defending the experiments to these mansplaining guinea pigs." Blatant sexism and rasicm against white males might be okay to idiot millennials but to me it's not okay. This is what a constant focus on "white injustice against minoritys everywhere!!" has done to our society. Casual sexism and racism is totally okay, as long as it's directed against the perpetrators of sexism and racism? Slate / Huffington Post / other outlets are full of this garbage.
- bbctol 9y agoWhat... isn't okay about this? It's a description of the interviewee, you don't have to agree with it.
- yorwba 9y agoIt is completely out of place in this article and blatantly stereotyping. Just because some situation happens to fit a trendy topic like "mansplaining white males" doesn't mean it has to be explicitly pointed out.
- mikeash 9y agoIt's not stereotyping to mention that a particular group of people actually predominantly or entirely consists of a certain race or sex.
- yorwba 9y agoI'm not a native English speaker, so I had to look it up, but I think I'm using the word correctly: https://en.wiktionary.org/wiki/stereotype#English https://en.wiktionary.org/wiki/stereotype#English - A person who is regarded as embodying or conforming to a set image or type.
- 9y ago
- xKingfisher 9y agoThe Slate Star Codex has a good article on the same topic. http://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...
- tome 9y agoThis is a great article. Why is it flagged?
- yorwba 9y agoClickbaity headline, takes too long to take up speed ... Also, why can't I vouch for it like with flagged comments?
- tome 9y agoThis is a interesting site on a tangentially related issue: http://www.joseduarte.com/ http://www.joseduarte.com/
- erikj 9y agoNot "science" but "a science", psychology, which is notoriously problematic with regards to reproducibility.
- gremlinsinc 9y agoPerhaps ESP is actually proof that we live in a simulation/hologram. Seeing the future, is somehow tapping into the simulation and being able to on a subconscious level guess at what comes next.
- captn3m0 9y agobranch prediction, you mean.
- Kinnard 9y agoWhy was this flagged?
- dang 9y agoUsers flagged it. Hard to say why, but it might be because they don't think HN is able to sustain a high-quality discussion about this topic.
- tptacek 9y agoUsers in this case were probably wrong; this is a straightforward article about the replication crisis, not a story about ESP. Unfortunately, the title is so bad that the thread is swamped with people who'd rather discuss ESP. A better title, synthesized from the last graf of the introduction, might be "Daryl Bem, ESP, and the Replication Crisis".
- dang 9y agoI agree completely but IMO the topic is still too triggery for HN to discuss it without losing it, and a title change may not be enough. We can send the submitter a repost invite, though, with a different title, and see if starting from scratch would help.
- Kinnard 9y agoWould it be a good idea for people to have to state a short rationale for flagging? Just clicking a flag button is too cheap . . . I've even done it accidentally a number of times.
- DanBC 9y agoIt's a toggle, so you can unflag submissions.
- frgtpsswrdlame 9y agoWhen this topic flipped to [flagged] yesterday the top comment thread was the one about racism/sexism. That may have influenced it.