3 ms·
It's about the replication crisis and it's well written so I'd consider reading the whole thing when/if you have the time. Here's a few of the most important pa
by frgtpsswrdlame 9y ago
It's about the replication crisis and it's well written so I'd consider reading the whole thing when/if you have the time. Here's a few of the most important parts.
Having served for a time as an associate editor of JPSP, Bem knew his methods would be up to snuff. With about 100 subjects in each experiment, his sample sizes were large. He’d used only the most conventional statistical analyses. He’d double- and triple-checked to make sure there were no glitches in the randomization of his stimuli.
Even with all that extra care, Bem would not have dared to send in such a controversial finding had he not been able to replicate the results in his lab, and replicate them again, and then replicate them five more times. His finished paper lists nine separate ministudies of ESP. Eight of those returned the same effect.
...
But for most observers, at least the mainstream ones, the paper posed a very difficult dilemma. It was both methodologically sound and logically insane. Daryl Bem had seemed to prove that time can flow in two directions—that ESP is real. If you bought into those results, you’d be admitting that much of what you understood about the universe was wrong. If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk.
...
These dodgy methods were clearly rife in academic science. A 2011 survey of more than 2,000 university psychologists had found that more than half of those researchers admitted using them. But how badly could they really screw things up? By running 15,000 simulations, Simmons, Nelson, and Simonsohn showed that a researcher could almost double her false-positive rate (often treated as if it were 5 percent) with just a single, seemingly innocuous manipulation. And if a researcher combined several questionable (but common) research practices—fiddling with the sample size and choosing among dependent variables after the fact, for instance—the false-positive rate might soar to more than 60 percent.
...
That’s more or less Bem’s position. “The critics said that I put psychologists in an uncomfortable position and that they’d have to revise their views of the physical world or their views on research practice,” he told me. “I think both are true. I still believe in psi, but I also think that methods in the field need to be cleaned up.”
- toyg 9y ago> If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk. That's... not really difficult to admit.
- frgtpsswrdlame 9y agoIt is if you've dedicated your life to academic psychology.
- junkculture 9y agoWell written? I stopped reading at "mansplaining guinea pigs." Anyone who resorts to such tropes can't be taken seriously.