4 ms·

Anyone care to give a tl;dr version?

by sedeki 9y ago

frgtpsswrdlame 9y ago
It's about the replication crisis and it's well written so I'd consider reading the whole thing when/if you have the time. Here's a few of the most important parts. Having served for a time as an associate editor of JPSP, Bem knew his methods would be up to snuff. With about 100 subjects in each experiment, his sample sizes were large. He’d used only the most conventional statistical analyses. He’d double- and triple-checked to make sure there were no glitches in the randomization of his stimuli. Even with all that extra care, Bem would not have dared to send in such a controversial finding had he not been able to replicate the results in his lab, and replicate them again, and then replicate them five more times. His finished paper lists nine separate ministudies of ESP. Eight of those returned the same effect. ... But for most observers, at least the mainstream ones, the paper posed a very difficult dilemma. It was both methodologically sound and logically insane. Daryl Bem had seemed to prove that time can flow in two directions—that ESP is real. If you bought into those results, you’d be admitting that much of what you understood about the universe was wrong. If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk. ... These dodgy methods were clearly rife in academic science. A 2011 survey of more than 2,000 university psychologists had found that more than half of those researchers admitted using them. But how badly could they really screw things up? By running 15,000 simulations, Simmons, Nelson, and Simonsohn showed that a researcher could almost double her false-positive rate (often treated as if it were 5 percent) with just a single, seemingly innocuous manipulation. And if a researcher combined several questionable (but common) research practices—fiddling with the sample size and choosing among dependent variables after the fact, for instance—the false-positive rate might soar to more than 60 percent. ... That’s more or less Bem’s position. “The critics said that I put psychologists in an uncomfortable position and that they’d have to revise their views of the physical world or their views on research practice,” he told me. “I think both are true. I still believe in psi, but I also think that methods in the field need to be cleaned up.”
- toyg 9y ago
  > If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk. That's... not really difficult to admit.
  - frgtpsswrdlame 9y ago
    It is if you've dedicated your life to academic psychology.
- junkculture 9y ago
  Well written? I stopped reading at "mansplaining guinea pigs." Anyone who resorts to such tropes can't be taken seriously.
throwaway91111 9y ago
Basically, a gentleman appears to have shown high quality scientific studies showing some effect implying the existence of ESP. So either ESP is a real thing, or our metric for scientific reliability is not effective at weeding out fraudulent claims. However.... it's really quite an enjoyable article to read if you have the time.
lightedman 9y ago
Basically the guy has very sound and rigorous methodology, and has achieved higher-than-random-chance results (to the tune of a mere 53% correct guessing based on variables in one experiment involving hidden erotic pictures.)
- sammydavis 9y ago
  I suppose claiming ESP is real is like claiming cold fusion is real. You better have some damn good proof because there have been a lot of shoddy claims that didn't hold up in practice. So has this been replicated by outsiders based on his paper? I personally am extremely skeptical. If we did discover something like ESP is possible, it would be the biggest discovery so far this century. It would open up possible consideration of religion, and all those people who claim to astral travel, could any of that stuff be real? It would be cool and scary. But naw, it's got to be bullshit :-)
- tptacek 9y ago
  No, he has a methodology that is facially sound but fundamentally unserious: his team explores a gigantic space of possible experiments that would demonstrate an effect and reports only on those that support his claim. As a result, his results don't reliably replicate. He's using heavily biased statistics, just on a higher order.

Anyone care to give a tl;dr version?

by sedeki 9y ago

Anyone care to give a tl;dr version?

It's about the replication crisis and it's well written so I'd consider reading the whole thing when/if you have the time. Here's a few of the most important parts. Having served for a time as an associate editor of JPSP, Bem knew his methods would be up to snuff. With about 100 subjects in each experiment, his sample sizes were large. He’d used only the most conventional statistical analyses. He’d double- and triple-checked to make sure there were no glitches in the randomization of his stimuli. Even with all that extra care, Bem would not have dared to send in such a controversial finding had he not been able to replicate the results in his lab, and replicate them again, and then replicate them five more times. His finished paper lists nine separate ministudies of ESP. Eight of those returned the same effect. ... But for most observers, at least the mainstream ones, the paper posed a very difficult dilemma. It was both methodologically sound and logically insane. Daryl Bem had seemed to prove that time can flow in two directions—that ESP is real. If you bought into those results, you’d be admitting that much of what you understood about the universe was wrong. If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk. ... These dodgy methods were clearly rife in academic science. A 2011 survey of more than 2,000 university psychologists had found that more than half of those researchers admitted using them. But how badly could they really screw things up? By running 15,000 simulations, Simmons, Nelson, and Simonsohn showed that a researcher could almost double her false-positive rate (often treated as if it were 5 percent) with just a single, seemingly innocuous manipulation. And if a researcher combined several questionable (but common) research practices—fiddling with the sample size and choosing among dependent variables after the fact, for instance—the false-positive rate might soar to more than 60 percent. ... That’s more or less Bem’s position. “The critics said that I put psychologists in an uncomfortable position and that they’d have to revise their views of the physical world or their views on research practice,” he told me. “I think both are true. I still believe in psi, but I also think that methods in the field need to be cleaned up.”

toyg 9y ago
> If you rejected them, you’d be admitting something almost as momentous: that the standard methods of psychology cannot be trusted, and that much of what gets published in the field—and thus, much of what we think we understand about the mind—could be total bunk. That's... not really difficult to admit.
- frgtpsswrdlame 9y ago
  It is if you've dedicated your life to academic psychology.
junkculture 9y ago
Well written? I stopped reading at "mansplaining guinea pigs." Anyone who resorts to such tropes can't be taken seriously.

throwaway91111 9y ago

Basically, a gentleman appears to have shown high quality scientific studies showing some effect implying the existence of ESP. So either ESP is a real thing, or our metric for scientific reliability is not effective at weeding out fraudulent claims. However.... it's really quite an enjoyable article to read if you have the time.

lightedman 9y ago

Basically the guy has very sound and rigorous methodology, and has achieved higher-than-random-chance results (to the tune of a mere 53% correct guessing based on variables in one experiment involving hidden erotic pictures.)

sammydavis 9y ago
I suppose claiming ESP is real is like claiming cold fusion is real. You better have some damn good proof because there have been a lot of shoddy claims that didn't hold up in practice. So has this been replicated by outsiders based on his paper? I personally am extremely skeptical. If we did discover something like ESP is possible, it would be the biggest discovery so far this century. It would open up possible consideration of religion, and all those people who claim to astral travel, could any of that stuff be real? It would be cool and scary. But naw, it's got to be bullshit :-)
tptacek 9y ago
No, he has a methodology that is facially sound but fundamentally unserious: his team explores a gigantic space of possible experiments that would demonstrate an effect and reports only on those that support his claim. As a result, his results don't reliably replicate. He's using heavily biased statistics, just on a higher order.