4 ms·
If you can get past the linkbait title and some questionable word choices on the part of the author, this is actually an excellent article. The article is not
by nkurz 9y ago
If you can get past the linkbait title and some questionable word choices on the part of the author, this is actually an excellent article. The article is not trying to convince you that ESP is real, at least not in the sense of in-the-world reality. Rather, it's an article about the standards of scientific proof, showing that there are cases where standard statistical practices can "prove" apparently absurd results.
The issue is that in general, it's hard to tell the difference between "absurd" and "unexpected". In theory, the scientific method is about designing experiments that produce replicable results that cannot be explained by current theory, and then refining (or replacing) the theory until the can explain the new results, while still making correct predictions about all cases covered by the old theory.
But what do you do when results are obtained that violate the foundations of science, such as the time order of cause and effect? Naturally, one should start by being skeptical of the experiment. Was the data accidentally recorded wrong? Was the data inappropriately filtered before being analyzed? Is the experimenter lying about the data that was obtained?
Usually, thinking about these issues yields some apparent cause of error that would explain the unexpected results without violating one's basic beliefs. Unfortunately, an apparent reason for disbelief can often be found even if the results truly are impeccable. Often, the result is that doubters continue their disbelief, and the believers continue believing, until one faction or the other retires from the field and the other belief becomes "consensus".
What Bem has done is to design an experiment that surpasses the statistical standards of many fields, yet "proves" a result that on the surface seems impossible. Most of the usual scientific errors have been avoided, and his methodology and analysis are better than most. One possibility is that our current conceptions of causality are wrong: we think that for A to cause B, A must happen before B, but in fact this is not a requirement.
Another possibility is that something is grossly wrong with our current interpretation of scientific results, and many longstanding theories which are considered "scientifically" proven may in fact be mistakes, or at least have no more "proof" than Bem has managed to show for ESP. For most scientists, both of these answers are problematic, yet it would seem that at least one of them must be true. One twist is that some believe that Bem's goal is not actually to prove that ESP is correct, rather to show that the foundations of science are faulty: http://andrewgelman.com/2013/08/25/a-new-bem-theory/ http://andrewgelman.com/2013/08/25/a-new-bem-theory/.
- ianai 9y agoYour comment exceeds the article. Well done. This is why I read HN
- brylie 9y agoAgreed. I often go straight to the comments.
- wuch 9y agoThere is a video from a debate with Daryl Bem about those results. It is worth a look, because he admits to using various methods that increasing the degrees of freedom, and not taking those into account during statistical analysis. Essentially, he honestly admits to p-hacking, but doesn't seem to recognize anything wrong in that. On those grounds, I think, it is little misleading to suggest that those studies surpasses the statistical standards of many fields. The statistical tests are completely misapplied, but you can't tell just by looking at publications.
- nkurz 9y agoDo you have a link to the debate you mean? I'd be interested to see it, but when I searched I found several. One theory is that while the initial experiments were p-hacked to some extent, the replications (by definition) were not. Some trustworthy sources argue that the problem is that the replications actually weren't, but I haven't looked into the specifics closely enough to know whether that's the case. That said, I thought Bem came across quite well in this interview: http://skeptiko.com/daryl-bem-responds-to-parapsychology-debunkers http://skeptiko.com/daryl-bem-responds-to-parapsychology-deb.... I think, it is little misleading to suggest that those studies surpasses the statistical standards of many fields I didn't mean to imply that the statistical standards were good ones, rather that the standards in some fields are abysmally low. For example, there's a high profile case going on with the Cornell Food Lab, where the extremely prolific lead researcher seemed not just blissfully unaware but proud of the unseen pitfalls around him: https://web.archive.org/web/20170312041524/http:/www.brianwansink.com/phd-advice/the-grad-student-who-never-said-no https://web.archive.org/web/20170312041524/http:/www.brianwa.... The statistical tests are completely misapplied, but you can't tell just by looking at publications. I think that's true, but (in my opinion) the problem is finding papers outside the very hard sciences where they _aren't_ completely misapplied. For example, I feel there is an insurmountable issue with applying formal statistics to any sort of meta-studies, where the data is at best a convenience sample, and (almost?) all conclusions are conditional on the unknown biases of the sample. This of course doesn't mean that all (or even most) meta-analyses produce the wrong answer, but I think it does mean that they should be treated as rhetorical rather than logical arguments.