3 ms·
This counter-paper reads to me like spitefulness couched in scientific terms. Wagenmakers is, of course, on the very comfortable ground of defending the status
by utunga 16y ago
This counter-paper reads to me like spitefulness couched in scientific terms. Wagenmakers is, of course, on the very comfortable ground of defending the status quo from someone who is in all probability wrong.
But to be fair, we should take things on their merits. Contrasting the two papers, Bem's paper sticks very closely to the standard form and 'rules of engagement' (if you will) of published academic research. This counter paper on the other hand, has only the surface appearance of proper scientific method. For example, in making one if its main points it refers to a hypothetical casino where Bem could've made an 'infinite' amount of money, and to the one million dollar Randi prize, neither of which reach the standard of proper experimentalist scientific evidence. It does this by way of justification to one of their main points which is that because 'extraordinary claims require extraordinary evidence' we should be able to set the prior expectation of H1 - in their words 'for illustrative purposes' - to .00000000000000000001, which they then go on to demonstrate, makes the results non-significant.
But where does the 0.00000000000000000001 come from? It could, just as easily, be twice or half that figure. Thus not falsifiable and therefore not justifiable as an extra bit of arithmetic that Bem's paper must qualify.
To put this in terms this audience will understand, that's kind of like saying well no, because you are using Java and 'everybody knows' Java is slow I think we should multiply your benchmark figures by, oh, let's say, one half. And then we see that java throughput is quite poor, as expected. In fact, we should multiply all java benchmarks by some number like a half but I'm not going to be specific about it because actually its pretty much just an arbitrary number I made up. So, reading past the reference to Bayes and some nice formulas that's just arithmetic in my book.
Note, I'm not saying that there are no flaws in the Bem paper. Everybody can see that it's very likely they'll be something wrong with it (and the desk drawer problem to which the above paper also makes extensive evidence is a likely though not conclusive contender) but I think its only reasonable to hold up the criticism's to the same standard as what they are criticizing. Perhaps that way you'll be more likely to find the actual truth of the matter.
- jfager 16y agoContrasting the two papers, Bem's paper sticks very closely to the standard form and 'rules of engagement' (if you will) of published academic research. That's kind of the problem: published academic research is broken in a lot of ways. If you haven't read it yet, the New Yorker's recent article on the decline effect touches on several of the reasons why: http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer?printable=true¤tPage=1 http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_... And some prominent medical journals started fighting against publication bias a few years ago: http://www.smh.com.au/articles/2004/09/09/1094530773888.html http://www.smh.com.au/articles/2004/09/09/1094530773888.html I think you're holding this response to a much higher burden of proof than it needs to meet to be a proper refutation of Bem's claims. You're right that the response doesn't appear to use the "proper scientific method". But that's because it doesn't use it at all, and it doesn't need to. There's no hypothesis to test and no experiment to run in order to point out flaws in a paper that does claim to be the result of the scientific method.