The Dutch skeptical researchers whose work Gelman comments on here are exceptionally willing to share author manuscripts for free downloads on the World Wide We

The Dutch skeptical researchers whose work Gelman comments on here are exceptionally willing to share author manuscripts for free downloads on the World Wide Web. However, the particular paper that Gelman comments on here the most, a paper still in press (not yet formally published) has turned into a dead link. I get a 404 message when I follow the link from Gelman's blog post. Searching around on the lead author's (Eric-Jan Wagenmakers's) website, I see the notice "Wagenmakers, E.-J., Wetzels, R., Borsboom, D., Kievit, R., & van der Maas, H. L. J. (in press). A skeptical eye on psi. In May, E., & Marwaha, S. (Eds.), Extrasensory Perception: Support, Skepticism, and Science. ABC-CLIO. NB. I was requested to remove this preprint temporarily, until the publisher has given explicit consent to publish it here. If you seek a preprint please send me an Email." And I'll note for the record that many of the papers by Eric-Jan Wagenmakers and his co-authors continue to be available on his website,[1] whether published yet or not, but please be sure that he is allowed to carry out his agreements with publishers so that those papers stay available. The work of that group of authors is very important and will do much to improve the science of psychology. Gelman also comments favorably in his blog post about the work of Uri Simonsohn and his colleagues, who have devoted much thought to the issue of "p-hacking." Simonsohn is a professor of psychology with a better than average understanding of statistics. He and his colleagues are concerned about making scientific papers more reliable. Many of the interesting issues brought up by the comments on the article kindly submitted here become much more clear after reading Simonsohn's various articles[2] about p values and what they mean, and other aspects of interpreting published scientific research. Simonsohn provides an abstract (which links to a full, free download of a funny, thought-provoking paper)[3] with a "twenty-one word solution" to some of the practices most likely to make psychology research papers unreliable. He has a whole site devoted to avoiding "p-hacking,"[4] an all too common practice in science that can be detected by statistical tests. You can use the p-curve software on that site for your own investigations into p values found in published research. He also has a paper on evaluating replication results[5] (an issue we discuss from time to time here on Hacker News) with more specific tips on that issue. "Abstract: "When does a replication attempt fail? The most common standard is: when it obtains p>.05. I begin here by evaluating this standard in the context of three published replication attempts, involving investigations of the embodiment of morality, the endowment effect, and weather effects on life satisfaction, concluding the standard has unacceptable problems. I then describe similarly unacceptable problems associated with standards that rely on effect-size comparisons between original and replication results. Finally, I propose a new standard: Replication attempts fail when their results indicate that the effect, if it exists at all, is too small to have been detected by the original study. This new standard (1) circumvents the problems associated with existing standards, (2) arrives at intuitively compelling interpretations of existing replication results, and (3) suggests a simple sample size requirement for replication attempts: 2.5 times the original sample." [1] http://ejwagenmakers.com/papers.html http://ejwagenmakers.com/papers.html [2] http://opim.wharton.upenn.edu/~uws/ http://opim.wharton.upenn.edu/~uws/ [3] http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2160588 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2160588 [4] http://www.p-curve.com/ http://www.p-curve.com/ [5] http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2259879 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2259879