3 ms·
Theory-testing in psychology and physics: a methodological paradox (1967) [pdf]
- gwern 12y agoExcerpts of Meehl 1967: https://plus.google.com/103530621949492999968/posts/4sSbzwnPkwd https://plus.google.com/103530621949492999968/posts/4sSbzwnP...
- capnrefsmmat 12y agoMy favorite quote from this has to be Meanwhile our eager-beaver researcher, undismayed by logic-of-science considerations and relying blissfully on the “exactitude” of modern statistical hypothesis-testing, has produced a long publication list and been promoted to a full professorship. In terms of his contribution to the enduring body of psychological knowledge, he has done hardly anything. His true position is that of a potent-but-sterile intellectual rake, who leaves in his merry path a long train of ravished maidens but no viable scientific offspring. The problem boils down to the insistence on statistical significance testing. Whenever you test an intervention (priming experiments or whatever a psychologist might look for), it's reasonable to expect there's some effect -- even if it's negligibly small or practically useless. But significance testing is about proving that the effect is not zero, so you will always get a significant result as long as you collect enough data. This is why I am so frustrated when I see a psychology paper claiming something like "Women wear pink shirts while more fertile" and the paper says the effect is statistically significant, but doesn't say how big it is. Maybe it's a completely unimportant effect. This is why someone (I forget who) suggested that null hypothesis significance testing (NHST) should be renamed Statistical Hypothesis Inference Testing, for the aptness of the resulting acronym. There's a whole legion of other common problems (enough to write a book on [see my profile]), but in psychology especially this is a big one.
- misnome 12y agoProbably also worth a read is Feynman's Caltech commencement address, which talks a little about Psychology (or the state thereof in the 70's - I don't know enough to judge whether things have really changed, and doubt the media provides a balanced perspective) http://neurotheory.columbia.edu/~ken/cargo_cult.html http://neurotheory.columbia.edu/~ken/cargo_cult.html