3 ms·
The scenario presented by the author seems more explicit and nefarious than the typical "p-hunting" that I've seen in academia. Though I've seen the situation h
by jkyle 11y ago
The scenario presented by the author seems more explicit and nefarious than the typical "p-hunting" that I've seen in academia. Though I've seen the situation he presents, successively trying different models or parameters until you find the expected effect, it's pretty obviously unethical.
What I've seen much, much more of goes something like this. Say your lab does research that requires a very large quantity of data that's very expensive to acquire. In academia, labor is relatively cheap and funding can be difficult to come by. So a dataset that consumed potentially 100's of thousands of funding dollars represents a significant resource investment.
So what do you do when the study is done? You try to figure out every way possible to reuse that data. And the most common way of doing that is running it using previous models looking for effects...enough people keep looking long enough and, voila!, you find a significance.
In the end, it's the same bias. But the motivation is different and when I've pointed out this is "p-hunting" most just give a blank stare.