2 ms·
Isn't the solution just to put funding into reproducing results? It's fine to have a wide-filter for exploratory research, but a big problem is that it can be
by aidenn0 13y ago
Isn't the solution just to put funding into reproducing results? It's fine to have a wide-filter for exploratory research, but a big problem is that it can be decades before anybody goes back to reproduce.
Let's assume that 90% of results are false positives. Using a higher power experiment with even the same standard of p=0.05 would result in rejecting most of those, while hopefully keeping the majority of the true positives (due to the higher power of the new test). This would result in going from 90% false positives to less than half false positives, a considerable improvement.
It just seems that nobody ever advanced their career by trying to reproduce even a landmark result in their field.
- niels_olson 13y agoThe issue in day to day science is that people will work their numbers to get p=0.05. They'll have a hypothesis, that, say, lactating adenomas confer a protective effect against DCIS. And they'll pull, say, all lactating adenoma cases from 2001 to 2013. If that doesn't work, they might actually try dropping the 2001 data and just using the 2002 to 2013 data if it gets them to p=0.05. The result is more brittle (we often test findings by asking "How would the p value change if we added one negative case?") but the alternative (pun intended) wouldn't otherwise be published. This search for p=0.05 also leads to a lot of hair-splitting studies: take two diagnoses, say, usual and atypical ductal hyperplasia. Now, if you can find some constellation of parameters that define a middle category, say, "borderline ductal hyperplasia", you have a wide-open field to all sorts of p=0.05's, even if there's no change in treatment or outcome. You can say "cases previously characterized as UDH with a <parameter x> greater than <x> are 67% more likely to have <parameter y> (p=0.002)" because you lumped together a bunch of stuff that people already mostly agreed on anyway.