4 ms·
When I see a headline like this my instinct is to jump to "overfitting". In a study like this they use P=0.05 as their cut off for significance. One in 20 such
by gamedori 7y ago
When I see a headline like this my instinct is to jump to "overfitting".
In a study like this they use P=0.05 as their cut off for significance. One in 20 such studies would be expected to give meaningless results. But they had the choice of cohort (all kids or boys and girls separately), so 1 in 10. They measured urine concentrations of fluorine at three time points during pregnancy, which makes for 7 different potential combinations of input variables, for a total of 14 different potential analyses, where 1 in 20 is expected to be significant. Of course, they didn't preregister, so who knows whether their analysis changed on the fly..
- dtech 7y agoGood points. It's even more than 1 in 20 when you realize that studies does no get published with p > 0.05, so in the "published studies" sample this effect should be even larger.