3 ms·
If you follow their methods, specifically [63], they are modelling the proportion of null hypotheses in their tests. Performing FDR by group is a reasonable ap
by Real_S 9y ago
If you follow their methods, specifically [63], they are modelling the proportion of null hypotheses in their tests. Performing FDR by group is a reasonable approach, and does not necessarily cause P-value inflation. This is different from a Bonferroni correction, in which total numbers of tests done is the only factor. These FDR models of the proportion of null hypotheses rely on the distribution of p-values, not their raw count (assuming a sufficient count of P-values is present in which to model). Dividing the FDR correction as they did provides more information about group-specific influences, and should not be considered p-value hacking.
Nevertheless, thank you for calling this out. There is way too much p-hacking in science, especially biology.
[63] Benjamini Y, Krieger AM, Yekutieli D (2006) Adaptive linear step-up procedures that control the false discovery rate. Biometrika 93:491–507.
- carbocation 9y agoRight. My point is actually that they're doing an FDR within each group, not across the whole study. So, the adjusted P values are accurate within each group, but to look globally, you either need to then Bonferroni or re-perform the FDR globally. I'm not saying that my point is in disagreement with yours, by the way, I'm just trying to clarify that my point is independent of whether they had done in-group FDR or in-group Bonferroni. I actually built an online FDR calculator previously, since I enjoy the topic so much: https://tools.carbocation.com/FDR https://tools.carbocation.com/FDR