3 ms·
Multiple comparisons? 1 million independent tests? Hello? >Your P value is 0.0002. This doesn't count as discovery, but clinically I know what my judgment is g
by eanzenberg 11y ago
Multiple comparisons? 1 million independent tests? Hello?
>Your P value is 0.0002. This doesn't count as discovery, but clinically I know what my judgment is going to be.
>Let's go 5 sigma (p < 3e-7) for discoveries and reserve 0.05 < p < 3e-7 for stuff we should take closer looks at.
^Means you should start a new trial with n > 20 given the same placebo/drug split.
- minimaxir 11y agoIn the scientific world, it is economically infeasable (cost/time) for 1 million tests for a given hypothesis. Even n > 20 can be difficult for certain studies. Bootstrapping the results to simulate 1 million trials won't fix the aforementioned issue either.
- _snydly 11y ago> Even n > 20 can be difficult for certain studies. For some expensive experiments even n=3 is difficult to reach. "Should we do another replicate and go from n=2 to n=3, or should we hire another post-doc?" is a relatively common (rhetorical) question.