3 ms·
Does not matter. The effects on African American populations were probably not part of the pre defined endpoints, so yes, it is perfectly acceptable to say the
by ekianjo 8y ago
Does not matter. The effects on African American populations were probably not part of the pre defined endpoints, so yes, it is perfectly acceptable to say the study is negative. Looking at subgroups a posteriori is essentially what leads to p hacking and this is the behavior that we want to discourage in sciences.
- Leary 8y agoShouldn't one consider what the p-value is in weighing whether it is a fluke or not?
- jcranmer 8y agoIn theory, you can do some more statistical tests to study if the subgroup analysis has statistically significant results. In practice, those tests are probably going to be skipped or done improperly, and it's difficult to tease out if the subgroup analysis suffers from the statistical fraud of reanalyzing until you get a positive result. Redoing the test from the start with the subgroup as the primary endpoint would always get you more accurate data, and history has pretty clearly shown that most instances of "we looked at the subgroups and found something where it works!" tend to not actually work when done properly anyways.
- ekianjo 8y agoThe study is designed to evaluate precise and well-defined endpoints. Any additional observation not planned from the start does not have the same rigor (since the study does not control for all thousands of variables at once, obviously), so in practice you should never draw conclusions from what was not agreed to evaluate from the start.