4 ms·

Shouldn't one consider what the p-value is in weighing whether it is a fluke or not?

• jcranmer 8y ago
  In theory, you can do some more statistical tests to study if the subgroup analysis has statistically significant results. In practice, those tests are probably going to be skipped or done improperly, and it's difficult to tease out if the subgroup analysis suffers from the statistical fraud of reanalyzing until you get a positive result. Redoing the test from the start with the subgroup as the primary endpoint would always get you more accurate data, and history has pretty clearly shown that most instances of "we looked at the subgroups and found something where it works!" tend to not actually work when done properly anyways.
• ekianjo 8y ago
  The study is designed to evaluate precise and well-defined endpoints. Any additional observation not planned from the start does not have the same rigor (since the study does not control for all thousands of variables at once, obviously), so in practice you should never draw conclusions from what was not agreed to evaluate from the start.