> You avoid the green jelly bean problem entirely. Not entirely. Quite often, the big fishing expedition research is done on things like epidemiological data s

> You avoid the green jelly bean problem entirely. Not entirely. Quite often, the big fishing expedition research is done on things like epidemiological data sets, where you've got a million monkeys with a million typewriters trying to publish a million papers, all based on one data set. Under that kind of situation, assuming that everything works out perfectly, after the first pass you'll have some random non-negative number of hypotheses that were collected from fishing that you're going to test again. And those tests will also have a 5% type I error rate, so the fishing expedition will still have a problem with multiple comparisons. Your cumulative risk of a type I error will be mitigated, but not reduced to the traditional 5% (or whatever alpha you were shooting for). Meanwhile, your risk of a type II error will have been increased considerably: You're replacing one test of a relatively high statistical power with two tests of relatively lower power, and set up a situation where a type II error in either of them yields a type II error in the overall test. Not saying that there's no value to approaching things that way, but it's not the free lunch you're suggesting it is.