3 ms·

"A Bayseian FDA regulator would b more forgiving". He absolutely would not. It should be obvious to anyone who gives the matter any thought that it must be more

"A Bayseian FDA regulator would b more forgiving". He absolutely would not. It should be obvious to anyone who gives the matter any thought that it must be more probable that you'll conclude drug A is better than drug B at the 5% level at some point over the course of an open ended experiment than that that will be the case at the end of a trial with a specific number of runs. In fact, the open ended stop when you win method is the equivalent of running an enormous trial and then re-anlyzing the result at each point and publishing the most favorable point as the result.

loup-vaillant 8y ago
The "open ended stops when you win" method, has to take into account contrary evidence into account. If you make 10 experiments, and 3 of those say your new and improve medication doesn't work, you have to take those into accounts, or else you're just cheating. The only remedy against publication bias based cheating is to publish everything, including the failures. That will take care of the "wait until I get a 1 in 20 fluke" just so you can get past the p<0.05 threshold.