3 ms·
> The probability of getting a p-value below 0.05 is 5%, the probability of getting a p-value below 0.25 is 25%, the probability of getting a p-value below 1 is
by bonoboTP 7y ago
> The probability of getting a p-value below 0.05 is 5%, the probability of getting a p-value below 0.25 is 25%, the probability of getting a p-value below 1 is 100%.
But getting a p value between 0.82 and 0.87 is also just 5% probability, so could also be seen as a probably-won't-happen-by-chance event. Sure it's clear most of the time which 5% is the significant one,but not always, for example paradoxically the same result can be significant or not, depending on what your intended stopping criterion was, even if you happened to have to stop at the same time using either criterion. This is because the two variants would lead to declaring a different 5% portion of the null's possible outcomes as the significant part.