4 ms·
> Maybe a difference: A p-value is what a researcher has in mind before the hypothesis test as a 'cutoff value', say, 5%, for alpha while alpha is whatever the
by kgwgk 11y ago
> Maybe a difference: A p-value is what a researcher has in mind before the hypothesis test as a 'cutoff value', say, 5%, for alpha while alpha is whatever the statistical hypothesis test says, for the data and the test, is the probability of Type I error, e.g., 3.141592653 or some such random number. So, since the 3.14 is less than the p-value of 5%, the experimenter rejects the null hypothsis. MAYBE this is the difference, if any, between p-value and alpha, in which case we're talking a triviality.
This is completely wrong: you're confusing the alpha and the p-value. Maybe you meant that alpha (5% in your example) is the preset type I error rate (significance level) and the p-value (here 3.14%) is calculated from the data. But conceptually the alpha and the p-value are different things and shouldn't be mixed. 3.14% is definitely not "for the data and the test, [...] the probability of Type I error".
If you're doing hypothesis testing, you calculate 3.14 which is less than 5 (alpha) and you reject the null hypothesis. If the calculation had yielded 0.69 instead of 3.14 you would also reject the null hypothesis. But you cannot consider the second result to be "stronger" than the first one. They are both below the threshold and that's all that matters.
If you're using p-values as evidential measures, the second result is indeed "stronger" because 0.69<3.14. But in this framework there are no error rates involved.
> The paper you referenced also starts to get wound up over beta = 1 - alpha as the power of a test.
I've not looked at the paper carefully, but it's unlikely. Power = 1 - beta, it is not directly related to alpha. (Maybe it's just a typing error, I'm not sure what is your point. I mention it mostly for the benefit of other readers that might be confused by your definition.)