3 ms·

This is false. The null hypothesis is either correct or false. The veracity of the null hypothesis does not change as long as the experiments are repeated the s

• Tloewald 12y ago
  How does this contradict what I said? Enlighten me? I said the theory -- by which I meant the non-null hypothesis -- is not endorsed by a low p value (indeed, this is a major point of the article). A low p-value says "hey they doesn't look like random data" not "your brilliant hypothesis is probably true". The data might not look random because of a methodological error, outright fraud, or a confound. This is particularly important when you consider people looking at data over and over again trying to find "an effect". Theoretically, the tests are supposed to get tougher and tougher each time you examine the data (add one degree of freedom) but in practice this doesn't happen. It doesn't matter much with large data sets, but the social sciences often use datasets where n is roughly 100, and you might only have 20 subjects in a cell.