2 ms·
That tool is giving you a standard frequentist p-value. Which means that if you insist on 95% confidence, and take multiple looks, then you'll have more than a
by btilly 14y ago
That tool is giving you a standard frequentist p-value. Which means that if you insist on 95% confidence, and take multiple looks, then you'll have more than a 5% chance of eventually getting to confidence randomly.
If you stop your tests on the p-value cutoffs that I gave in my third graph, you'll get very strong a priori guarantees that the decisions that you make will be right. The downside is that you won't have any guarantee of making decisions in reasonable time. But that is the subject of the next article.