In the context of A/B Testing, if you've decided to stop the test at "95% significance" then you'll stop at the 19 yes, 1 no spot (or maybe 38:2). If you're tes

In the context of A/B Testing, if you've decided to stop the test at "95% significance" then you'll stop at the 19 yes, 1 no spot (or maybe 38:2). If you're testing for 99% significance, you won't stop unless you get to 99:1 (or more likely, you'll need to test way past that because "no" number 2 will arrive before "yes" number 99, so you'll need to test to 198:2 or 297:3 or further. This is why you should never stop an A/B test once you've "hit your statistical significance". Always choose the number of tests you'd need to prove the significance before you start, and let it run even if it's "obviously winning" (or losing).