Also, you only need about 400 samples for 95% confidence on a proportion at the highest variance point of p1 = . 0.50. It used to drive me nuts when my CEO used

Also, you only need about 400 samples for 95% confidence on a proportion at the highest variance point of p1 = . 0.50. It used to drive me nuts when my CEO used to say “10 clicks out of 10,000 on this A/B test is in the noise”. No, that’s not in the noise, it’s almost certainly a clickthrough rate of close to 0.