4 ms·
You're right, I did have the wrong scale, and should have realized. I believe this plot to be correct. Black is binomial, red is normal, both for the difference
by sesqu 16y ago
You're right, I did have the wrong scale, and should have realized. I believe this plot to be correct. Black is binomial, red is normal, both for the difference in conversion rates.
http://img267.imageshack.us/img267/6691/binnormdifflikelihoodra.png http://img267.imageshack.us/img267/6691/binnormdifflikelihoo...
The normal approximation seems reasonable, and so a t-test shouldn't cause problems. I made a plot of the relative error of the normal approximation; the tails are indeed too fat. In particular, there's a bump to the left of 0, so a t-test would slightly overestimate p.
http://img205.imageshack.us/img205/6691/binnormdifflikelihoodra.png http://img205.imageshack.us/img205/6691/binnormdifflikelihoo...
- btilly 16y agoThe problem is that when you're near confidence you're in the tail, and the student t-test is extremely sensitive to the shape of said tail. If n is large enough this difference will be washed away as everything converges to normals. But with smaller sample sizes the difference can be quite significant.