I think a more important blunder is failing to understand statistics. "I’ve ran plenty of tests where one variation was the control, hitting 95% confidence, af

I think a more important blunder is failing to understand statistics. "I’ve ran plenty of tests where one variation was the control, hitting 95% confidence, after just a handful of conversions ... Because you need to run your tests long enough to get accurate results! You need to get past what I call “random coin syndrome”." Statistics don't work that way. A result is either significant (e.g. you can either draw conclusions with a 95% confidence rate), or it is not. If you reach 95% confidence after 14 conversions (10 vs 4), it either means your questions are too broad or you failed to control for external influences.