(Addressing the owners of visual.ly) I hope that "Chance to beat baseline" column isn't a naive measurement of "confidence" that's updated on every iteration.

(Addressing the owners of visual.ly) I hope that "Chance to beat baseline" column isn't a naive measurement of "confidence" that's updated on every iteration. I've mentioned this several times in threads such as these, but if you're blindly applying basic statistical analysis to a partially completed test, you are badly misinforming your customers. Please read this: http://www.evanmiller.org/how-not-to-run-an-ab-test.html http://www.evanmiller.org/how-not-to-run-an-ab-test.html Of course, if you are taking these results into account I apologise for mentioning this again and wish you every success.