Just to expand on this, David Robinson quantified this a while back: http://varianceexplained.org/r/bayesian-ab-testing/ http://varianceexplained.org/r/bayesian

Just to expand on this, David Robinson quantified this a while back: http://varianceexplained.org/r/bayesian-ab-testing/ http://varianceexplained.org/r/bayesian-ab-testing/ He ran some simulated A/B tests using my Bayesian A/B testing technique (which is now powering VWO). He showed that while peeking does ensure that loss < specified threshold, if you don't peek the loss will be even lower. So although peeking is still valid, that validity does come at a cost.