Statistically rigorous results, with minimal regret. The results are only statistically rigorous provided your bandit obeys relatively strong assumptions. As

Statistically rigorous results, with minimal regret. The results are only statistically rigorous provided your bandit obeys relatively strong assumptions. As another example, suppose you ran a 2-week test. Suppose that from week 1 to week 2, both conversion rates changed, but the delta between them remained roughly the same. A 50/50 A/B split doesn't mind this, and in fact still returns the right answer. Bandits do mind. I don't do p-values. I do Bayesian testing, same as you. I just recognize that in the real world, weaker assumptions are more robust to experimenter or model error, both of which are generally the dominant error mode. In web A/B testing, the latter is usually the most applicable, and for that, you cannot beat Thompson sampling on the average, no matter how clever yur scheme. This is simply not true. The Gittins Index beats Thompson sampling, subject again to the same strong assumptions. Look, I know the theoretical advantages of bandits and I advocate their use under some limited circumstances. I just find the stronger assumptions they require (or alternately the much heavier math requirements) mean they aren't a great replacement for A/B tests which are much simpler and easier to get right.