4 ms·
> The purpose of an A/B test isn't to always show the best performing result The key part is 'not always'. Using A/B tests just to find the better performing
by arendtio 7y ago
> The purpose of an A/B test isn't to always show the best performing result
The key part is 'not always'.
Using A/B tests just to find the better performing version is a valid use case too. And if you have the traffic, multi-armed bandits are a nice way of automating the whole procedure. Even scientifically speaking there is nothing wrong with them. Their biggest issue is that they require a lot of traffic for significant results.
- joshuamorton 7y agoIn practice, MAB should require less traffic than an A/B test for the same level of significance. Although it's much more difficult to rigorously describe that significance level with a MAB.