13 ms·
A/B testing has two big and obvious problems. One is that A/B testing can only lead to a local maxima in the best case scenario. The other is that A/B testing,
by bad_user 11y ago
A/B testing has two big and obvious problems. One is that A/B testing can only lead to a local maxima in the best case scenario. The other is that A/B testing, being about statistical hypothesis testing, is prone to interpretation problems, which is why the changes you introduce in the variation have to be small, otherwise you don't know what you're measuring.
In other words, yes it can help you optimize the color or size of a Buy Now button. But it can't help you build a product.
- stdbrouw 11y ago> One is that A/B testing can only lead to a local maxima in the best case scenario. What you mean is: it can only lead to a local maximum in the worst case scenario. Even blind guesses can occasionally lead to a global maximum, and A/B testing is better than a blind guess.
- teaneedz 11y agoAgain, another insightful and accurate assessment!