3 ms·
Would A/B testing encompass changes as large as what happened here? Having never done A/B testing, it sounds like an iterative process. Would continuous A/B t
by Splines 17y ago
Would A/B testing encompass changes as large as what happened here? Having never done A/B testing, it sounds like an iterative process. Would continuous A/B testing ever converge on a solution like the one arrived here?
I guess what I'm asking is this: Is A/B testing an evolutionary approach to developing a product? Are "revolutionary" leaps possible if you strictly follow the A/B testing methodology? (My guess: probably not, which implies that most people don't follow A/B exclusively, which sounds like common sense).
- inerte 17y agoYes, but not encouraged. The literature, or most common applications of A/B testing, is about little improvements. Phrases being changed, picture alignment, that kinda of thing. Of course, in theory, A/B as the name implies is about testing two different approaches, so again, in theory, any stuff completely different can be seen as an experiment for A/B. The problem is that humans have been trying different ways to do things for a loooooooooong time, and the whole point of trying to teach and learn A/B is to test things, measure, and walk towards your goal. So yes, you can "A/B test" two completely different things, but it will be so. much. harder to actually know what it worked. For example, you're A/B testing a bicycle and a Ferrari Testarossa. D'uh... obviously they'll work differently, but what caused? Price, color, speed, brand? That's why when A/B testing is mentioned it's about small, isolable, repeatable, incremental and evolutionary approach. Because it's a technique with this purpose in mind.