3 ms·
> I'm curious why I hear so much about A/B testing, but rarely hear about other statistical methods for assessing changes in usage patterns. Are they not being
by jcr 15y ago
> I'm curious why I hear so much about A/B testing, but rarely hear about other statistical methods for assessing changes in usage patterns. Are they not being used? Am I simply looking in the wrong places? Or is A/B testing really sufficient in most cases?
I believe part of the problem is you actually know the definition of
"A/B Testing" and the real definition (testing a single, specific
change) is mostly lost on the majority of those using the phrase. In
other words, "A/B Testing" in common use has essentially become slang
for "Multi-Variate Testing" and other forms of testing.
For example, if you only change the color of a button, you're (formally)
doing "A/B Testing" but if you change both the color and location of a
button, you're (formally) doing "Multi-Variate Testing." None the less,
you'll often see the later called the former in common usage.