3 ms·
Really love the animation, but I can't help but wonder if A/B testing couldn't be explained more powerfully. Within the example of the store for example, the r
by mtrimpe 13y ago
Really love the animation, but I can't help but wonder if A/B testing couldn't be explained more powerfully.
Within the example of the store for example, the real pain A/B testing addresses is cyclical variation leading to incorrect conclusions.
Working from that you could've shown the story of:
* the store owner rearranging his display and seeing sales go up
* but then the rest of the year sales actually going way down
* and when he changes things back to the old layout sales went waaay up again! How can that be?
* Cue image of cyclical charts and the facepalm of realizing it wasn't his redesign at all!
* Now ... on his website he can send visitors to different versions at the same time!
* So he'll never have a year of bad sales again!
- siddharthdeswal 13y agoThose are good points, but we were gunning for the marketer who isn't familiar with A/B testing. This because we see that a lot of our potential customers have usually heard the term somewhere but need a basic, easy-to-understand explanation. Do you think it achieves that purpose?
- e12e 13y agoPerhaps you should deploy an alternate version, and meassure conversion rates? ;-) Seriously, I think you make a great point of selling your service. I also think you gloss over things like cyclic variations -- but I'm not sure it's your job to educate people in the subtle art of fact oriented market research (or what one should call it). For those that don't have the time to learn, changing it probably wouldn't matter. For those that do -- chances are they probably will find it out from a more comprehensive resource (like, say, a good book).
- gbrindisi 13y ago> (like, say, a good book). Can you suggest one?
- siddharthdeswal 13y agoExactly. For someone who's just starting with A/B testing, this serves as a good/interesting introduction. But for someone who really wants to understand the detailed behind-the-scenes, this will obviously fall short. Coming to your suggestion, do you have a good book/website in mind?
- e12e 13y agoI'm afraid I don't have any books to recommend -- maybe someone else has?
- mtrimpe 13y agoYeah, you could interpret my feedback as saying that you're addressing the what but not the why. Looking at it as sales/marketing copy though I'd say that your biggest competitor by far isn't people not understanding what it is but people thinking "Why don't I just change my site and see what happens?" In that context I'd guess this story telling format is perfect for taking a concept like 'prevent acceptance of false hypotheses by controlling for external variation through randomized trials' and turning it into a visceral "Oh fuck. That's painful!" moment your target market can relate to.
- paraschopra 13y agoYes, we indeed missed explaining why A/B simultaneously is how it should be done v/s A after B. Thanks for the inputs.
- aymeric 13y agoYes I agree that it feels that the animation is not powerful enough.