2 ms·
Instead of just running A versus incremental-change A2, also run a B version that's radically different from A. Thus you reap the straightforward benefits of in
by duck 16y ago
Instead of just running A versus incremental-change A2, also run a B version that's radically different from A. Thus you reap the straightforward benefits of incremental improvements while also searching for something that could radically improve your revenue.
A very powerful concept that you don't see done very often.
- samlittlewood 16y agoIt is worth looking at "Simulated Annealing": http://en.wikipedia.org/wiki/Simulated_annealing http://en.wikipedia.org/wiki/Simulated_annealing Start your optimization process with a high "temperature" (leading to big changes), and let it "cool" over subsequent iterations.
- paraschopra 16y agoSee http://visualwebsiteoptimizer.com/split-testing-blog/how-aquasoft-increased-their-sales-by-20-doing-ab-split-tests-in-multiple-phases/ http://visualwebsiteoptimizer.com/split-testing-blog/how-aqu... as an example of testing radical change. See http://visualwebsiteoptimizer.com/split-testing-blog/using-ab-split-testing-to-refine-your-startups-positioning-90-increase-in-conversion-rate/ http://visualwebsiteoptimizer.com/split-testing-blog/using-a... as an example of subtle change. In my opinion, it is entirely misguided to blame local maxima problem on A/B testing. The methodology doesn't dictate what is that you want to test. It is all about having a hypothesis and then collecting data to prove/disprove it. The post looks like it is blames poor results on instruments while wrong experimental design is the true culprit.