4 ms·
A/B testing does not tell you what to do. It measures what you have done. HUGE difference, but a difference that very few people actually seem to understand. T
by JoelMarsh 14y ago
A/B testing does not tell you what to do. It measures what you have done. HUGE difference, but a difference that very few people actually seem to understand.
The last paragraph of this article made me shake my head though. If you do A/B tests and then use the losing version anyway - because it feels better in your gut - you're an idiot.
Choose a different path if you want, but if done properly, your A/B results are fact (among the options you tested).
- tgrass 14y agoThe author wrote that they have a vision of what the seatsearching experience should be like. A/B allows us to test what is. It doesn't test for what should be. If the goal is to force the evolution of a trend/habit/market, doesn't one have to go beyond split testing, and venture into the risky unknown?
- JoelMarsh 14y agoFrom the article: "A few months ago we tested a new design for our map UI. The new UI converted a bit worse. But we preferred it, so we kept it anyway." They A/B tested and then went with the LOSING version, because it felt better in their gut. Go for something radically different if you want, no problems there, but don't test if you're going to choose your favorite even when it loses. That's just dumb.
- aurelianito 14y agoWhy? They tested and it converted "a bit" worse. I am pretty sure that if it converted "a lot" worse, they would have not switched. So, the test gave them useful information.