3 ms·
Those small changes can be significant. A/B tests need to be as specific to one adjustment as possible. I've noticed flaws like this many times on ABtests.com
by dmix 17y ago
Those small changes can be significant.
A/B tests need to be as specific to one adjustment as possible.
I've noticed flaws like this many times on ABtests.com. But I guess they are like any war stories, its the overall idea that counts not the details.
- dcancel 17y agoAppreciate the feedback. Would love to make our site better, any suggestions would be very much appreciated. What are the primary flaws you're seeing?
- dmix 17y agoI was referring to the accuracy of the test results on abtests.com in general not the site in the link. Many of the A/B examples had multiple changes to the site but only noted one as making a difference. In your case adding a headline to the form was overlooked. You were using form submissions as a metric from what I can tell. So changes specifically to the signup form itself (such as the header) is likely to have as much of an impact as the video. Did you track how many people watched the video then signed up as a metric? That may be more telling.