3 ms·
In reverse order. #3. For real companies with completely unmotivated visitors, 60% conversion rates from visitor to paying customer is untenable. 5% is proba
by btilly 13y ago
In reverse order.
#3. For real companies with completely unmotivated visitors, 60% conversion rates from visitor to paying customer is untenable. 5% is probably too much to hope for.
#2. Absolutely. A/B tests should be focused on actual balanced decision points. "Should I sign up?" "Should I open this email?" "Should I go back to this site?" But not on people who are already committed to doing what they were going to do anyways.
#3. On sample sizes, it is important to think carefully about the maximum effort you're willing to put into a test. Be very, very cautious about accepting test results that arrive early. No, 99% confidence is not enough to stop with 200 conversions, and 99.9% probably isn't either. Don't worry about statistical significance when you kill tests that have run too long to be worth continuing. This is a line of reasoning that is sadly rare in our industry. (I keep meaning to write my next article on that topic. But http://elem.com/~btilly/ab-testing-multiple-looks/part2-limited-data.html http://elem.com/~btilly/ab-testing-multiple-looks/part2-limi... explains one way to come up with such a strategy.)
- pbiggar 13y agoLooks at OP's point about traffic source. Its likely that ones of your traffic sources converts way better than 5%, while unqualified people straight from google will convert much lower.