4 ms·
Ask HN: A/B testing with only 1-2 sign-ups per month
I would like to employ A/B testing for my web service homepage and sign-up. My site only gets 1-2 sign-ups per month.
How long would I need to run the experiment to get a reasonably decent idea of which version is statistically better? (I have no statistical background/training.)
The wikipedia page (http://en.wikipedia.org/wiki/A/B_Testing) is very light on details.
Any help would be greatly appreciated.
- Alex3917 17y agoBased on your current traffic? About 55 years.
- timmorgan 17y agoSo you're telling me there's a chance... :-)
- nate 17y agoHere's a great post on doing the math for this stuff http://blog.asmartbear.com/easy-statistics-for-adwords-ab-testing-and-hamsters.html http://blog.asmartbear.com/easy-statistics-for-adwords-ab-te... But yes, the bad news is it's going to take too long at your rate right now. I recommend looking at things like http://conceptfeedback.com http://conceptfeedback.com to get feedback in lieu of being able to test right now. You could also consider running a poll with Mechanical Turk workers. The turkers would answer a poll like which design is better for about a $0.05 a person. I've done the conceptfeedback.com for testing some new elements in http://tgethr.com http://tgethr.com before we had the traffic to start experimenting and the feedback was great.
- timmorgan 17y agoWonderful advice. Thank you.