3 ms·
See http://elem.com/~btilly/effective-ab-testing/ http://elem.com/~btilly/effective-ab-testing/ for my advice on how to put together an A/B platform. Sections
by btilly 16y ago
See http://elem.com/~btilly/effective-ab-testing/ http://elem.com/~btilly/effective-ab-testing/ for my advice on how to put together an A/B platform. Sections 2-4 are the most relevant.
The most important detail is the warning in slide 34 that you should not use the person_id to set the A/B test version. The reason is that if you make that mistake, then you can only run one test at a time, and can't easily drop versions of tests later. On the other hand if you make assignment random, then you are free to run tests in parallel, which is very useful for trying out lots of ideas. (Particularly when you have less traffic.)
- joshdotsmith 16y agoThanks for that! I actually just signed up with Optimizely today and really love the way they run A/B tests so far. I need to think a little deeper on how to integrate these tests with the rest of my metrics, but it's a really good (and easy) start. While I haven't reached a level of real statistical confidence yet with my first test, I've already been able to increase engagement with a single change on my landing page by 35%. Wow. clickable: http://www.optimizely.com/ http://www.optimizely.com/