2 ms·
Yeah, I'm almost done with what I'd call our MVP (and that's after spending a long time working on a very full-featured site that just wasn't as compelling as t
by joshdotsmith 16y ago
Yeah, I'm almost done with what I'd call our MVP (and that's after spending a long time working on a very full-featured site that just wasn't as compelling as the MVP itself), so these metrics are going to help me figure out where to head next.
I'm not sure yet how I'd go about building an A/B platform, but its on the docket of things I want to do.
Also, the reason I'd be running a cron job is to have hourly data for all of these metrics (to look for those changes over time).
- btilly 16y agoSee http://elem.com/~btilly/effective-ab-testing/ http://elem.com/~btilly/effective-ab-testing/ for my advice on how to put together an A/B platform. Sections 2-4 are the most relevant. The most important detail is the warning in slide 34 that you should not use the person_id to set the A/B test version. The reason is that if you make that mistake, then you can only run one test at a time, and can't easily drop versions of tests later. On the other hand if you make assignment random, then you are free to run tests in parallel, which is very useful for trying out lots of ideas. (Particularly when you have less traffic.)
- joshdotsmith 16y agoThanks for that! I actually just signed up with Optimizely today and really love the way they run A/B tests so far. I need to think a little deeper on how to integrate these tests with the rest of my metrics, but it's a really good (and easy) start. While I haven't reached a level of real statistical confidence yet with my first test, I've already been able to increase engagement with a single change on my landing page by 35%. Wow. clickable: http://www.optimizely.com/ http://www.optimizely.com/