4 ms·

Ask HN: how to iterate or A/B test while you have small conversion rates?

10 points by thenomad 16y ago

So, I have a little side project that's showing some promise and occasionally converting.I want to get down with the A/B testing, of course, because it's been very useful in other projects. Problem is, the conversions to sales don't occur often enough to be statistically useful - with one conversion per day on a very good week, from what I understand of A/B testing I'd need to run for about a month to get meaningful statistics. Not exactly rapid iteration! So, how can I iterate in this case? It's a single-page sales letter with a "buy" link at the bottom - I guess I could try to capture email addresses initially or even split the pages up, but I'm not sure how statistically relevant the results would be, or if changing my sales model like that would actually reduce conversions! I'm guessing I'm not the only guy in this position. Any ideas?

pdenya 16y ago
I'm in the same boat with my site. For now I've given up on A/B testing until I can at least double my traffic. This SEO service seems fairly cheap and came highly recommended by @aymeric: http://taskarmy.com/services/342-get-your-webpage-on-1st-page-of-google http://taskarmy.com/services/342-get-your-webpage-on-1st-pag...
asharp 16y ago
Define a series of heuristics. For example, lets say you have one heuristic that is the number of page views on a particular page, then the number of people who fill out a particular form, then the number of people who.... (Basically various things that should correlate with sales). The easy way is to then go through and first off measure the correlation between each heuristic and sales and then define some facsimile for the change in sales through something like the sum for all h(corr(h, s)*delta h) with h being the heuristic and s being sales. You can then use that as a new test heuristic for A/B testing.
- thenomad 16y ago
  Sadly it's not a "read, register, play around, 30 days run out, pay" kind of deal. The useful events are pretty much - 1) Arrival on page 2) Read down page 3) Click buy button 4) Pay. That's what's doing my head - too few things to measure! I'm debating artificially adding more steps in the chain, but I'm far from certain about that.
  - asharp 16y ago
    Yeah, however you know more then that per user. ie. how long the user is on the page, how many pages they view, where they came from (did they come from google/news.yc/etc?) If you're still stuck, send me an email! (You have no email on your profile page, btw)
    - thenomad 16y ago
      That's wierd - it's listed there... Thanks - I may just do that. Appreciated.
      - asharp 16y ago
        It's listed but it's only used for internal operations (iirc password resets/etc.) Only the user info section of your profile gets seen by others.
cschmidt 16y ago
Try to move up the conversion funnel if you can. For example, track the number of clicks on your buy link, as opposed to the total number of conversions. That will give you bigger numbers to A/B.