6 ms·

Conversion funnels that optimize themselves

33 points by adityar 13y ago

jcnnghm 13y ago
With this multivariate testing, how many visits does it take to get statistically significant data?
- mikkel 13y ago
  Calculating the number of tests needed to reach statistical certainty is not straightforward at all. The equation for simple A/B tests relies on the change of conversion rate from a control. More info here: https://developer.amazon.com/sdk/ab-testing/reference/ab-math.html https://developer.amazon.com/sdk/ab-testing/reference/ab-mat... That's for simple A/B tests - this is not simple A/B tests, instead it is setting up a search space of possible variations of a page and comparing many to many to find which is best. Splitting the data based on visitor referral, country, and more make the sample size needed even larger. TLDR: This solution needs a decent amount of traffic to have reasonable results. The exact numbers are not definite. Source: I co-created xander.io which is basically this concept but more open source.
  - adityar 13y ago
    that does look interesting.
- adityar 13y ago
  Depends on the tests. However a point to note is that this is not classical testing followed by hard-coded values. It's more like a multi-armed bandit-type optimization.
  - wfunction 13y ago
    Yeah, it sounds like the perfect kind of Q-learning problem.
nezza-_- 13y ago
I don't get your site. There's a BIG green button at the top and when I click it nothing happens.. Then there's an empty crystal ball and below that two other buttons that actually do something.. I have to click again to at least get a very basic idea on what you do. Besides that the project actually sounds quite interesting!
- adityar 13y ago
  I really should make that link go to a page that says "See what I mean?"
lingben 13y ago
are you using this to optimize your own site? if yes, do an open case study to show its power if not, why not?
- Duhck 13y ago
  Came here to ask this. Your call to actions, etc are kind of lack-luster (although I have been waiting for a product like this for quite some time). Are you optimizing your own funnel(s) with this? Please tell!
  - adityar 13y ago
    product is in development and testing at this time so,no. But, that's going to be a persuasive argument - we eat our own dog food.
    - lingben 13y ago
      wait, so you are not willing to use your own product now but you expect others to pay for it and use it now? :/
viana007 13y ago
Site is off =( This Site44-hosted website has been temporarily suspended due to excessive usage.
- adityar 13y ago
  now fixed
rabidonrails 13y ago
down :(
ivan_ah 13y ago
This is a very interesting idea because it drastically lowers the bareer to entry for someone wishing to try A/B testing. I think business intelligence should be part of any company's core competences, but there is a market for this kind of automation too. Some more general feedback: - big call-to-action link in the middle of the top goes nowhere! - better styling everywhere - pricing should fit on one screen! - marketing: focus on GET RESULTS, and IMPROVE CONVERSIONS and less on the technical side---my guess is that your main customers will be business people not technical folk, so talk about the RESULTS and not so much about the tech. keywords: ML, OPTIMIZE, BETTER, AUTOMATED, SMART, etc... - who are you targeting with this pricing? I think if your product really works it is delivering the same value as a full time employee doing A/B testing and optimizing the website for you --- so you should charge more. Also -- the ppl who are most likely to benefit from this are high traffic websites (1000+ visitors for stat. signif?) so you should focus on them as your target market. I think it will be much better to get 2-3 BIG customers than chase mom-and-pop and their little website. - maybe have a demo? (tell user to keep reloading a page/iframe) so they see the different versions. On a separate note, I think the notion of an "interactive" sales pitch that is custom generated for the user is interesting in its own right. Check out the link to my book on my hn profile page --- I have little conversation-like bubbles which add a bit of interactivity. This concept could go so much further though... Think "webpage as a conversation." Good job on the coding. Now go open the right .css file and make your site look pretty ;)
- dougmccune 13y ago
  since I'm assuming they're eating their own dogfood isn't it just a matter of time before the page morphs into a version that addresses all your points? The big call-to-action that goes nowhere will be tested out... maybe the pricing page you get that doesn't fit on one screen is just one version that will get phased out :) Maybe I'll check back in a few weeks, and if the site has morphed into the best sales page ever then it means the tech works
- adityar 13y ago
  Thanks for the feedback. My initial target is small/medium web app developers and not so much the big guys (yet). At this stage, some A/B & design saavy is required and not fully eliminated. There's a lot of demo possibilities (one that comes up very often - eat your own dogfood much?) which I think will improve conversions.
fraserharris 13y ago
This is a perfect example of building a technology, not a product. This is Optimizely without the amazing user interface that allows marketers to do this work.
- adityar 13y ago
  is that good or bad?

33 points by adityar 13y ago

jcnnghm 13y ago

With this multivariate testing, how many visits does it take to get statistically significant data?

mikkel 13y ago
Calculating the number of tests needed to reach statistical certainty is not straightforward at all. The equation for simple A/B tests relies on the change of conversion rate from a control. More info here: https://developer.amazon.com/sdk/ab-testing/reference/ab-math.html https://developer.amazon.com/sdk/ab-testing/reference/ab-mat... That's for simple A/B tests - this is not simple A/B tests, instead it is setting up a search space of possible variations of a page and comparing many to many to find which is best. Splitting the data based on visitor referral, country, and more make the sample size needed even larger. TLDR: This solution needs a decent amount of traffic to have reasonable results. The exact numbers are not definite. Source: I co-created xander.io which is basically this concept but more open source.
- adityar 13y ago
  that does look interesting.
adityar 13y ago
Depends on the tests. However a point to note is that this is not classical testing followed by hard-coded values. It's more like a multi-armed bandit-type optimization.
- wfunction 13y ago
  Yeah, it sounds like the perfect kind of Q-learning problem.

nezza-_- 13y ago

I don't get your site. There's a BIG green button at the top and when I click it nothing happens.. Then there's an empty crystal ball and below that two other buttons that actually do something.. I have to click again to at least get a very basic idea on what you do. Besides that the project actually sounds quite interesting!

adityar 13y ago
I really should make that link go to a page that says "See what I mean?"

lingben 13y ago

are you using this to optimize your own site? if yes, do an open case study to show its power if not, why not?

Duhck 13y ago
Came here to ask this. Your call to actions, etc are kind of lack-luster (although I have been waiting for a product like this for quite some time). Are you optimizing your own funnel(s) with this? Please tell!
- adityar 13y ago
  product is in development and testing at this time so,no. But, that's going to be a persuasive argument - we eat our own dog food.
  - lingben 13y ago
    wait, so you are not willing to use your own product now but you expect others to pay for it and use it now? :/

viana007 13y ago

Site is off =( This Site44-hosted website has been temporarily suspended due to excessive usage.

adityar 13y ago
now fixed

rabidonrails 13y ago

down :(

ivan_ah 13y ago

This is a very interesting idea because it drastically lowers the bareer to entry for someone wishing to try A/B testing. I think business intelligence should be part of any company's core competences, but there is a market for this kind of automation too. Some more general feedback: - big call-to-action link in the middle of the top goes nowhere! - better styling everywhere - pricing should fit on one screen! - marketing: focus on GET RESULTS, and IMPROVE CONVERSIONS and less on the technical side---my guess is that your main customers will be business people not technical folk, so talk about the RESULTS and not so much about the tech. keywords: ML, OPTIMIZE, BETTER, AUTOMATED, SMART, etc... - who are you targeting with this pricing? I think if your product really works it is delivering the same value as a full time employee doing A/B testing and optimizing the website for you --- so you should charge more. Also -- the ppl who are most likely to benefit from this are high traffic websites (1000+ visitors for stat. signif?) so you should focus on them as your target market. I think it will be much better to get 2-3 BIG customers than chase mom-and-pop and their little website. - maybe have a demo? (tell user to keep reloading a page/iframe) so they see the different versions. On a separate note, I think the notion of an "interactive" sales pitch that is custom generated for the user is interesting in its own right. Check out the link to my book on my hn profile page --- I have little conversation-like bubbles which add a bit of interactivity. This concept could go so much further though... Think "webpage as a conversation." Good job on the coding. Now go open the right .css file and make your site look pretty ;)

dougmccune 13y ago
since I'm assuming they're eating their own dogfood isn't it just a matter of time before the page morphs into a version that addresses all your points? The big call-to-action that goes nowhere will be tested out... maybe the pricing page you get that doesn't fit on one screen is just one version that will get phased out :) Maybe I'll check back in a few weeks, and if the site has morphed into the best sales page ever then it means the tech works
adityar 13y ago
Thanks for the feedback. My initial target is small/medium web app developers and not so much the big guys (yet). At this stage, some A/B & design saavy is required and not fully eliminated. There's a lot of demo possibilities (one that comes up very often - eat your own dogfood much?) which I think will improve conversions.

fraserharris 13y ago

This is a perfect example of building a technology, not a product. This is Optimizely without the amazing user interface that allows marketers to do this work.

adityar 13y ago
is that good or bad?