5 ms·

How To Increase Site Performance Through A/B Split

12 points by jmonegro 17y ago

jacquesm 17y ago
AB testing is a great tool to make your site perform better, if you're not doing it yet you probably should!
geuis 17y ago
We've done 2 of these tests in the last few weeks using a small in-house framework I wrote and its already showing improvements in overall page views. Our initial tests have been small and were done completely client-side with javascript, using an existing internal reporting solution. Its very easy to set this kind of thing up without even needing complicated 3rd party tools.
modoc 17y ago
Does anyone have feedback on the Google A/B testing tool, Website Optimizer? It looks neat, but I haven't tried it yet.
- patio11 17y ago
  Having A/B testing is better than not having it, and Website Optimizer is quick to implement and holds the hand of people who don't understand what an A/B test is yet. However, because it is implemented in Javascript, the impact on page load times is substantial (download page A and assets, execute Javascript, download page B, actually see page). Since you will often want to use it on high value pages like, oh, your checkout funnel that is less than optimal.
  - sachinag 17y ago
    Does something like Mixpanel do A/B testing better, or is there a hosted solution that you'd recommend? (Anything to not have to roll one's own because laziness is a virtue.)
  - burnout1540 17y ago
    If you follow Google's instructions and put the Control Script within the <head> tag on page A, the JavaScript will get executed and the user will get redirected to page B before page A and its assets load. Also, if the user gets put into group A, then all of page A is allowed to load and the user is not redirected at all.
paraschopra 17y ago
Though simple A/B tests are better than not doing any tests at all, the true power of testing is realized when you are doing targeted tests. In most of the solutions currently in the market, you can only do blanket tests: that is, all visitors see the same test. Much better approach would be to do tests for a segment of visitors because it is quite likely that your organic traffic would have different optima than your direct traffic. Similarly, repeat visitors would respond differently than first time visitors. Also, websites usually have multiple goals. For example, while your A/B test may tell you that you have reached optima as far as clicks on the banner is concerned, you won't know if you have compromised on other goals such as visitor engagement, newsletter signups, etc. The point here is that you must measure performance on multiple goals to see the correct tradeoff. My team has been developing a powerful testing solution called Wingify (http://www.wingify.com/ http://www.wingify.com/). We have limited number of private beta accounts available. If anyone is interested, mail me at paras@wingify.com Oh, and BTW, there is a screencast of using Wingify for setting up an experiment as well. You might find it helpful for an introduction to A/B testing - http://www.wingify.com/video-demos/create_experiment/ http://www.wingify.com/video-demos/create_experiment/

12 points by jmonegro 17y ago

AB testing is a great tool to make your site perform better, if you're not doing it yet you probably should!

geuis 17y ago

We've done 2 of these tests in the last few weeks using a small in-house framework I wrote and its already showing improvements in overall page views. Our initial tests have been small and were done completely client-side with javascript, using an existing internal reporting solution. Its very easy to set this kind of thing up without even needing complicated 3rd party tools.

modoc 17y ago

Does anyone have feedback on the Google A/B testing tool, Website Optimizer? It looks neat, but I haven't tried it yet.

patio11 17y ago
Having A/B testing is better than not having it, and Website Optimizer is quick to implement and holds the hand of people who don't understand what an A/B test is yet. However, because it is implemented in Javascript, the impact on page load times is substantial (download page A and assets, execute Javascript, download page B, actually see page). Since you will often want to use it on high value pages like, oh, your checkout funnel that is less than optimal.
- sachinag 17y ago
  Does something like Mixpanel do A/B testing better, or is there a hosted solution that you'd recommend? (Anything to not have to roll one's own because laziness is a virtue.)
- burnout1540 17y ago
  If you follow Google's instructions and put the Control Script within the <head> tag on page A, the JavaScript will get executed and the user will get redirected to page B before page A and its assets load. Also, if the user gets put into group A, then all of page A is allowed to load and the user is not redirected at all.

paraschopra 17y ago

Though simple A/B tests are better than not doing any tests at all, the true power of testing is realized when you are doing targeted tests. In most of the solutions currently in the market, you can only do blanket tests: that is, all visitors see the same test. Much better approach would be to do tests for a segment of visitors because it is quite likely that your organic traffic would have different optima than your direct traffic. Similarly, repeat visitors would respond differently than first time visitors. Also, websites usually have multiple goals. For example, while your A/B test may tell you that you have reached optima as far as clicks on the banner is concerned, you won't know if you have compromised on other goals such as visitor engagement, newsletter signups, etc. The point here is that you must measure performance on multiple goals to see the correct tradeoff. My team has been developing a powerful testing solution called Wingify (http://www.wingify.com/ http://www.wingify.com/). We have limited number of private beta accounts available. If anyone is interested, mail me at paras@wingify.com Oh, and BTW, there is a screencast of using Wingify for setting up an experiment as well. You might find it helpful for an introduction to A/B testing - http://www.wingify.com/video-demos/create_experiment/ http://www.wingify.com/video-demos/create_experiment/