4 ms·

This is a great question! The inspiration for Optimizely came from my experience during the Obama campaign where I was the Director of Analytics. We used Googl

This is a great question! The inspiration for Optimizely came from my experience during the Obama campaign where I was the Director of Analytics. We used Google Website Optimizer extensively and even tried Omniture Test & Target. The biggest pain point for us was setup time & implementation. We were all pretty technical guys but we only had so many hours in the day. Having to go back to the code base and remove add noscript blocks everywhere was a huge tax on our resources. Omniture has a similar pain called "mBox hell" caused by having to keep going back to your CMS or code base to make changes every time you want to run a new experiment. We probably only had the opportunity to run 10% of the experiments we wanted to during the campaign. In the end it meant an incremental $60 million we were able to raise but I'm convinced that if we had Optimizely during the campaign that number would have been a lot bigger. Google Website Optimizer is a great product but if you are short on time and don't want to muck with your code give Optimizely a try. Certainly let us know if you disagree!

paolomaffei 16y ago
I see you're talking of A/B tests for BIG and medium sites then. If you only do let's say < 10.000 Unique Visits a month you're probably gonna do 1, 2, heck even 5 A/B tests, not more, so that wouldn't be a big time tax, would it? Also I'm not sure about other opensource CMSes but this one for wordpress you just have to copy the javascripts on a page and create the test on GWO.
- petekoomen 16y ago
  Great question. Even when it's just copying (and later removing) several javascript snippets on a page this can be a large hassle for several reasons: 1. For anyone who is non-technical, this can be daunting. Are the snippets in the right places? Was this deployed correctly? Have I set up my conversion goals correctly (GWO only allows for a single goal)? More often than not, tasks like this are passed from marketing onto IT, who are usually overworked as it is. 2. Even for technical folks, deploying and verifying a new version of one's site just to enable (and subsequently disable) a test can be a big hassle. In our experience, it's often the marketers, inside sales, and product managers in an organization who have the strongest interest in A/B testing. We're trying to eliminate the burden (and subsequent pain) that testing places on IT/engineering, freeing those stakeholders to test to their hearts' content! :)

paolomaffei 16y ago

I see you're talking of A/B tests for BIG and medium sites then. If you only do let's say < 10.000 Unique Visits a month you're probably gonna do 1, 2, heck even 5 A/B tests, not more, so that wouldn't be a big time tax, would it? Also I'm not sure about other opensource CMSes but this one for wordpress you just have to copy the javascripts on a page and create the test on GWO.

petekoomen 16y ago
Great question. Even when it's just copying (and later removing) several javascript snippets on a page this can be a large hassle for several reasons: 1. For anyone who is non-technical, this can be daunting. Are the snippets in the right places? Was this deployed correctly? Have I set up my conversion goals correctly (GWO only allows for a single goal)? More often than not, tasks like this are passed from marketing onto IT, who are usually overworked as it is. 2. Even for technical folks, deploying and verifying a new version of one's site just to enable (and subsequently disable) a test can be a big hassle. In our experience, it's often the marketers, inside sales, and product managers in an organization who have the strongest interest in A/B testing. We're trying to eliminate the burden (and subsequent pain) that testing places on IT/engineering, freeing those stakeholders to test to their hearts' content! :)