4 ms·

I've been using Optimizely, but after hearing from about VWO from COUNTLESS people I think I'm going to make the switch. Plus, bootstrappers should support oth

by bdunn 14y ago

I've been using Optimizely, but after hearing from about VWO from COUNTLESS people I think I'm going to make the switch. Plus, bootstrappers should support other bootstrappers, eh?

petercooper 14y ago
I'd be keen to hear the difference from someone who has used both in anger. I'm with VWO (because it gets mentioned around here so much - so I went there first) and it's awesome but it's always good to keep your options open.
- dsiroker 14y ago
  Congrats, Paras! As for a comparison, Optimizely uses Akamai which has 100,000 nodes in 75 countries. VWO uses Cloudfront and does two round-trips. Cloudfront has 32 points of presence in 12 countries. Optimizely response time: Slowest: 289ms, Average: 65ms, Slowest: 19ms VWO response time: Slowest 1983ms, Average: 276ms, 171ms Source: http://till.klampaeckel.de/blog/archives/100-Shopping-for-a-CDN.html http://till.klampaeckel.de/blog/archives/100-Shopping-for-a-... Unique features to Optimizely that VWO does not have: - Mobile web support & editor - Dynamic text insertion - Staging server support - Human-readable results - Documented API - Advanced developer tools - Cross-browser testing for Chrome, Safari, Firefox, IE, iOS, and Android - One-click integrations with Google Analytics, Omniture SiteCatalyst, KISSmetrics, MixPanel - Implementation is 1-line of code vs 21 lines of code - Pricing starts at $19/mon vs $49/mon - Optimizely comes with a 100% money back satisfaction guarantee with 99.999% service level agreement - Optimizely support offers web, phone, email, and a dedicated account manager, as well as the CEO's direct cell for Platinum customers In terms of adoption by the top 10,000 websites on the internet Optimizely is 6x bigger than VWO. Sources: http://trends.builtwith.com/analytics/Optimizely http://trends.builtwith.com/analytics/Optimizely http://trends.builtwith.com/analytics/Visual-Website-Optimizer http://trends.builtwith.com/analytics/Visual-Website-Optimiz... Note: I co-founded Optimizely and used Google Website Optimizer, Omniture Test&Target, and VWO but by no means an expert in their products.
  - paraschopra 14y ago
    Thanks Dan. Appreciate you pitching in. However, actually, you have old statistics. We recently rolled out a dynamic, custom built CDN that sped up our response times by 7x. Our average is now 45ms. We spent a lot of time perfecting our system. See the details here: http://visualwebsiteoptimizer.com/split-testing-blog/geo-distributed-architecture/ http://visualwebsiteoptimizer.com/split-testing-blog/geo-dis... Plus, you are wrong about almost all of your points: - We do have a well documented API. Many customers use it on a daily basis to integrate VWO into their systems. http://visualwebsiteoptimizer.com/split-testing-blog/announcing-visual-website-optimizer-api/ http://visualwebsiteoptimizer.com/split-testing-blog/announc... - Our 21 lines of code is unique as it provides an asynchronous fail safe method. With 1-line of code, websites still have a theoretical single point a failure where code is fetched from your servers/CDN. If your code isn't able to get fetched for any reason, standard timeout of browser (30 seconds) applies and since it is synchronous page load stalls. With our asynchronous approach, we completely decouple the dependence (even if it doesn't look elegant, it certainly is much preferable to our customers). Here are the details: http://visualwebsiteoptimizer.com/split-testing-blog/asynchronous-code/ http://visualwebsiteoptimizer.com/split-testing-blog/asynchr... - I'm not sure what you mean by Human-readable results. Certainly we have some great looking reports that our customers love. 50+ case studies on our websites is a testimony to that http://visualwebsiteoptimizer.com/case-studies.php http://visualwebsiteoptimizer.com/case-studies.php In fact, one independent website that tracks UI from color blindness perspective says we have one of the best charts out there http://wearecolorblind.com/example/visual-website-optimizer/ http://wearecolorblind.com/example/visual-website-optimizer/ - We offer developer CSS/JavaScript API that allows complete modification of a website without touching any backend code - Our tests run on mobiles as well, a lot of our enterprise customers regularly test their mobile landing pages - We have one-click integration for Google Analytics, and have plugins for KISSMetrics, SiteCatalyst, etc. - ALL employees of Wingify (including me the CEO and our CTO) do support for all levels of customers (not just enterprise ones). In fact, support is bed rock of our offering. My number is there on our about us page http://visualwebsiteoptimizer.com/about-us.php http://visualwebsiteoptimizer.com/about-us.php According to best of my knowledge, apart from cross-browser previews I do not think there is any feature that VWO lacks. You may want to cross-check your facts. Our backend delivery infrastructure prioritizes on speed and fall-back rather than elegance of how the code looks. However, you did not mention the features VWO has that Optimizely currently lacks: - Usability testing. Our users can get quantitative feedback and improvement ideas on what they should be A/B testing http://visualwebsiteoptimizer.com/split-testing-blog/geo-behavioral-targeting-usability-testing/ http://visualwebsiteoptimizer.com/split-testing-blog/geo-beh... - Heatmaps and Clickmaps to show where people are clicking on their pages. Our customers LOVE This feature and some solely use VWO for its heatmap functionality http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-clickmaps-awesomeness/ http://visualwebsiteoptimizer.com/split-testing-blog/ab-test... - Enterprise security options such as whitelisting office IPs for login and realtime activity notifications to administrator http://visualwebsiteoptimizer.com/split-testing-blog/halloween-gifts-announcements/ http://visualwebsiteoptimizer.com/split-testing-blog/hallowe... - Free 30 day trial without requiring a credit card. - Numerous other small features that I won't go into (test labels, test organization, frequently used goals, roles/permissions, account management, reports exportable to CSV/PDF, etc.)

by bdunn 14y ago

I've been using Optimizely, but after hearing from about VWO from COUNTLESS people I think I'm going to make the switch. Plus, bootstrappers should support other bootstrappers, eh?

petercooper 14y ago

I'd be keen to hear the difference from someone who has used both in anger. I'm with VWO (because it gets mentioned around here so much - so I went there first) and it's awesome but it's always good to keep your options open.

dsiroker 14y ago
Congrats, Paras! As for a comparison, Optimizely uses Akamai which has 100,000 nodes in 75 countries. VWO uses Cloudfront and does two round-trips. Cloudfront has 32 points of presence in 12 countries. Optimizely response time: Slowest: 289ms, Average: 65ms, Slowest: 19ms VWO response time: Slowest 1983ms, Average: 276ms, 171ms Source: http://till.klampaeckel.de/blog/archives/100-Shopping-for-a-CDN.html http://till.klampaeckel.de/blog/archives/100-Shopping-for-a-... Unique features to Optimizely that VWO does not have: - Mobile web support & editor - Dynamic text insertion - Staging server support - Human-readable results - Documented API - Advanced developer tools - Cross-browser testing for Chrome, Safari, Firefox, IE, iOS, and Android - One-click integrations with Google Analytics, Omniture SiteCatalyst, KISSmetrics, MixPanel - Implementation is 1-line of code vs 21 lines of code - Pricing starts at $19/mon vs $49/mon - Optimizely comes with a 100% money back satisfaction guarantee with 99.999% service level agreement - Optimizely support offers web, phone, email, and a dedicated account manager, as well as the CEO's direct cell for Platinum customers In terms of adoption by the top 10,000 websites on the internet Optimizely is 6x bigger than VWO. Sources: http://trends.builtwith.com/analytics/Optimizely http://trends.builtwith.com/analytics/Optimizely http://trends.builtwith.com/analytics/Visual-Website-Optimizer http://trends.builtwith.com/analytics/Visual-Website-Optimiz... Note: I co-founded Optimizely and used Google Website Optimizer, Omniture Test&Target, and VWO but by no means an expert in their products.
- paraschopra 14y ago
  Thanks Dan. Appreciate you pitching in. However, actually, you have old statistics. We recently rolled out a dynamic, custom built CDN that sped up our response times by 7x. Our average is now 45ms. We spent a lot of time perfecting our system. See the details here: http://visualwebsiteoptimizer.com/split-testing-blog/geo-distributed-architecture/ http://visualwebsiteoptimizer.com/split-testing-blog/geo-dis... Plus, you are wrong about almost all of your points: - We do have a well documented API. Many customers use it on a daily basis to integrate VWO into their systems. http://visualwebsiteoptimizer.com/split-testing-blog/announcing-visual-website-optimizer-api/ http://visualwebsiteoptimizer.com/split-testing-blog/announc... - Our 21 lines of code is unique as it provides an asynchronous fail safe method. With 1-line of code, websites still have a theoretical single point a failure where code is fetched from your servers/CDN. If your code isn't able to get fetched for any reason, standard timeout of browser (30 seconds) applies and since it is synchronous page load stalls. With our asynchronous approach, we completely decouple the dependence (even if it doesn't look elegant, it certainly is much preferable to our customers). Here are the details: http://visualwebsiteoptimizer.com/split-testing-blog/asynchronous-code/ http://visualwebsiteoptimizer.com/split-testing-blog/asynchr... - I'm not sure what you mean by Human-readable results. Certainly we have some great looking reports that our customers love. 50+ case studies on our websites is a testimony to that http://visualwebsiteoptimizer.com/case-studies.php http://visualwebsiteoptimizer.com/case-studies.php In fact, one independent website that tracks UI from color blindness perspective says we have one of the best charts out there http://wearecolorblind.com/example/visual-website-optimizer/ http://wearecolorblind.com/example/visual-website-optimizer/ - We offer developer CSS/JavaScript API that allows complete modification of a website without touching any backend code - Our tests run on mobiles as well, a lot of our enterprise customers regularly test their mobile landing pages - We have one-click integration for Google Analytics, and have plugins for KISSMetrics, SiteCatalyst, etc. - ALL employees of Wingify (including me the CEO and our CTO) do support for all levels of customers (not just enterprise ones). In fact, support is bed rock of our offering. My number is there on our about us page http://visualwebsiteoptimizer.com/about-us.php http://visualwebsiteoptimizer.com/about-us.php According to best of my knowledge, apart from cross-browser previews I do not think there is any feature that VWO lacks. You may want to cross-check your facts. Our backend delivery infrastructure prioritizes on speed and fall-back rather than elegance of how the code looks. However, you did not mention the features VWO has that Optimizely currently lacks: - Usability testing. Our users can get quantitative feedback and improvement ideas on what they should be A/B testing http://visualwebsiteoptimizer.com/split-testing-blog/geo-behavioral-targeting-usability-testing/ http://visualwebsiteoptimizer.com/split-testing-blog/geo-beh... - Heatmaps and Clickmaps to show where people are clicking on their pages. Our customers LOVE This feature and some solely use VWO for its heatmap functionality http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-clickmaps-awesomeness/ http://visualwebsiteoptimizer.com/split-testing-blog/ab-test... - Enterprise security options such as whitelisting office IPs for login and realtime activity notifications to administrator http://visualwebsiteoptimizer.com/split-testing-blog/halloween-gifts-announcements/ http://visualwebsiteoptimizer.com/split-testing-blog/hallowe... - Free 30 day trial without requiring a credit card. - Numerous other small features that I won't go into (test labels, test organization, frequently used goals, roles/permissions, account management, reports exportable to CSV/PDF, etc.)