3 ms·

There was a Javascript split-test framework posted yesterday: http://news.ycombinator.com/item?id=1408637 http://news.ycombinator.com/item?id=1408637. So I'm wo

by edash 16y ago

There was a Javascript split-test framework posted yesterday: http://news.ycombinator.com/item?id=1408637 http://news.ycombinator.com/item?id=1408637. So I'm wondering (because I don't know): What are the advantages of using Rails vs Javascript to do a/b testing? And of course, advantages of either of these "custom" approaches over Google Website Optimizer?

patio11 16y ago
With regards to GWO, see http://www.bingocardcreator.com/abingo/compare http://www.bingocardcreator.com/abingo/compare . Google doesn't use GWO on any page of importance and neither should you. On the general topic of doing A/B testing server side (with Rails or what have you) versus doing it client side in Javascript: 1) Doing it in Javascript sacrifices one-line A/B tests, and that is a cost I'm totally unwilling to put up with because it means I will be too lazy to actually do A/B tests. (Take a look at the framework posted yesterday and what they have to do two swap two divs. That is fairly representative. Note that it gets harder, fast, if you start doing anything the least bit complicated.) 2) Javascript is not your friend for load times. A/Bingo and similarly optimized server-side frameworks are, essentially, free in terms of user-perceptible performance.
- dminor 16y ago
  I may have mentioned this before, but the multivariate version of GWO doesn't change your URL space and doesn't need to be keyed off of page views.
- lukestevens 16y ago
  "Google doesn't use GWO on any page of importance and neither should you." That's not true, Youtube used GWO to run a successful, massive multivariate experiment on its US front page, which qualifies as a page of importance in my book: http://youtube-global.blogspot.com/2009/08/look-inside-1024-recipe-multivariate.html http://youtube-global.blogspot.com/2009/08/look-inside-1024-...

by edash 16y ago

patio11 16y ago

With regards to GWO, see http://www.bingocardcreator.com/abingo/compare http://www.bingocardcreator.com/abingo/compare . Google doesn't use GWO on any page of importance and neither should you. On the general topic of doing A/B testing server side (with Rails or what have you) versus doing it client side in Javascript: 1) Doing it in Javascript sacrifices one-line A/B tests, and that is a cost I'm totally unwilling to put up with because it means I will be too lazy to actually do A/B tests. (Take a look at the framework posted yesterday and what they have to do two swap two divs. That is fairly representative. Note that it gets harder, fast, if you start doing anything the least bit complicated.) 2) Javascript is not your friend for load times. A/Bingo and similarly optimized server-side frameworks are, essentially, free in terms of user-perceptible performance.

dminor 16y ago
I may have mentioned this before, but the multivariate version of GWO doesn't change your URL space and doesn't need to be keyed off of page views.
lukestevens 16y ago
"Google doesn't use GWO on any page of importance and neither should you." That's not true, Youtube used GWO to run a successful, massive multivariate experiment on its US front page, which qualifies as a page of importance in my book: http://youtube-global.blogspot.com/2009/08/look-inside-1024-recipe-multivariate.html http://youtube-global.blogspot.com/2009/08/look-inside-1024-...