There are many more papers and presentations on this topic at Ron Kohavi's MS group's home page, http://www.exp-platform.com/Pages/default.aspx http://www.exp-p

There are many more papers and presentations on this topic at Ron Kohavi's MS group's home page, http://www.exp-platform.com/Pages/default.aspx http://www.exp-platform.com/Pages/default.aspx. Most of these papers are worth a read, as they dissect how and where online experimentation can really muck things up if not handled properly. Some are more statistical than others, but most all are relatively easy reads.