Check out Ron Kohavi's publications for a way to calculate this: http://robotics.stanford.edu/~ronnyk/ronnyk-bib.html http://robotics.stanford.edu/~ronnyk/ronny

Check out Ron Kohavi's publications for a way to calculate this: http://robotics.stanford.edu/~ronnyk/ronnyk-bib.html http://robotics.stanford.edu/~ronnyk/ronnyk-bib.html (He has a bunch of publications that are just the same thing repackaged, so it probably doesn't matter which one you read.) For example, if you have a conversion rate of 5% and want to detect an increase of 5% you need about 120'000 hits.