Agreed completely. To put it more simply: don't use AB testing to actually create your site by throwing mud on the wall. First design a good site, and then use

Agreed completely. To put it more simply: don't use AB testing to actually create your site by throwing mud on the wall. First design a good site, and then use AB testing to slowly tweak it to the closest best choices. Just to add: unless you are testing google.com or some other top 10 site, #2 is not going to work well as each bucket will be too small to complete the AB test properly. eg. 3/100 users in bucket 92 converting is no proof that it is better than bucket 47 with 2/100 users converting.