This grain of salt article about A/B testing is smart. Specifically, this is something that I learned too: "Do user testing. Get people in and ask them to use

This grain of salt article about A/B testing is smart. Specifically, this is something that I learned too: "Do user testing. Get people in and ask them to use your product. You’re going to get a lot more information a lot faster and have a higher degree of confidence in the results." Expanding on this, I think that it's important not just to rally a group of your friends or other webheads. The closer that you can get to assembling a group of target users, the better. On ecommerce sites with a checkout, one of my favorite tricks is to send a manual e-mail to a handful of people who have become customers. Clear and simple, I say that I'd like to talk to them about the checkout process, mention that there will be 3-5 other customers on the conference call and offer them a significant coupon (25% +free shipping or something) to participate. During the call, I talk to them about their shopping experience and show them a revision or two of different designs. I ask them to (fake) buy a product using the revisions. The feedback (especially with early designs) is consistently some of the most valuable I receive and it's surprising how much of a positive effect talking to people has. With one phone call asking for feedback, you often gain a repeat customer.