There are a lot of published "case studies" in the internet marketing field that consist of a few hundred views and a handful of conversions. It is even more em

There are a lot of published "case studies" in the internet marketing field that consist of a few hundred views and a handful of conversions. It is even more embarrassing considering you often need 100,000+ unique visitors and thousands of conversions to find real winners.. and you still have to deal with a reality that a real 'winner' may result in a drop off of sales (in lead generation), an increase in charge backs (if your conversion was a sale), etc. This accounts for a sliver of the regression to the mean mentioned in the whitepaper. Tests have value, but just making your site/app very simple and completely non-confusing to the viewer can do something years of split tests will not. I suggest running tests and monitoring metrics as you implement design changes, not so much as a magic eight ball, but to ensure you avoid truly catastrophic UI fuck ups.