One of the better explanations of a/b testing I've read is that there are three types of uses on your site: those who will covert no matter what, those who will

One of the better explanations of a/b testing I've read is that there are three types of uses on your site: those who will covert no matter what, those who will not convert no matter what, and those who might convert. Testing is for that final group. Thinking in those terms, you have to consider if such a large percentage fits into that "might" group. I've never personally witnessed a 60% improvement at any volume I'd bother testing on and I've been doing tests for many years. I also think a lot of the "change the button" tests or "increased email sign up by x times" are dubious. Unless the conversion is someone giving you money, then you still have steps to go before you see real business improvement. There are lots of ways I can minipulate my traffic to make certain funnel steps look better and more optimized, but the only thing I really care about is what's coming out of the funnel. So all those extra email signups or button pushes mean nothing if the group who perform those actions on your b version still aren't interested in an actual purchase.