This is a huge problem with paid marketing as well. Many folks will look at a conversion rate completely ignorant of sample size and allocate thousands of dolla

This is a huge problem with paid marketing as well. Many folks will look at a conversion rate completely ignorant of sample size and allocate thousands of dollars in budget to something which they have no idea performs better or worse. The real problem (as you allude to in the article) is that the demand for accurate tools is not really there. Vendors don't build in accurate stats because only a tiny portion of their client base understand/demands them.