The blog reads don't do A/B tests for "small changes". On the contrary I actually think A/B tests over large number of users work great for small changes than u

The blog reads don't do A/B tests for "small changes". On the contrary I actually think A/B tests over large number of users work great for small changes than user testing with a small set. Many users cannot articulate the benefit of the small change, eye tracking or getting mental map of what the user thinks is very hard, with small set of users concluding something might be so skewed a decision. Additionally the blog is misguiding >> After 11,000 tests and 400 conversions it clearly showed that the instructions made a 30% difference. Below are the actual A/B test numbers from the blog: Original: 193/5943 => 3.24% Combination 1: 236/6058 => 3.89% 1. quoting this as 30% improvement is wrong. Google only said if you use combination 1 you would see 20% higher leads. 2. Since the difference between the conversion numbers is not very high one has to run it for a very long time and even on the first instance combination1 was no better than original. Given 1 & 2 Google did mark the results as inconclusive telling us not to be fooled. Edit: rephrased