Sample size was 4500+ distributed between the two versions. In this case, the sample size was not small but the conversion rate of the page was very low. Which

Sample size was 4500+ distributed between the two versions. In this case, the sample size was not small but the conversion rate of the page was very low. Which is why a 600% increase was even possible. That's usually a problem with A/B testing in the real world. There are so many websites whose conversion rates hover in the 0.xx% range. Thoughtful hypotheses almost always result in significant gains. Not because they did something magical, but because they simply communicated their business offer better.