From the article: "You can perform power calculations by using an online calculator or a statistical package like R. If time is short, a simple rule of thumb i

From the article: "You can perform power calculations by using an online calculator or a statistical package like R. If time is short, a simple rule of thumb is to use 6000 conversion events in each group if you want to detect a 5% uplift. Use 1600 if you only want to detect uplifts of over 10%. These numbers will give you around 80% power to detect a true effect." These numbers assume approximately 10% of changes give true uplift (aka actual improvement). If the incidence of improvement is rarer you would need more than 6000 events to identify a 5% increase. (OT: that term sounds like cult language -- you have three chances to achieve true uplift)