This methodology is flawed, because in practice the conversion rate changes over time. There are different effects that cause this temporal dependence not inclu

This methodology is flawed, because in practice the conversion rate changes over time. There are different effects that cause this temporal dependence not including the natural effect on the product of certain techniques (for example testing a very loud and painful to escape from upsell will cause some people to agree into the upsell and never see it again, and others to feel pissed off). Other causes of temporal dependence are different mix of traffic in terms of geography and demographics at different times of day and week. Even using proper Wilson confidence intervals with good methodology with tens of millions of impressions per group, we would see day to day variations in rate outside confidence intervals of the previous day, way more frequently than one would expect (one would expect a 95% confidence interval to be exceeded once every few weeks instead of every couple of days). The proper methodology is to estimate by bootstrapping on a good selection of dangerous variables, including time.