The author discusses "pull forward", where the real impact of a change is to make people purchase earlier, but we don't necessarily observe incremental purchase

The author discusses "pull forward", where the real impact of a change is to make people purchase earlier, but we don't necessarily observe incremental purchases. This isn't necessarily bad; I'd rather have a dollar today than a dollar next week. This can be quantified by plotting the incremental conversions observed by day x. We migh see a big initial lift that degrades over time. If it eventually degrades to zero, there are no truly incremental conversions, just pull-forward. But if we end up pulling forward a meaningful number of purchases by a month or more, that can be valuable to the business! I wouldn't immediately jump to a complicated mathematical model to handle this situation, I would consider the business implications first and foremost. I also urge anyone considering Bayesian methods for A/B testing to read up on the likelihood principle vs the strong repeated sampling principle (I documented my thoughts here [0]). Bayesian methods always satisfy the likelihood principle; frequentist methods always satisfy repeated sampling. In many situations both methods satisfy both principles, and then the two approaches will give similar answers. But based on many years doing A/B testing, I wouldn't give up repeated sampling lightly. Bayesian and frequentist methods are not blindly interchangeable. On the other hand, if repeated sampling is not important in your use case, then by all means prefer the Bayesian approach! I just want people to consider the trade offs. [0]: https://adventuresinwhy.com/post/bayesian_ab_testing/ https://adventuresinwhy.com/post/bayesian_ab_testing/