I think the article's claim is only valid if you expect future sales to be uncorrelated with past sales. Suppose that most of your revenue comes from recurring

I think the article's claim is only valid if you expect future sales to be uncorrelated with past sales. Suppose that most of your revenue comes from recurring subscriptions of varying prices. Then, to use an extreme example for clarity, suppose that last month's revenue per person for the control and treatment groups respectively was not $95 and $101 as in the article, but $10 and $100. I guess you just got really, really unlucky with your random selection, such that most of your high-paying users landed in the treatment group. Then you wait a month and measure again, and it turns out this month's revenue per person was exactly the same in each group, $10 and $100. By this article's logic, you should ignore the past values and only look at this month's $10 and $100, and conclude that the treatment made revenue 900% higher. In reality, the results probably just mean that most people didn't change their subscription level from month to month. Because individual users' spending on the subscription is highly correlated from month to month, you should measure this month's spending compared to last month's baseline.