> So, I would venture that this idea is a bad one, but sounds good on paper. It's a great idea, it's just vulnerable to non-stationary effects (novelty effect,

> So, I would venture that this idea is a bad one, but sounds good on paper. It's a great idea, it's just vulnerable to non-stationary effects (novelty effect, seasonality, etc). But it's actually no worse than fixed time horizon testing for your example if you run the test less than a year. You A/B test that copy for a month, push everyone to A, and you're still not going to realize it's actually worse.