The paper's intro mentions some of the issues. The biggest one IMO is a Simpson's Paradox issue that arises when (1) the difference between A and B is nonstatio

The paper's intro mentions some of the issues. The biggest one IMO is a Simpson's Paradox issue that arises when (1) the difference between A and B is nonstationary, (2) the probability of being assigned to treatment or control is random at any given instant, but varies over time, being manipulated by dynamic optimization algorithms attempting to maximize signal / profit / etc. Aside - this paper isn't the first time this issue has been noticed, or even the first time it's been addressed with rigorous statistics. Optimizely did it back in 2017-2018: https://support.optimizely.com/hc/en-us/articles/5326213705101-Stats-Accelerator-Acceleration-Under-Time-Varying-Signals https://support.optimizely.com/hc/en-us/articles/53262137051... It would be nice if these authors mentioned the prior art and explained how their work improves upon it.