The way A/B testing is practiced by actual people in the actual world, it is fundamentally broken. No one EVER tests for mean-reversion over time. 17 years I'

The way A/B testing is practiced by actual people in the actual world, it is fundamentally broken. No one EVER tests for mean-reversion over time. 17 years I've seen companies do A/B tests. I doubt I've seen a single convincing, durable result the whole time.