3 ms·
I think what you say is practically spot on. Yet I'd like to add that I don't think that testing frameworks (at this point it would be misleading to call them
by smueller1234 8y ago
I think what you say is practically spot on.
Yet I'd like to add that I don't think that testing frameworks (at this point it would be misleading to call them strictly A/B testing) HAVE to only reflect short term gains. It's hard to come up with (proxy) metrics that hold that kind of short term optimization in check.
Just to pick one fairly obvious example for e-commerce: you can track returns/customer service contacts as a health metric or even explicitly assign a value to them to fold them into the primary metric you're optimizing.
These kinds of safeguards typically need longer recording periods, so it's potentially a lot of data science and engineering effort to build tools that can handle the long term data collection and analysis. But it's not impossible. It's rather something people love to pretend is not a problem.