3 ms·
I would say this is the fate of all products where the decision makers are focused on short term metric gains instead of long term quality. Data driven decision
by jvans 3y ago
I would say this is the fate of all products where the decision makers are focused on short term metric gains instead of long term quality. Data driven decision making encourages myopic product vision in a lot of scenarios.
Good data organizations acknowledge that all metrics can be manipulated and take changes in metrics with a grain of salt. They combine a/b testing with first principles thinking and make imperfect decisions based on imperfect information. They don't always get it right, but they embrace uncertainty and critically evaluate outcomes both "good" and "bad", because they know a/b test metrics don't tell the full story.
Bad data organizations focus on the a/b test metrics and ignore common sense logic, because short term metric wins are rewarded. Rinse and repeat for years on end and the product evolves to puzzling place like you mention. It was all the result of someone winning an a/b test over and over, and not critically thinking about the long term.
Also see: expedia, quora, facebook, instagram, etc.