I've lived in the data-driven mindset for many years, and ultimately found that it still consists of mostly guesswork, opinions and strong biases everywhere. Pe

I've lived in the data-driven mindset for many years, and ultimately found that it still consists of mostly guesswork, opinions and strong biases everywhere. People make product decisions and they will seek the data to validate what they already want [1] anyway. It should complement, not replace talking to users, and is not a requirement to build great products. When it comes to simple anonymized telemetry, one user or subset of users, spamming the same error over and over will skew your data, and unless you track them even more you won't be able to tell the difference. As I said earlier, it's a slippery slope to stand on. Comprehensive telemetry/tracking is fine for say, an e-commerce website or any kind of cloud application where you're already receiving all the user input anyway. It's also fine, with consent, for desktop apps, especially complex ones like IDEs etc. It is not OK for third-party software packages that you'll ship to your own users - regardless of any promise to not add tracking to the resulting build, since at this point they already broke that wall and it only takes one "well-intentioned" PM to make it happen. [1] a system that evolves automatically by running stochastic A/B tests is something many have dreamed up including myself. I'm really curious what that would end up looking like. That's being truly data-driven!