3 ms·
The "metrics are normal but users complain" example is something I've experienced repeatedly as a user. It's incredibly frustrating and leaves a really bad impr
by tgsovlerkhgsel 6y ago
The "metrics are normal but users complain" example is something I've experienced repeatedly as a user. It's incredibly frustrating and leaves a really bad impression when an app/site is utterly, obviously, reproducably broken and stays in that state for a week.
The affected users don't care that it works for others, and you can only afford to ignore it if you are OK with not only losing those users forever, but also the negative reviews and word of mouth.
The probably biggest (in terms of company size/affected users) example of this would be Twitter's "something went wrong" on mobile. I'm sure that they have some sort of metrics showing them that everything is fine, but plenty of users on HN complain that they are pretty consistently getting a generic error message when they open a Twitter link until they reload the whole page. (I assume it's separate from the issue mentioned in the article, since it was still ongoing recently.)
Also, because errors are much more noticeable than everything working, a 10% error rate is pretty much perceived as "never works".
A subtopic of that: The affected users don't care why it is broken, and it is broken until it is fixed for them. If you fix the issue, but the fixed version hasn't yet made it to your users' phones (e.g. because it's only fixed in the development version, or your version hasn't made it through some review process), then your app is still broken. (Looking at you, Newpipe and Firefox: Newpipe was broken for weeks, apparently because F-Droid, their official distribution channel, is shipping an outdated version, and when trying to fix that by downloading the newest APK, I ran into a bug in Firefox Mobile that prevents downloads from working - which is fixed in some developer version but not the public one, and has been broken for at least a week).
- k__ 6y agoI once read, a huge problem of data driven approaches is that people tend to measure what is easy to measure and not what should be measured.