4 ms·

+1 for "let's run this big UI change using our TEST platform past some physical rooms full of typical users to see if they hate it" rather than just firing it i

+1 for "let's run this big UI change using our TEST platform past some physical rooms full of typical users to see if they hate it" rather than just firing it into prod with some buzzwords to justify traumatising millions of previously satisfied customers! Personal favourite was Spotify removing their playback widget to "improve the customer experience" and oh my God the backlash was ferocious. It was entirely moronic and screamed of Product Manager -> Customer with zero actual user feedback gathering: https://community.spotify.com/t5/Android/Android-Widget-Removal/td-p/4800301 https://community.spotify.com/t5/Android/Android-Widget-Remo... 26 pages of complaints produced this response, basically we removed a thing you love and use constantly, but you can vote for it as a feature request: "We always take retiring features in Spotify very seriously, and do want your continued feedback. The best way for us to measure your comments are via votes in our Idea Exchange. For this reason we're going to lock this thread. We encourage you to move to the Idea Exchange area where you can vote and comment on feature requests." Approaches and terms like "iterating fast", "responding to change", I dunno, just feels like a poor substitute for better planning and evaluation because those can be hard.

Silhouette 4y ago
The irony is that it often takes only a tiny number of test users to spot a glaring usability problem that could sink a major UI change. Someone (possibly the Nielsen Norman Group?) did some research on this a while ago and found that even single digits of test users would be enough to collect very useful feedback. People are so obsessed with scale and techniques like A/B testing today that they seem to forget that kind of test can only tell you whether one of the options is significantly better. It can't tell you whether you're finding the best choice between excellent options or the least of evils between dreadful ones.
- brianmcc 4y ago
  Totally. You could imagine even a tiny handful of trusted users who consistently give valuable, reliable, feedback. Microsoft seems a company just oblivious to user feedback - Beta users howling about disastrous Windows changes mooted, and they just.... ignore them. I don't get it.

Silhouette 4y ago

The irony is that it often takes only a tiny number of test users to spot a glaring usability problem that could sink a major UI change. Someone (possibly the Nielsen Norman Group?) did some research on this a while ago and found that even single digits of test users would be enough to collect very useful feedback. People are so obsessed with scale and techniques like A/B testing today that they seem to forget that kind of test can only tell you whether one of the options is significantly better. It can't tell you whether you're finding the best choice between excellent options or the least of evils between dreadful ones.

brianmcc 4y ago
Totally. You could imagine even a tiny handful of trusted users who consistently give valuable, reliable, feedback. Microsoft seems a company just oblivious to user feedback - Beta users howling about disastrous Windows changes mooted, and they just.... ignore them. I don't get it.