4 ms·

I understand the frustration of the developers affected by this UI change, but I think that actively asking users to rate an app they have downloaded may lead t

I understand the frustration of the developers affected by this UI change, but I think that actively asking users to rate an app they have downloaded may lead to better ratings in a statistical sense. User ratings greatly suffer from self selection bias (http://en.wikipedia.org/wiki/Self-selection_bias http://en.wikipedia.org/wiki/Self-selection_bias). Asking users who didn't initially rate an app do so as part of some other workflow might reduce that bias a little bit. That said, I'm very distrustful of user ratings in general, both as a developer and as a consumer, because people often rate stuff based on criteria that don't concern me at all.

magicalist 13y ago
It would also help if the graphs in the post started at 0 stars on the y-axis, because the drops look much more severe than they really are. I imagine in an ultra-competitive niche that the drop in the third graph from 4.72 to 4.54 could be a painful drop, but the first graph has a drop of .03 stars, and the second is a drop of .05 stars. Perfectly in line with a very slight adjustment (though I'm sure it's alarming when you first notice it happening). To be fair, the graph formatting doesn't look like the author's fault, just the analytics page automatically zooms and crops like that.
- davedx 13y ago
  Can you rate something 0 stars?
quarterto 13y ago
actively asking users to rate an app they have downloaded may lead to better ratings But it's not asking them to rate the app, it's asking them to rate the suggestion. And since rating a suggestion is usually a thumbs-up/thumbs-down, it primes users to select the extreme ratings.
- fauigerzigerk 13y ago
  >But it's not asking them to rate the app, it's asking them to rate the suggestion. No, if this is really the UI in question then it is not asking users to rate the suggestion: http://blog.testfairy.com/wp-content/uploads/2014/01/quick_suggest.png http://blog.testfairy.com/wp-content/uploads/2014/01/quick_s... It's clearly asking users to rate one app to get suggestions for other apps.
  - zango 13y ago
    Sadly, many users just don't read past the title. And users familiar with other suggestion engines (pandora, etc) expect to teach the system their taste and relevance.

magicalist 13y ago

It would also help if the graphs in the post started at 0 stars on the y-axis, because the drops look much more severe than they really are. I imagine in an ultra-competitive niche that the drop in the third graph from 4.72 to 4.54 could be a painful drop, but the first graph has a drop of .03 stars, and the second is a drop of .05 stars. Perfectly in line with a very slight adjustment (though I'm sure it's alarming when you first notice it happening). To be fair, the graph formatting doesn't look like the author's fault, just the analytics page automatically zooms and crops like that.

davedx 13y ago
Can you rate something 0 stars?

quarterto 13y ago

actively asking users to rate an app they have downloaded may lead to better ratings But it's not asking them to rate the app, it's asking them to rate the suggestion. And since rating a suggestion is usually a thumbs-up/thumbs-down, it primes users to select the extreme ratings.

fauigerzigerk 13y ago
>But it's not asking them to rate the app, it's asking them to rate the suggestion. No, if this is really the UI in question then it is not asking users to rate the suggestion: http://blog.testfairy.com/wp-content/uploads/2014/01/quick_suggest.png http://blog.testfairy.com/wp-content/uploads/2014/01/quick_s... It's clearly asking users to rate one app to get suggestions for other apps.
- zango 13y ago
  Sadly, many users just don't read past the title. And users familiar with other suggestion engines (pandora, etc) expect to teach the system their taste and relevance.