8 ms·

Google Play Store Ratings Drop

103 points by radley 13y ago

simcop2387 13y ago
Cache since it's down right now: http://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Fblog.testfairy.com%2Fgoogle-play-store-ratings-drop%2F http://webcache.googleusercontent.com/search?q=cache%3Ahttp%...
- tgasson 13y ago
  It also had an update post http://blog.testfairy.com/google-play-store-ratings-drop-update http://blog.testfairy.com/google-play-store-ratings-drop-upd... http://webcache.googleusercontent.com/search?q=cache:z4GGDYq_4qUJ:blog.testfairy.com/google-play-store-ratings-drop-update/+&cd=2&hl=en&ct=clnk http://webcache.googleusercontent.com/search?q=cache:z4GGDYq...
kaeawc 13y ago
Has this been fixed? The post is dated Jan 1st.
coin 13y ago
"Error establishing a database connection"
mtgx 13y ago
I can definitely see how trying to touch the icon could've hit the 1/2 star rating instead.
Pxtl 13y ago
I don't know if google is evil, but I'm pretty sure they're kinda stupid. Seriously, how many UI whoopsies are we going to hear about this week?
dredmorbius 13y ago
That's a nice bit of legwork to show that poor UI design translates directly into erroneous user interactions. In conjunction with other recent Google stories (https://news.ycombinator.com/item?id=7107325 https://news.ycombinator.com/item?id=7107325) it's both cautionary, largely exculpatory of users (Google caused this problem), and speaks more than slightly to Google's hubris (I've seen plenty of poor UI from them over the past several years). An interesting aspect of this dynamic is that the object and subject of the error are asymmetric in this case. It's one thing to accidentally out yourself through "not reading instructions" or "just winging it" or "accidentally hitting the wrong control". It's another to accidentally take an action that hurts another person -- here in a significant way as it directly affects their reputation. Hacker News is of course hardly blameless. I've more than once downvoted a comment or flagged or upvoted a story, especially on mobile, trying to access other controls on the site. At least the story flagging can be undone, though the other actions cannot be. But that's only if I care enough to do so.
- banachtarski 13y ago
  Google design sucks. They've never in the history of the company made a solid effort to nail UI and it's not part of their culture at all. Every one of their products is marred by UI inefficiencies, clutter, or inconsistency.
  - jmduke 13y ago
    * They've never in the history of the company made a solid effort to nail UI and it's not part of their culture at all.* Regardless of how you feel about how it turned out, they clearly have spent the past two years focusing on unifying design and UX across their products.
    - dredmorbius 13y ago
      One of the worst sins of UI is "unification". It's helpful to have meaningfully consistent metaphors and themes across applications. For the UNIX / Linux commandline, that's processes and pipes, BSD and GNU argument flags, and the like. There are various competing keyboard accelerator schemes: emacs, vi, Wordperfect keyboard templates, DOS, Windows 3.x, Windows 9x/NT, Apple's Mac hotkeys, GNOME and KDE.... There are fundamental layout, font, and color palette schemes. But one of the worst overall design features I see is taking a given scheme and applying it without regard to suitability across a wide range of applications with a wide range of use cases. For Google this results in the insanity of persistent browser elements occupying 25-30% of vertical screen real estate in Gmail, Groups, Book search, and other products. When the workaround is for me to either push my browser window half-height above my viewable monitor frame and drag down the bottom of the window, to be able to present my work front and center, or to open the elements inspector and delete elements from the DOM just to be able to see the relevant work, something's very, very wrong. Microsoft of course was infected with similar idiocy with its modal, non-resizeable dialogs and wizards. Even GNOME, for all its UI/UX brain death, generally provides for resizable dialogs, and allows text to be copied from its error pop-ups (under Windows the only recourse was and to the best of my knowledge remains screenshotting the damned thing). But ... Google and UI/UX? Not so much.
daphneokeefe 13y ago
Site is down already.
product50 13y ago
The number of times I have seen the database connection error for an obscure site which appears on Hacker News is alarming. It is not like millions of users are trying to get to their site that they go down. If you can't host it don't - why can't sites just use something like Posthaven or something - to ensure more reliability.
ajjai 13y ago
I see similar behaviour on my app.. Many 1 star ratings without any comments.. Initially I was convinced it must be competitor playing games.. But the trend continued.. Not sure how long Google will take to address this issue..
abcd_f 13y ago
So here's the UI in question - http://blog.testfairy.com/wp-content/uploads/2014/01/quick_suggest.png http://blog.testfairy.com/wp-content/uploads/2014/01/quick_s... - and apparently the large stars represent not a suggestion rating, but an app rating. So dismissing suggestion as a bad one you'd basically give a 1 star rating to an app. Or so the devs claim. Coming from the iOS side, I just don't get it. How can you rate an app that you haven't downloaded? And then if you did download it, why is it being quick-suggested to you again? Furthermore, if you did in fact download it, removed it and then reacted negatively to a suggestion to download it again - how's your low-star reaction is NOT really a rating for an app itself? Also, if you look closely at the graphs, the "crashing" involves rating slide from 4.67 to 4.38 and from 4.79 to 4.74. It's still an unpleasant development for sure, but that's not a "crash" or a "drop", that's a dip. The matter appears to be completely blown out of its proportions.
- trurl42 13y ago
  It's a suggestion to rate the shown app, to get better app recommendations. It only shows apps you have downloaded.
- ZeroGravitas 13y ago
  Also, if the dip affects everyone equally, then its more like a rescaling (presumably on average these reviews are lower than before, I'd guess because no-one bothers to review a 2-3 star item, only if they love it or really hate it)
  - Kudos 13y ago
    Not until the users view of those ratings rescale too.
  - igordcard 13y ago
    If the dip affected everyone equally it wouldn't be that bad, as most of the saturation found above the 4.8 would be leveraged... However these suggestions will be rated, in average, as greater as the overall empathy and knowledge users have regarding the kind of app in question. For instance, regarding my app Tabata Trainer (which went from solid 4.57 in December 9th to 4.21 at the time of this post), how many persons identify with the word "Tabata"? Most of them will never give a 5. If someone creates a game with a very similar name to a widely known and adorable game, then people will probably rate the suggestion highly. This kind of stuff will just lead apps to be rated unequally... and bad (really bad) apps will raise their ratings, which is shameful.
blueskin_ 13y ago
Google are crap at UI design. It's all Metro, and then they do things like this.
pilif 13y ago
I might be an overly conservative old fart, but there was a time not so long ago, where your success wasn't depending on the UI decision (or any other decision) of a corporate overlord. Back then, you made a product and it was immediately available for all the world to use. Yes, you had to spend some time/money/effort for promoting it and yes, the success might have been dependent on your ability to promote it. But you didn't have to ask for permission to build it and there was no way to get screwed because the UI design of an entity you have no control over has changed slightly from one day to another. I think it's a real shame that these days seem to be over more and more.
- andybak 13y ago
  > Yes, you had to spend some time/money/effort for promoting it and yes, the success might have been dependent on your ability to promote it. And you're back in exactly the same situation. Where are you going to promote it and who else has control over that communication channel?
mixedbit 13y ago
When Google started to require users to use G+ and real names to post a review on the Play store, a number of reviews dropped significantly. The poor design decision described in the article may be a result of an effort to fix this dropped user engagement rate.
fauigerzigerk 13y ago
I understand the frustration of the developers affected by this UI change, but I think that actively asking users to rate an app they have downloaded may lead to better ratings in a statistical sense. User ratings greatly suffer from self selection bias (http://en.wikipedia.org/wiki/Self-selection_bias http://en.wikipedia.org/wiki/Self-selection_bias). Asking users who didn't initially rate an app do so as part of some other workflow might reduce that bias a little bit. That said, I'm very distrustful of user ratings in general, both as a developer and as a consumer, because people often rate stuff based on criteria that don't concern me at all.
- magicalist 13y ago
  It would also help if the graphs in the post started at 0 stars on the y-axis, because the drops look much more severe than they really are. I imagine in an ultra-competitive niche that the drop in the third graph from 4.72 to 4.54 could be a painful drop, but the first graph has a drop of .03 stars, and the second is a drop of .05 stars. Perfectly in line with a very slight adjustment (though I'm sure it's alarming when you first notice it happening). To be fair, the graph formatting doesn't look like the author's fault, just the analytics page automatically zooms and crops like that.
  - davedx 13y ago
    Can you rate something 0 stars?
- quarterto 13y ago
  actively asking users to rate an app they have downloaded may lead to better ratings But it's not asking them to rate the app, it's asking them to rate the suggestion. And since rating a suggestion is usually a thumbs-up/thumbs-down, it primes users to select the extreme ratings.
  - fauigerzigerk 13y ago
    >But it's not asking them to rate the app, it's asking them to rate the suggestion. No, if this is really the UI in question then it is not asking users to rate the suggestion: http://blog.testfairy.com/wp-content/uploads/2014/01/quick_suggest.png http://blog.testfairy.com/wp-content/uploads/2014/01/quick_s... It's clearly asking users to rate one app to get suggestions for other apps.
moondowner 13y ago
Maybe it's related to this new awful app review trend? https://news.ycombinator.com/item?id=7044833 https://news.ycombinator.com/item?id=7044833
- magicalist 13y ago
  They're at least referring to the same phenomenon. The followup article to this one that tgasson posted[1] quotes and links to that very HN thread. [1] https://news.ycombinator.com/item?id=7113614 https://news.ycombinator.com/item?id=7113614
hannahmitt 13y ago
Interesting. I'd wondered about an inflection point in my graph as well, but in the opposite direction. Our average rating has been climbing much faster since Dec 10. Looks like this feature is influential, but not necessarily negative.
k00pa 13y ago
I added a dialog that asks the user rate the app, remind later or just ignore it. The trick is to add the dialog to show up on like 5-10 startup off the app. If the user is starting the application after 5 tries, he is going to like it. And provided the way to rate it in couple of clicks, the user usually gives positive ratings.
mschuster91 13y ago
I think legislation should be made (maybe anti-trust laws?) which requires companies to provide adequate, free 1st-level user support for paid services, and reasonably priced user support for free services. The current way of big companies "just doing stuff" which deeply affects other people/businesses who do not have a single way to contact the offender is a nightmare which has to stop. As the market obviously fails here, legislation is required.
xeroxmalf 13y ago
Or the other new problem[0]: "Actually a great app, but I left 1 star so my review would be seen first!" [0] https://news.ycombinator.com/item?id=7044833 https://news.ycombinator.com/item?id=7044833
PaulHoule 13y ago
Google Play is a joke. Maybe it's better if you have a different version of Android but when I search for anything broad I see a bunch of free shovelware apps. I'd like to have an option to see just the paid apps or the most expensive apps just to clear out the noise and the spam.
fharper1961 13y ago
I'm Android dev, and I think the "Want quick suggestions?" feature is a good addition to Google Play, because it encourages users to rate apps. Yes, the average rating for my app has gone down since Google made this change. But rankings are relative, not absolute, and all apps are being treated equally, so it shouldn't make a difference in the rankings. What wasn't mentioned is that the number of ratings has increased substantially. I get at least twice as many ratings as before. In my opinion, more users giving ratings will result in more reliable rankings.
- zango 13y ago
  If Google will fix the user interface of the box, than it will be acceptable. (1) let the user correct a rating if tapped by accident (2) offer the option to leave a comment
nmohideen 13y ago
On my 2.3.3 phone, the quick rating widget takes the entire width of the phone and about 15% of the height. Any flick that originates on a star is irreversibly treated as a rating. Once the star is touched, there is no way to cancel the action. My flick style never results in any accidental 5 stars (because it is near the right edge). Wonder how many of my app users are inadvertently screwing my rating because of this design. I am surprised that Google's has such a low regard for UI design. They hide their contact forms, and don't respond when you happen to find the form.

103 points by radley 13y ago

Cache since it's down right now: http://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Fblog.testfairy.com%2Fgoogle-play-store-ratings-drop%2F http://webcache.googleusercontent.com/search?q=cache%3Ahttp%...

tgasson 13y ago
It also had an update post http://blog.testfairy.com/google-play-store-ratings-drop-update http://blog.testfairy.com/google-play-store-ratings-drop-upd... http://webcache.googleusercontent.com/search?q=cache:z4GGDYq_4qUJ:blog.testfairy.com/google-play-store-ratings-drop-update/+&cd=2&hl=en&ct=clnk http://webcache.googleusercontent.com/search?q=cache:z4GGDYq...

kaeawc 13y ago

Has this been fixed? The post is dated Jan 1st.

coin 13y ago

"Error establishing a database connection"

mtgx 13y ago

I can definitely see how trying to touch the icon could've hit the 1/2 star rating instead.

Pxtl 13y ago

I don't know if google is evil, but I'm pretty sure they're kinda stupid. Seriously, how many UI whoopsies are we going to hear about this week?

dredmorbius 13y ago

That's a nice bit of legwork to show that poor UI design translates directly into erroneous user interactions. In conjunction with other recent Google stories (https://news.ycombinator.com/item?id=7107325 https://news.ycombinator.com/item?id=7107325) it's both cautionary, largely exculpatory of users (Google caused this problem), and speaks more than slightly to Google's hubris (I've seen plenty of poor UI from them over the past several years). An interesting aspect of this dynamic is that the object and subject of the error are asymmetric in this case. It's one thing to accidentally out yourself through "not reading instructions" or "just winging it" or "accidentally hitting the wrong control". It's another to accidentally take an action that hurts another person -- here in a significant way as it directly affects their reputation. Hacker News is of course hardly blameless. I've more than once downvoted a comment or flagged or upvoted a story, especially on mobile, trying to access other controls on the site. At least the story flagging can be undone, though the other actions cannot be. But that's only if I care enough to do so.

banachtarski 13y ago
Google design sucks. They've never in the history of the company made a solid effort to nail UI and it's not part of their culture at all. Every one of their products is marred by UI inefficiencies, clutter, or inconsistency.
- jmduke 13y ago
  * They've never in the history of the company made a solid effort to nail UI and it's not part of their culture at all.* Regardless of how you feel about how it turned out, they clearly have spent the past two years focusing on unifying design and UX across their products.
  - dredmorbius 13y ago
    One of the worst sins of UI is "unification". It's helpful to have meaningfully consistent metaphors and themes across applications. For the UNIX / Linux commandline, that's processes and pipes, BSD and GNU argument flags, and the like. There are various competing keyboard accelerator schemes: emacs, vi, Wordperfect keyboard templates, DOS, Windows 3.x, Windows 9x/NT, Apple's Mac hotkeys, GNOME and KDE.... There are fundamental layout, font, and color palette schemes. But one of the worst overall design features I see is taking a given scheme and applying it without regard to suitability across a wide range of applications with a wide range of use cases. For Google this results in the insanity of persistent browser elements occupying 25-30% of vertical screen real estate in Gmail, Groups, Book search, and other products. When the workaround is for me to either push my browser window half-height above my viewable monitor frame and drag down the bottom of the window, to be able to present my work front and center, or to open the elements inspector and delete elements from the DOM just to be able to see the relevant work, something's very, very wrong. Microsoft of course was infected with similar idiocy with its modal, non-resizeable dialogs and wizards. Even GNOME, for all its UI/UX brain death, generally provides for resizable dialogs, and allows text to be copied from its error pop-ups (under Windows the only recourse was and to the best of my knowledge remains screenshotting the damned thing). But ... Google and UI/UX? Not so much.

daphneokeefe 13y ago

Site is down already.

product50 13y ago

The number of times I have seen the database connection error for an obscure site which appears on Hacker News is alarming. It is not like millions of users are trying to get to their site that they go down. If you can't host it don't - why can't sites just use something like Posthaven or something - to ensure more reliability.

ajjai 13y ago

I see similar behaviour on my app.. Many 1 star ratings without any comments.. Initially I was convinced it must be competitor playing games.. But the trend continued.. Not sure how long Google will take to address this issue..

abcd_f 13y ago

So here's the UI in question - http://blog.testfairy.com/wp-content/uploads/2014/01/quick_suggest.png http://blog.testfairy.com/wp-content/uploads/2014/01/quick_s... - and apparently the large stars represent not a suggestion rating, but an app rating. So dismissing suggestion as a bad one you'd basically give a 1 star rating to an app. Or so the devs claim. Coming from the iOS side, I just don't get it. How can you rate an app that you haven't downloaded? And then if you did download it, why is it being quick-suggested to you again? Furthermore, if you did in fact download it, removed it and then reacted negatively to a suggestion to download it again - how's your low-star reaction is NOT really a rating for an app itself? Also, if you look closely at the graphs, the "crashing" involves rating slide from 4.67 to 4.38 and from 4.79 to 4.74. It's still an unpleasant development for sure, but that's not a "crash" or a "drop", that's a dip. The matter appears to be completely blown out of its proportions.

trurl42 13y ago
It's a suggestion to rate the shown app, to get better app recommendations. It only shows apps you have downloaded.
ZeroGravitas 13y ago
Also, if the dip affects everyone equally, then its more like a rescaling (presumably on average these reviews are lower than before, I'd guess because no-one bothers to review a 2-3 star item, only if they love it or really hate it)
- Kudos 13y ago
  Not until the users view of those ratings rescale too.
- igordcard 13y ago
  If the dip affected everyone equally it wouldn't be that bad, as most of the saturation found above the 4.8 would be leveraged... However these suggestions will be rated, in average, as greater as the overall empathy and knowledge users have regarding the kind of app in question. For instance, regarding my app Tabata Trainer (which went from solid 4.57 in December 9th to 4.21 at the time of this post), how many persons identify with the word "Tabata"? Most of them will never give a 5. If someone creates a game with a very similar name to a widely known and adorable game, then people will probably rate the suggestion highly. This kind of stuff will just lead apps to be rated unequally... and bad (really bad) apps will raise their ratings, which is shameful.

blueskin_ 13y ago

Google are crap at UI design. It's all Metro, and then they do things like this.

pilif 13y ago

I might be an overly conservative old fart, but there was a time not so long ago, where your success wasn't depending on the UI decision (or any other decision) of a corporate overlord. Back then, you made a product and it was immediately available for all the world to use. Yes, you had to spend some time/money/effort for promoting it and yes, the success might have been dependent on your ability to promote it. But you didn't have to ask for permission to build it and there was no way to get screwed because the UI design of an entity you have no control over has changed slightly from one day to another. I think it's a real shame that these days seem to be over more and more.

andybak 13y ago
> Yes, you had to spend some time/money/effort for promoting it and yes, the success might have been dependent on your ability to promote it. And you're back in exactly the same situation. Where are you going to promote it and who else has control over that communication channel?

mixedbit 13y ago

When Google started to require users to use G+ and real names to post a review on the Play store, a number of reviews dropped significantly. The poor design decision described in the article may be a result of an effort to fix this dropped user engagement rate.

fauigerzigerk 13y ago

I understand the frustration of the developers affected by this UI change, but I think that actively asking users to rate an app they have downloaded may lead to better ratings in a statistical sense. User ratings greatly suffer from self selection bias (http://en.wikipedia.org/wiki/Self-selection_bias http://en.wikipedia.org/wiki/Self-selection_bias). Asking users who didn't initially rate an app do so as part of some other workflow might reduce that bias a little bit. That said, I'm very distrustful of user ratings in general, both as a developer and as a consumer, because people often rate stuff based on criteria that don't concern me at all.

magicalist 13y ago
It would also help if the graphs in the post started at 0 stars on the y-axis, because the drops look much more severe than they really are. I imagine in an ultra-competitive niche that the drop in the third graph from 4.72 to 4.54 could be a painful drop, but the first graph has a drop of .03 stars, and the second is a drop of .05 stars. Perfectly in line with a very slight adjustment (though I'm sure it's alarming when you first notice it happening). To be fair, the graph formatting doesn't look like the author's fault, just the analytics page automatically zooms and crops like that.
- davedx 13y ago
  Can you rate something 0 stars?
quarterto 13y ago
actively asking users to rate an app they have downloaded may lead to better ratings But it's not asking them to rate the app, it's asking them to rate the suggestion. And since rating a suggestion is usually a thumbs-up/thumbs-down, it primes users to select the extreme ratings.
- fauigerzigerk 13y ago
  >But it's not asking them to rate the app, it's asking them to rate the suggestion. No, if this is really the UI in question then it is not asking users to rate the suggestion: http://blog.testfairy.com/wp-content/uploads/2014/01/quick_suggest.png http://blog.testfairy.com/wp-content/uploads/2014/01/quick_s... It's clearly asking users to rate one app to get suggestions for other apps.

moondowner 13y ago

Maybe it's related to this new awful app review trend? https://news.ycombinator.com/item?id=7044833 https://news.ycombinator.com/item?id=7044833

magicalist 13y ago
They're at least referring to the same phenomenon. The followup article to this one that tgasson posted[1] quotes and links to that very HN thread. [1] https://news.ycombinator.com/item?id=7113614 https://news.ycombinator.com/item?id=7113614

hannahmitt 13y ago

Interesting. I'd wondered about an inflection point in my graph as well, but in the opposite direction. Our average rating has been climbing much faster since Dec 10. Looks like this feature is influential, but not necessarily negative.

k00pa 13y ago

I added a dialog that asks the user rate the app, remind later or just ignore it. The trick is to add the dialog to show up on like 5-10 startup off the app. If the user is starting the application after 5 tries, he is going to like it. And provided the way to rate it in couple of clicks, the user usually gives positive ratings.

mschuster91 13y ago

I think legislation should be made (maybe anti-trust laws?) which requires companies to provide adequate, free 1st-level user support for paid services, and reasonably priced user support for free services. The current way of big companies "just doing stuff" which deeply affects other people/businesses who do not have a single way to contact the offender is a nightmare which has to stop. As the market obviously fails here, legislation is required.

xeroxmalf 13y ago

Or the other new problem[0]: "Actually a great app, but I left 1 star so my review would be seen first!" [0] https://news.ycombinator.com/item?id=7044833 https://news.ycombinator.com/item?id=7044833

PaulHoule 13y ago

Google Play is a joke. Maybe it's better if you have a different version of Android but when I search for anything broad I see a bunch of free shovelware apps. I'd like to have an option to see just the paid apps or the most expensive apps just to clear out the noise and the spam.

fharper1961 13y ago

I'm Android dev, and I think the "Want quick suggestions?" feature is a good addition to Google Play, because it encourages users to rate apps. Yes, the average rating for my app has gone down since Google made this change. But rankings are relative, not absolute, and all apps are being treated equally, so it shouldn't make a difference in the rankings. What wasn't mentioned is that the number of ratings has increased substantially. I get at least twice as many ratings as before. In my opinion, more users giving ratings will result in more reliable rankings.

zango 13y ago
If Google will fix the user interface of the box, than it will be acceptable. (1) let the user correct a rating if tapped by accident (2) offer the option to leave a comment

nmohideen 13y ago

On my 2.3.3 phone, the quick rating widget takes the entire width of the phone and about 15% of the height. Any flick that originates on a star is irreversibly treated as a rating. Once the star is touched, there is no way to cancel the action. My flick style never results in any accidental 5 stars (because it is near the right edge). Wonder how many of my app users are inadvertently screwing my rating because of this design. I am surprised that Google's has such a low regard for UI design. They hide their contact forms, and don't respond when you happen to find the form.