4 ms·

It's time to take into account the dates on the Android app average rating

124 points by fesja 13y ago

Karunamon 13y ago
FTA: We don’t have a bad average rating (3.98 with almost 2.900 ratings), but I’m pretty sure it doesn’t reflect the quality of the last version we launched. Really? The whole thesis of this article is that app ratings are broken because you have a 4 out of 5?
- rurounijones 13y ago
  No, he is saying that most of those votes are irrelevant to the current downloadable version and should be treated as such, the actual score is posted as an aside and is moot.
- fesja 13y ago
  no, the idea for the article comes from the changes in iOS7 that have improved the number of our downloads. So it affects all apps that started a few years ago. But it's also true that I'm a perfectionist myself, so if we have spent thousands of hours on the app, I want it to have the fair rating. And with the current algorithm it hasn't.
  - madeofpalk 13y ago
    IIRC, iOS has grouped ratings by version (as show latest by default) for longer than iOS 7. I believe it was introduced in like iOS 4 or 5?
cyphax 13y ago
Seems like common sense to me? If I want to know if an application is any good, I look at the ratings and if it has a lot of good ratings, I look at the latest comments to make sure that the lower ratings are for older versions. Surely it'd be a bit more convenient for Google to take versions of apps into account, and additionally devices, too, if they can (a lot of times an app will be okay for most devices, but users of a certain device, different from my own, might have troubles), but it's -- in my experience -- not too much of a hassle.
- sjwalter 13y ago
  You can filter reviews by device for devices you own. See: https://play.google.com/store/apps/details?id=de.worldiety.athentech.perfectlyclear https://play.google.com/store/apps/details?id=de.worldiety.a..., for instance. Check out the reviews section. Click on it to expand and notice the reviews filters.
  - cyphax 13y ago
    That is awesome, thanks! I think in my situation I'm better off not using that filter, as I don't have a phone that lots of people seem to have. :D And they have a filter for newest version, too. Maybe they should mark reviews that aren't for the latest version, perhaps by changing the font color to something a little lighter. (It does super suck that you need Google+ if you want to write a review though, they force that on us, so I've never written a review myself nor will I.)
coob 13y ago
The iOS app store weights by version, I'm not sure which is preferable.
samsaga2 13y ago
The problem is not the average rating. The problem is the faked ratings. I think it's not normal that a lot of ios applications has five starts. It cannot be that a new app that anybody knows about it has a lot of positive comments.
- fesja 13y ago
  just received an email from http://www.androidrate.com/ http://www.androidrate.com/ offering "Boost your app with 5 star ratings and positive reviews". Yes, Google should ban them.
randartie 13y ago
I think at a very least, votes older than a certain time should not be counted unless a new version has not been released.
eksith 13y ago
A version/ratings graph might really help here. That will also dissuade devs from frequently releasing updates to hide the fact that their product is sub-par. If you have a whole bunch of recent releases with a handful of early 5-star ratings that are glowing with praise, something is fishy.
liotier 13y ago
Just group by version. Time-based decay might work too but linking a rating to a specific version (maybe only a major version - there are pro & cons) has more meaning.
- masklinn 13y ago
  > linking a rating to a specific version (maybe only a major version - there are pro & cons) has more meaning. The major issue there is that new versions suddenly reset ratings entirely. Using e.g. an exponential decay of history model means the weight of older ratings falls, but it remains as a trail in the overall rating.
  - cheeaun 13y ago
    The iOS App Store does that. It shows 'Current version' and 'All versions' (toggleable between both).
    - knff 13y ago
      The Play store (version 4.3.11) does this too, actually. There's an options dropdown under All Reviews that lets you filter by only the current version and/or your device.
- bentcorner 13y ago
  Showing some sort of sparkline would be helpful (i.e., overlay major versions with their reviews over time), although may be difficult to convey with the limited real estate. Maybe something to expose if you wanted to dig deeper into the review data (users probably don't care that much for $0.99). I'm guessing this is probably already available to developers anyway?
uwemaurer 13y ago
You can see a graph of the ratings over time here: http://www.appbrain.com/app/touristeye-travel-guide/com.touristeye http://www.appbrain.com/app/touristeye-travel-guide/com.tour... (in the "Changelog" tab)
- fesja 13y ago
  thanks!
Imagenuity 13y ago
The date should not only be taken into account for app RATING, but for app RANKING. I see many apps that haven't been updated in over two years that are ranked higher (due to more downloads) than current releases similarly or higher rated with fewer downloads. I would favor an time-based exponential decay type of factoring of ranking. (http://en.wikipedia.org/wiki/Exponential_decay http://en.wikipedia.org/wiki/Exponential_decay)
- TheCraiggers 13y ago
  Wouldn't that just favor apps that push trivial updates once a week that either do nothing, or just add bloat to software that already fulfilled its purpose quite well? Personally, I like the idea that if I release an app that's actually feature complete and relatively free of bugs, it should be favorably ranked and not biased against due to it not being "new". A great example of this is ConnectBot, which is a great ssh client for android. I can't remember if I've ever seen an update for it in the years I've been using it. Does this mean it should be ranked lower than JuiceSSH just because Juice is newer / has updated about once a month since it released? I'd say no. Then again, I'm not a fan of the proposal of TFA for this same reason. Frankly, I think the store already gives us the tools to make our own decision by letting us filter by our device and by the current version.
  - gte910h 13y ago
    You don't have to do things stupidly. You can say "Older than X, decay", "has feature Y, decay". This is the company that makes pagerank, I'm confident they could come up with a useful ranking and rating algorithm
    - TheCraiggers 13y ago
      True that Google came up with PageRank, and have some smart people thinking up ways to rank things I may want. It's also true that as long as PageRank has been around, there have been people exploiting how it works for their own personal gain. See: My "update every week to boost my score" example above. Personally, I sort of like the thought that if somebody releases a crap app and gets a bunch of 1-star ratings they are likely going to stick for awhile. Harsh, I know, but it does give companies some incentive to get it mostly right at first. Speaking of, something I'd like to see is a developer ranking. Even something so simple as averaging all their apps' ratings would be helpful. Done right, it would help identify those soundalike apps, too.
awjr 13y ago
Amazon does a similar thing to seller ratings: https://images-na.ssl-images-amazon.com/images/G/01/sellerfeedback/time-weighted-graph-2._V389181558_.png https://images-na.ssl-images-amazon.com/images/G/01/sellerfe... . Ratings have an initial slow impact (this is more to do with getting the goods to the user and allowing the returns process to run it's course) then the feedback has a 100% affect that tails off over a year or two. A similar approach could be used by google.
ollybee 13y ago
A well known tor hidden service weights customer feedback by age, spend history volume, and variety of orders. I think this would work well for android apps. you might want to change spend history for number of apps downloaded but with higher rating for paid app downloads.
coin 13y ago
I refuse to up vote any medium.com post until they stop disabling pinchzoom on mobile devices.
k3n 13y ago
It's also time to take into account those who don't have a G+ profile...
daphneokeefe 13y ago
Ratings trends over time can go in the other direction too. I got bit by an update the other day on a mature and very popular app which just released a tragically flawed update. I over confidently accepted the update and then started reading the latest reviews which we're shouting in all caps not to get this update. And what you said about excluding ppl who aren't on G+.
pbreit 13y ago
Pretty much all ratings should be on some sort of rolling average based on time and or quantity.

124 points by fesja 13y ago

FTA: We don’t have a bad average rating (3.98 with almost 2.900 ratings), but I’m pretty sure it doesn’t reflect the quality of the last version we launched. Really? The whole thesis of this article is that app ratings are broken because you have a 4 out of 5?

rurounijones 13y ago
No, he is saying that most of those votes are irrelevant to the current downloadable version and should be treated as such, the actual score is posted as an aside and is moot.
fesja 13y ago
no, the idea for the article comes from the changes in iOS7 that have improved the number of our downloads. So it affects all apps that started a few years ago. But it's also true that I'm a perfectionist myself, so if we have spent thousands of hours on the app, I want it to have the fair rating. And with the current algorithm it hasn't.
- madeofpalk 13y ago
  IIRC, iOS has grouped ratings by version (as show latest by default) for longer than iOS 7. I believe it was introduced in like iOS 4 or 5?

cyphax 13y ago

Seems like common sense to me? If I want to know if an application is any good, I look at the ratings and if it has a lot of good ratings, I look at the latest comments to make sure that the lower ratings are for older versions. Surely it'd be a bit more convenient for Google to take versions of apps into account, and additionally devices, too, if they can (a lot of times an app will be okay for most devices, but users of a certain device, different from my own, might have troubles), but it's -- in my experience -- not too much of a hassle.

sjwalter 13y ago
You can filter reviews by device for devices you own. See: https://play.google.com/store/apps/details?id=de.worldiety.athentech.perfectlyclear https://play.google.com/store/apps/details?id=de.worldiety.a..., for instance. Check out the reviews section. Click on it to expand and notice the reviews filters.
- cyphax 13y ago
  That is awesome, thanks! I think in my situation I'm better off not using that filter, as I don't have a phone that lots of people seem to have. :D And they have a filter for newest version, too. Maybe they should mark reviews that aren't for the latest version, perhaps by changing the font color to something a little lighter. (It does super suck that you need Google+ if you want to write a review though, they force that on us, so I've never written a review myself nor will I.)

coob 13y ago

The iOS app store weights by version, I'm not sure which is preferable.

samsaga2 13y ago

The problem is not the average rating. The problem is the faked ratings. I think it's not normal that a lot of ios applications has five starts. It cannot be that a new app that anybody knows about it has a lot of positive comments.

fesja 13y ago
just received an email from http://www.androidrate.com/ http://www.androidrate.com/ offering "Boost your app with 5 star ratings and positive reviews". Yes, Google should ban them.

randartie 13y ago

I think at a very least, votes older than a certain time should not be counted unless a new version has not been released.

eksith 13y ago

A version/ratings graph might really help here. That will also dissuade devs from frequently releasing updates to hide the fact that their product is sub-par. If you have a whole bunch of recent releases with a handful of early 5-star ratings that are glowing with praise, something is fishy.

liotier 13y ago

Just group by version. Time-based decay might work too but linking a rating to a specific version (maybe only a major version - there are pro & cons) has more meaning.

masklinn 13y ago
> linking a rating to a specific version (maybe only a major version - there are pro & cons) has more meaning. The major issue there is that new versions suddenly reset ratings entirely. Using e.g. an exponential decay of history model means the weight of older ratings falls, but it remains as a trail in the overall rating.
- cheeaun 13y ago
  The iOS App Store does that. It shows 'Current version' and 'All versions' (toggleable between both).
  - knff 13y ago
    The Play store (version 4.3.11) does this too, actually. There's an options dropdown under All Reviews that lets you filter by only the current version and/or your device.
bentcorner 13y ago
Showing some sort of sparkline would be helpful (i.e., overlay major versions with their reviews over time), although may be difficult to convey with the limited real estate. Maybe something to expose if you wanted to dig deeper into the review data (users probably don't care that much for $0.99). I'm guessing this is probably already available to developers anyway?

uwemaurer 13y ago

You can see a graph of the ratings over time here: http://www.appbrain.com/app/touristeye-travel-guide/com.touristeye http://www.appbrain.com/app/touristeye-travel-guide/com.tour... (in the "Changelog" tab)

fesja 13y ago
thanks!

Imagenuity 13y ago

The date should not only be taken into account for app RATING, but for app RANKING. I see many apps that haven't been updated in over two years that are ranked higher (due to more downloads) than current releases similarly or higher rated with fewer downloads. I would favor an time-based exponential decay type of factoring of ranking. (http://en.wikipedia.org/wiki/Exponential_decay http://en.wikipedia.org/wiki/Exponential_decay)

TheCraiggers 13y ago
Wouldn't that just favor apps that push trivial updates once a week that either do nothing, or just add bloat to software that already fulfilled its purpose quite well? Personally, I like the idea that if I release an app that's actually feature complete and relatively free of bugs, it should be favorably ranked and not biased against due to it not being "new". A great example of this is ConnectBot, which is a great ssh client for android. I can't remember if I've ever seen an update for it in the years I've been using it. Does this mean it should be ranked lower than JuiceSSH just because Juice is newer / has updated about once a month since it released? I'd say no. Then again, I'm not a fan of the proposal of TFA for this same reason. Frankly, I think the store already gives us the tools to make our own decision by letting us filter by our device and by the current version.
- gte910h 13y ago
  You don't have to do things stupidly. You can say "Older than X, decay", "has feature Y, decay". This is the company that makes pagerank, I'm confident they could come up with a useful ranking and rating algorithm
  - TheCraiggers 13y ago
    True that Google came up with PageRank, and have some smart people thinking up ways to rank things I may want. It's also true that as long as PageRank has been around, there have been people exploiting how it works for their own personal gain. See: My "update every week to boost my score" example above. Personally, I sort of like the thought that if somebody releases a crap app and gets a bunch of 1-star ratings they are likely going to stick for awhile. Harsh, I know, but it does give companies some incentive to get it mostly right at first. Speaking of, something I'd like to see is a developer ranking. Even something so simple as averaging all their apps' ratings would be helpful. Done right, it would help identify those soundalike apps, too.

awjr 13y ago

Amazon does a similar thing to seller ratings: https://images-na.ssl-images-amazon.com/images/G/01/sellerfeedback/time-weighted-graph-2._V389181558_.png https://images-na.ssl-images-amazon.com/images/G/01/sellerfe... . Ratings have an initial slow impact (this is more to do with getting the goods to the user and allowing the returns process to run it's course) then the feedback has a 100% affect that tails off over a year or two. A similar approach could be used by google.

ollybee 13y ago

A well known tor hidden service weights customer feedback by age, spend history volume, and variety of orders. I think this would work well for android apps. you might want to change spend history for number of apps downloaded but with higher rating for paid app downloads.

coin 13y ago

I refuse to up vote any medium.com post until they stop disabling pinchzoom on mobile devices.

k3n 13y ago

It's also time to take into account those who don't have a G+ profile...

daphneokeefe 13y ago

Ratings trends over time can go in the other direction too. I got bit by an update the other day on a mature and very popular app which just released a tragically flawed update. I over confidently accepted the update and then started reading the latest reviews which we're shouting in all caps not to get this update. And what you said about excluding ppl who aren't on G+.

pbreit 13y ago

Pretty much all ratings should be on some sort of rolling average based on time and or quantity.