4 ms·

Overall there seems to be an epidemics of "fakeness" in Internet ratings - even looking on IMDb for movie ratings I see e.g. a movie 7.7/10 but first few pages

by bitL 11y ago

Overall there seems to be an epidemics of "fakeness" in Internet ratings - even looking on IMDb for movie ratings I see e.g. a movie 7.7/10 but first few pages are unanimously 1-4/10 by what seem like genuine reviewers complaining about being deceived by IMDb to watch the movie...

seizethecheese 11y ago
Which is why I always use Metacritic. The issue with Metacritic is that anything with a specific social message will be artificially reviewed higher since critics have a much higher incentive to preserve reputation than random reviewers on IMDB.
- frik 11y ago
  For me the star ratings on IMDb often correlate with my own rating. But Metacritic and RottenTomatoes are often very skewed in a different direction and I really don't care to analyze why they conclude with such a different rating. Also with video games, Metacritic is pretty useless in my opinion - it's far better to read just a few sites I trust and they give me a better rating overview than a broad range of many bought ("100") or clueless ("20") critic reviews or user reviews.
  - 4ad 11y ago
    I completely agree. I find IMBb rating 98% accurate. No, I won't necessarily like a 8.1 move better than a 8.0 movie, but I will like it more than a 7.5 movie. And a 6.9 movie will always be crap to me. I can think of very few movies where IMBb rating didn't agree with me at all, in either direction. And it's usually with movies that have a very social/philosophical/american message. But these are biased even more on the non-IMDb sites, like rotten tomatoes! It might not work for everyone, but it sure works for me.
minimaxir 11y ago
For movies, IMDB average scores are exclusively between 5 and 8: a Four Point Scale. In contrast, Rotten Tomatoes and Metacritic scores hit the full gamut reliably: (I wrote a full statistical blog post on this with visualizations: http://minimaxir.com/2016/01/movie-revenue-ratings/ http://minimaxir.com/2016/01/movie-revenue-ratings/ ) I did a similar thing for Amazon reviews many years ago but it was not very insightful. (http://minimaxir.com/2014/06/reviewing-reviews/ http://minimaxir.com/2014/06/reviewing-reviews/)
- kale 11y ago
  Average scores should trend towards the center. I'm not sure if it's an example of the central limit theorem (I think that deals with different distributions tending towards normal), but you'd expect IMDB scores to trend toward the middle.
  - minimaxir 11y ago
    Central limit theorem assumes that events are independent and random. Which is not the case. (For one, there is a selection bias in that people would only review/vote on a product of they strongly like or strongly dislike it) The lack of trend toward the middle implies that the scale is not great. But there is no incentive to fix it.
mikestew 11y ago
Even worse are movie ratings on iTunes. Though from reading the reviews I can't tell if they're fake, or written by twelve year olds that will watch anything that moves (and I'm not entirely convinced it even needs to move). Regardless, bad slasher flick that went straight to the DVD sale bin? Four stars!!!!111! The plot was paper thin, and the acting was terrible, but I still loved it!!! I've watched it like eight times1111 Thankfully, iTunes displays Rotten Tomatoes ratings.
CM30 11y ago
You also need to keep in mind that bandwagon hopping and fan flame wars/arguments/raids play a large part in these ratings (especially for movies and games on sites like IMDB or Metacritic). So if something has a large, extremely passionate fandom, ratings can easily be skewed way higher than expected. If it's got a large (or at least vocal) group that seriously hates a movie or game, its ratings will be rock bottom. For example, Call of Duty Infinite War and its really bad Youtube ratings. Or the really, really critical coverage of Paper Mario Colour Splash and Metroid Prime Federation Force. Not saying fake reviews and ratings don't exist for commercial gain, they do. But an angry (or really fanboyish) internet mob can cause this phenomenon about as much.

by bitL 11y ago

seizethecheese 11y ago

Which is why I always use Metacritic. The issue with Metacritic is that anything with a specific social message will be artificially reviewed higher since critics have a much higher incentive to preserve reputation than random reviewers on IMDB.

frik 11y ago
For me the star ratings on IMDb often correlate with my own rating. But Metacritic and RottenTomatoes are often very skewed in a different direction and I really don't care to analyze why they conclude with such a different rating. Also with video games, Metacritic is pretty useless in my opinion - it's far better to read just a few sites I trust and they give me a better rating overview than a broad range of many bought ("100") or clueless ("20") critic reviews or user reviews.
- 4ad 11y ago
  I completely agree. I find IMBb rating 98% accurate. No, I won't necessarily like a 8.1 move better than a 8.0 movie, but I will like it more than a 7.5 movie. And a 6.9 movie will always be crap to me. I can think of very few movies where IMBb rating didn't agree with me at all, in either direction. And it's usually with movies that have a very social/philosophical/american message. But these are biased even more on the non-IMDb sites, like rotten tomatoes! It might not work for everyone, but it sure works for me.

minimaxir 11y ago

For movies, IMDB average scores are exclusively between 5 and 8: a Four Point Scale. In contrast, Rotten Tomatoes and Metacritic scores hit the full gamut reliably: (I wrote a full statistical blog post on this with visualizations: http://minimaxir.com/2016/01/movie-revenue-ratings/ http://minimaxir.com/2016/01/movie-revenue-ratings/ ) I did a similar thing for Amazon reviews many years ago but it was not very insightful. (http://minimaxir.com/2014/06/reviewing-reviews/ http://minimaxir.com/2014/06/reviewing-reviews/)

kale 11y ago
Average scores should trend towards the center. I'm not sure if it's an example of the central limit theorem (I think that deals with different distributions tending towards normal), but you'd expect IMDB scores to trend toward the middle.
- minimaxir 11y ago
  Central limit theorem assumes that events are independent and random. Which is not the case. (For one, there is a selection bias in that people would only review/vote on a product of they strongly like or strongly dislike it) The lack of trend toward the middle implies that the scale is not great. But there is no incentive to fix it.

mikestew 11y ago

Even worse are movie ratings on iTunes. Though from reading the reviews I can't tell if they're fake, or written by twelve year olds that will watch anything that moves (and I'm not entirely convinced it even needs to move). Regardless, bad slasher flick that went straight to the DVD sale bin? Four stars!!!!111! The plot was paper thin, and the acting was terrible, but I still loved it!!! I've watched it like eight times1111 Thankfully, iTunes displays Rotten Tomatoes ratings.

CM30 11y ago

You also need to keep in mind that bandwagon hopping and fan flame wars/arguments/raids play a large part in these ratings (especially for movies and games on sites like IMDB or Metacritic). So if something has a large, extremely passionate fandom, ratings can easily be skewed way higher than expected. If it's got a large (or at least vocal) group that seriously hates a movie or game, its ratings will be rock bottom. For example, Call of Duty Infinite War and its really bad Youtube ratings. Or the really, really critical coverage of Paper Mario Colour Splash and Metroid Prime Federation Force. Not saying fake reviews and ratings don't exist for commercial gain, they do. But an angry (or really fanboyish) internet mob can cause this phenomenon about as much.