5 ms·

*highest IMDB rated IMDB ratings are of pretty dubious quality. They also fall victim to a recency bias - looking at their Top 250 list, Avengers Endgame is cu

by untog 7y ago

*highest IMDB rated IMDB ratings are of pretty dubious quality. They also fall victim to a recency bias - looking at their Top 250 list, Avengers Endgame is currently listed as the number nine best film ever made, which... come on.

tpm 7y ago
It's also currently second highest rated series on csfd (czech and slovak movie db) which makes it less likely that the ratings are gamed or otherwise dubious.
notus 7y ago
It might also be worth pointing out that the majority of IMDB raters are males in their 30's so it is going to be heavily biased towards what appeals to males in their 30's. I think there is a lot of overlap between what different genders and age groups enjoys still but it still shows a pretty clear bias. You can filter the ratings down by gender or age group to get a better impression.
signal11 7y ago
They are also subject to brigading[1], although the movie in question has recovered a bit[2]. [1] https://fivethirtyeight.com/features/the-story-behind-the-worst-movie-on- https://fivethirtyeight.com/features/the-story-behind-the-wo... [2] https://www.imdb.com/title/tt2574698/ratings https://www.imdb.com/title/tt2574698/ratings
- myroon5 7y ago
  first link missing the last part: https://fivethirtyeight.com/features/the-story-behind-the-worst-movie-on-imdb/ https://fivethirtyeight.com/features/the-story-behind-the-wo...
KyleOS 7y ago
I don't know if that is down to recency bias, but rather the sheer number of fans for that particular genre. There are not many movies in the top 50 you could reasonably make that same argument for. Also, I'm not sure you could make that argument for IMDB's top 250 TV series [1], but maybe that's just my own preferences speaking. [1] https://www.imdb.com/chart/toptv/ https://www.imdb.com/chart/toptv/
- dmurray 7y ago
  Three of the top ten aired in 2019, and all ten aired this century. I think it's fair to say we are in a golden age of TV series, but it's also just that TV shows don't age as well as film, which causes a recency bias.
ssully 7y ago
Incredibly dubious. I have found user ratings for things that draw 'passionate' fan bases (movies, video games, etc) to be nearly useless over the last 3 to 5 years. Review bombings are becoming all too common over issues that more often then not have nothing to do with the actual product. Luckily, I haven't really seen this with books yet (at least on Goodreads), but I am sure it has just as much potential to be hit in a similar way.
- ameister14 7y ago
  Goodreads ratings are massively dubious to the point of almost uselessness, frankly.
  - ssully 7y ago
    Luckily I haven't had a bad experience yet. I've found some reviews to be a little more positive then I would expect, but I would take people being overly generous then the review cesspools I have stumbled upon on Steam or IMBD/Rotten Tomatoes.
    - ameister14 7y ago
      I've seen a pretty good amount of brigading on Goodreads, especially on less known books
- avip 7y ago
  And what is good, Phaedrus, and what is not good — need we ask anyone to tell us these things? Yes Socrates, let's consult the mob.
- jimmaswell 7y ago
  I disagree with review consensus so often they might as well be white noise to me.
pledg 7y ago
Also comparing a mini-series of 5 episodes against multi-year of 60+
oh_sigh 7y ago
There is also a...non-recency bias in other rankings like AFI, where movies from the 50s and 60s are way overrepresented in the standings. Was "some like it hot" really the funniest movie of all time?
zepearl 7y ago
Agree. I usually use http://themoviedb.org http://themoviedb.org to have an at least slightly more reliable idea of how a film is - not perfect but I can usually get an idea if I compare the rating of the candidate movie against other films which I liked and disliked in the past. P.S.: the guys of tmdb have (or at least they used to have) as well a nice API to download stuff (ratings, posters/images, etc... if I remember correctly when I programmed a simple media player looong time back). Edit - link to API docs: https://developers.themoviedb.org/3/getting-started/introduction https://developers.themoviedb.org/3/getting-started/introduc...
minimaxir 7y ago
Here's a heatmap distribution of IMDB rankings of movies by the year they were released: https://minimaxir.com/img/imdb-data-analysis/imdb-4.png https://minimaxir.com/img/imdb-data-analysis/imdb-4.png It's less of a recency bias (the trend is mostly flat by year of movie release), but both ends of the spectrum are more extreme for movies in the last decade.
- cm2187 7y ago
  It would be more interesting to follow the evolution day by day of the rating from release date to until it stabilizes after a year or two.
  - minimaxir 7y ago
    If data snapshots exist for IMDb data (that do not involve scraping the website/archives of the website), I'm open to that.
    - cm2187 7y ago
      No need to scrap, you can download most of their db, including ratings. But you need to start doing this every few days to start building a history: https://datasets.imdbws.com/ https://datasets.imdbws.com/
cm2187 7y ago
I think the ratings are clearly manipulated. All new releases somehow start with stellar ratings before sinking in after a few months. That being said I was about to complain that Chernobyl didn't deal with the most interesting part, how the explosion happened in the first place, but the last episode did exactly that. I agree that it is an excellent mini-series. Though the explantion in the tv show pretty much mirror the wikipedia article.
Sukotto 7y ago
Personally, my "go to" list for movie rankings is http://www.phi-phenomenon.org/ http://www.phi-phenomenon.org/ which is a labor of love project attempting to generate a forced ranking of all-time best movies. The main downside of it is that it discounts recent (past 5+ years) movies. This is partly due to the infrequent updates, and partly because films take a while to appear in the data sources the site uses for its statistical analysis. (I am not affiliated with that site. I'm just a long-time, satisfied user of it)
- skinnymuch 7y ago
  The list seems to be a consensus popularity contest, right? The only top 50 movies from this century are LOTR films for example. While the 90s films, half are cult films that became massively hyped or Oscar films.
oliwarner 7y ago
These are self-selected audience scores. No more dubious than a single critic review, just different. Endgame is the final piece in a 22-film story arc. The people paying to watch it in the cinema, reviewing it are already fans before they've seen it. It won't be until home release where more moderate voices step in. I'm not sure you can really level the same claims against Chernobyl though. These are people who had an HBO subscription, and that really only suggests they like Game of Thrones. You can also argue these sorts of willing-audience reviews are actually a good thing, if you align with the reviewing audience. IMDb does make basic demographics available on their reviews.

by untog 7y ago

tpm 7y ago

It's also currently second highest rated series on csfd (czech and slovak movie db) which makes it less likely that the ratings are gamed or otherwise dubious.

notus 7y ago

It might also be worth pointing out that the majority of IMDB raters are males in their 30's so it is going to be heavily biased towards what appeals to males in their 30's. I think there is a lot of overlap between what different genders and age groups enjoys still but it still shows a pretty clear bias. You can filter the ratings down by gender or age group to get a better impression.

signal11 7y ago

They are also subject to brigading[1], although the movie in question has recovered a bit[2]. [1] https://fivethirtyeight.com/features/the-story-behind-the-worst-movie-on- https://fivethirtyeight.com/features/the-story-behind-the-wo... [2] https://www.imdb.com/title/tt2574698/ratings https://www.imdb.com/title/tt2574698/ratings

myroon5 7y ago
first link missing the last part: https://fivethirtyeight.com/features/the-story-behind-the-worst-movie-on-imdb/ https://fivethirtyeight.com/features/the-story-behind-the-wo...

KyleOS 7y ago

I don't know if that is down to recency bias, but rather the sheer number of fans for that particular genre. There are not many movies in the top 50 you could reasonably make that same argument for. Also, I'm not sure you could make that argument for IMDB's top 250 TV series [1], but maybe that's just my own preferences speaking. [1] https://www.imdb.com/chart/toptv/ https://www.imdb.com/chart/toptv/

dmurray 7y ago
Three of the top ten aired in 2019, and all ten aired this century. I think it's fair to say we are in a golden age of TV series, but it's also just that TV shows don't age as well as film, which causes a recency bias.

ssully 7y ago

Incredibly dubious. I have found user ratings for things that draw 'passionate' fan bases (movies, video games, etc) to be nearly useless over the last 3 to 5 years. Review bombings are becoming all too common over issues that more often then not have nothing to do with the actual product. Luckily, I haven't really seen this with books yet (at least on Goodreads), but I am sure it has just as much potential to be hit in a similar way.

ameister14 7y ago
Goodreads ratings are massively dubious to the point of almost uselessness, frankly.
- ssully 7y ago
  Luckily I haven't had a bad experience yet. I've found some reviews to be a little more positive then I would expect, but I would take people being overly generous then the review cesspools I have stumbled upon on Steam or IMBD/Rotten Tomatoes.
  - ameister14 7y ago
    I've seen a pretty good amount of brigading on Goodreads, especially on less known books
avip 7y ago
And what is good, Phaedrus, and what is not good — need we ask anyone to tell us these things? Yes Socrates, let's consult the mob.
jimmaswell 7y ago
I disagree with review consensus so often they might as well be white noise to me.

pledg 7y ago

Also comparing a mini-series of 5 episodes against multi-year of 60+

oh_sigh 7y ago

There is also a...non-recency bias in other rankings like AFI, where movies from the 50s and 60s are way overrepresented in the standings. Was "some like it hot" really the funniest movie of all time?

zepearl 7y ago

Agree. I usually use http://themoviedb.org http://themoviedb.org to have an at least slightly more reliable idea of how a film is - not perfect but I can usually get an idea if I compare the rating of the candidate movie against other films which I liked and disliked in the past. P.S.: the guys of tmdb have (or at least they used to have) as well a nice API to download stuff (ratings, posters/images, etc... if I remember correctly when I programmed a simple media player looong time back). Edit - link to API docs: https://developers.themoviedb.org/3/getting-started/introduction https://developers.themoviedb.org/3/getting-started/introduc...

minimaxir 7y ago

Here's a heatmap distribution of IMDB rankings of movies by the year they were released: https://minimaxir.com/img/imdb-data-analysis/imdb-4.png https://minimaxir.com/img/imdb-data-analysis/imdb-4.png It's less of a recency bias (the trend is mostly flat by year of movie release), but both ends of the spectrum are more extreme for movies in the last decade.

cm2187 7y ago
It would be more interesting to follow the evolution day by day of the rating from release date to until it stabilizes after a year or two.
- minimaxir 7y ago
  If data snapshots exist for IMDb data (that do not involve scraping the website/archives of the website), I'm open to that.
  - cm2187 7y ago
    No need to scrap, you can download most of their db, including ratings. But you need to start doing this every few days to start building a history: https://datasets.imdbws.com/ https://datasets.imdbws.com/

cm2187 7y ago

I think the ratings are clearly manipulated. All new releases somehow start with stellar ratings before sinking in after a few months. That being said I was about to complain that Chernobyl didn't deal with the most interesting part, how the explosion happened in the first place, but the last episode did exactly that. I agree that it is an excellent mini-series. Though the explantion in the tv show pretty much mirror the wikipedia article.

Sukotto 7y ago

Personally, my "go to" list for movie rankings is http://www.phi-phenomenon.org/ http://www.phi-phenomenon.org/ which is a labor of love project attempting to generate a forced ranking of all-time best movies. The main downside of it is that it discounts recent (past 5+ years) movies. This is partly due to the infrequent updates, and partly because films take a while to appear in the data sources the site uses for its statistical analysis. (I am not affiliated with that site. I'm just a long-time, satisfied user of it)

skinnymuch 7y ago
The list seems to be a consensus popularity contest, right? The only top 50 movies from this century are LOTR films for example. While the 90s films, half are cult films that became massively hyped or Oscar films.

oliwarner 7y ago

These are self-selected audience scores. No more dubious than a single critic review, just different. Endgame is the final piece in a 22-film story arc. The people paying to watch it in the cinema, reviewing it are already fans before they've seen it. It won't be until home release where more moderate voices step in. I'm not sure you can really level the same claims against Chernobyl though. These are people who had an HBO subscription, and that really only suggests they like Game of Thrones. You can also argue these sorts of willing-audience reviews are actually a good thing, if you align with the reviewing audience. IMDb does make basic demographics available on their reviews.