4 ms·

This is a classic case of learning nothing by averaging together two different populations (as acknowledged in the article, critically-acclaimed indies and main

This is a classic case of learning nothing by averaging together two different populations (as acknowledged in the article, critically-acclaimed indies and mainstream blockbusters). Its like averaging the height of an NBA team with a kindergarten class. It would probably be more informative to group movies by size of release (number of theaters they open in), as well as advertising budget. Just by eye there seems to be a correlation for well reviewed blockbusters to do better than poorly reviewed ones.

sago 11y ago
Right. For movies over $10m budget, there is a positive correlation, and because the OPs graphs are log, it isn't a minor one. Indies form the second clump high in the rankings. The negative correlation is caused, as the density contours show, by a knot of highly reviewed indies showing up in the data, confusing the positive correlation of the studio movies.
- minimaxir 11y ago
  Alright, I double-checked, with appropriate filters on data, and RT scores: Indie cluster alone: log-corr: -0.1243865 corr: -0.1673577 Blockbuster cluster alone: log-corr: 0.230726 corr: 0.2570628 Maybe I should have separated the clusters, and I'll definitely do that for further analysis (even though at best, the correlation is weak in either direction, and the log doesn't change much). I did, however, want to address the general reliance on RT score for any movie, so I stand by the post.
  - sago 11y ago
    > the general reliance on RT score for any movie The reliance of RT score for what? What are people routinely extrapolating from the RT score that you think is not valid? Extrapolating budget from the RT score is definitely invalid: you've shown that. But who does that? People might go the other way: extrapolate quality from a big budget (it must be good, it's a big blockbuster), but as you say, that correlation is valid (though noisy). So not quite sure what you're standing by, sorry. I don't want to sound rude and argumentative. I'm not trying to be a jerk, I'm very happy to concede your analysis is applicable where it is, I just genuinely don't understand what you're referring to here.
    - minimaxir 11y ago
      Very frequently I've heard the question "is this a good movie?" and the response is "what does the RT score say?" Many online stores include the RT score inline when you buy a movie too. This avenue doesn't immediately distinguish whether a movie is a blockbuster or indie or how much money it made. But then again, some people do read movie descriptions thoroughly...
      - sago 11y ago
        [edit: Read the article again, deleted previous response, sorry] So you think box office is a better correlate of 'good movie' than RT score? I didn't get that from the post, but if that's what you're saying, then yes, I concede you have shown that correlation is not valid.
        minimaxir 11y ago
        Maybe not better, but more important. I'll update the post with a footnote tomorrow clarifying things.
      - andreasvc 11y ago
        Commercial success and quality ("good movie") are obviously very different things. Critics care about the latter. Quality is hard to quantify, but that's no excuse to take such a poor proxy as sales. If you want to test whether Rotten Tomato scores are meaningful, I say you'd have to test whether they are meaningful to a particular individual with particular tastes; i.e., test the correlation of someone's judgments against a critic's.

sago 11y ago

Right. For movies over $10m budget, there is a positive correlation, and because the OPs graphs are log, it isn't a minor one. Indies form the second clump high in the rankings. The negative correlation is caused, as the density contours show, by a knot of highly reviewed indies showing up in the data, confusing the positive correlation of the studio movies.

minimaxir 11y ago
Alright, I double-checked, with appropriate filters on data, and RT scores: Indie cluster alone: log-corr: -0.1243865 corr: -0.1673577 Blockbuster cluster alone: log-corr: 0.230726 corr: 0.2570628 Maybe I should have separated the clusters, and I'll definitely do that for further analysis (even though at best, the correlation is weak in either direction, and the log doesn't change much). I did, however, want to address the general reliance on RT score for any movie, so I stand by the post.
- sago 11y ago
  > the general reliance on RT score for any movie The reliance of RT score for what? What are people routinely extrapolating from the RT score that you think is not valid? Extrapolating budget from the RT score is definitely invalid: you've shown that. But who does that? People might go the other way: extrapolate quality from a big budget (it must be good, it's a big blockbuster), but as you say, that correlation is valid (though noisy). So not quite sure what you're standing by, sorry. I don't want to sound rude and argumentative. I'm not trying to be a jerk, I'm very happy to concede your analysis is applicable where it is, I just genuinely don't understand what you're referring to here.
  - minimaxir 11y ago
    Very frequently I've heard the question "is this a good movie?" and the response is "what does the RT score say?" Many online stores include the RT score inline when you buy a movie too. This avenue doesn't immediately distinguish whether a movie is a blockbuster or indie or how much money it made. But then again, some people do read movie descriptions thoroughly...
    - sago 11y ago
      [edit: Read the article again, deleted previous response, sorry] So you think box office is a better correlate of 'good movie' than RT score? I didn't get that from the post, but if that's what you're saying, then yes, I concede you have shown that correlation is not valid.
      - minimaxir 11y ago
        Maybe not better, but more important. I'll update the post with a footnote tomorrow clarifying things.
    - andreasvc 11y ago
      Commercial success and quality ("good movie") are obviously very different things. Critics care about the latter. Quality is hard to quantify, but that's no excuse to take such a poor proxy as sales. If you want to test whether Rotten Tomato scores are meaningful, I say you'd have to test whether they are meaningful to a particular individual with particular tastes; i.e., test the correlation of someone's judgments against a critic's.