7 ms·
A statistical analysis of Rotten Tomatoes
- SwtCyber 1y agoWhat really gets me is how effective the "Certified Fresh" badge still is, even for people who know the system is flawed
- retox 1y agoObvious to anyone using the site or aware of is ratings, including the author, but it is good to see some analysis as evidence.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- daft_pink 1y agoI will say that when I used to go to the theaters, which was before the pandemic and I started a family I used metacritic. I found that any time I went to something that was red, I absolutley regretted it and it was terrible. Yellow was more hit or miss and top green scores were pretty good. Exceptions were comedy where a lower score could still mean a good film, and politics oriented films, where a bad film with a media approved message could get a really good score even if it sucked. It’s sad to not get a reliable indicator of that and someone should just resurrect the old score and call it Bad Apples. Since the actual score seems transparent, why not develop a competitor.
- qwertytyyuu 1y agoWould there be some selection bias as well? As info about movies becomes readily available, generally the people who go see movies would have decided that they would probably enjoy said movie, and write favourable reviews
- mxxx 1y agoIf they're actual movie reviewers then their job is to go see films regardless of whether they think they'd personally enjoy them. Some of the best reviews come from reviewers who have to go and see something they absolutely hate and would never go see for their own entertainment.
- nzealand 1y agoFun fact. You can click into Popcornmeter and see verified audience reviews versus all reviews. E.g. Audience who went to see 2025's Snow White loved it. Those who haven't seen it hated it. Who is more biased?
- mxxx 1y agoRegardless of the introduction of sycophantic reviewers, the 3/5 = fresh thing has always been a pretty half-ass threshold imo, and that a fact that a film can be "100% fresh" on RT on the basis of every single reviewer saying "yeah it's nothing special but it's fine, 3 stars" is fairly easy to misinterpret.
- cubefox 1y agoYeah. Pixar movies are often close to 100%. IMDb ratings are usually far more reasonable.
- pbsds 1y agoa mean rating of 3 can only be 100% fresh if the variance is 0
- chongli 1y agoWouldn't we expect the most truly mediocre movies to have the lowest variance in opinions?
- throwup238 1y agoThat might be why the Marvel movies score so highly.
- bhaney 1y agoThat is what "every single reviewer saying '[...] 3 stars'" means, yeah.
- famouswaffles 1y agoYeah and his point is that's never going to happen lol. People bring up the 100% point a lot and it's a bit silly because a movie with a significant number of ratings is never going to have that kind of distribution. That's why it's always a hypothetical never backed with actual examples. It's one of those things that sounds plausible until you look at the numbers. Movies close to 100% have pretty high average scores and Movies with majority 3/5's are nowhere near 100%. Yeah 100% for RT doesn't mean 10/10, but that's it.
- aaron695 1y ago[dead]
- bufferoverflow 1y agoRottenTomatoes has been rotten for over a decade. IMDB user ratings are much more useful, but still far from perfect.
- relwin 1y agoIMDB also had a ratings inflation of about 0.5 points about the same time (anecdata), where movies over 5.5 were watchable back then. Now a 6.0+ demarcates a watchable movie. Or my threshold changed in the past 10 years or so...
- dcreater 1y agoStopped reading when I saw a bar chart being used for correlating critic and audience scores.
- mjamil 1y agoIt’s interesting that people pay close attention to one-size-fits all number (regardless of the pros and cons of the methodology used to derive said number). I find RT really useful for collating the reviews from “top” reviewers in one place: over the years, I know how my interests align with the tastes of particular reviewers, and I don’t have to look in multiple places to get a snapshot view of their opinions.
- nebula8804 1y agoIts absolutely hopeless. The rankings are gamed because Rotten Tomato still carries some semblance of credibility (although in this post Trump winning era, sources of credibility can get eroded more quickly) It does not get clearer than when a political movie comes out. 2018 was an interesting year, two movies came out that really allowed me to get a clearer picture of what was going on: "Knock Down The House" and "Death of a Nation". When "Knock Down The House" (documentary featuring the leftwing US politician AOC) came out, I got interested in scraping the data off of Rotten Tomatoes and studying it. Before making any moves, I first watched the movie for myself in a theater (and also got to see a live Q&A with the director to understand her thought process) The movie had at the time a 100% rating from critics and ~80% from viewers. After watching it, I would concur with the viewer ranking but felt that the critic ranking was unusually high. Seriously? 100%? (It has now gone down to 99% but still). In regards to the viewer ranking I conceded that I was probably biased which is why I also ran this experiment on Death of a Nation (also saw in theaters but to a room with only one other moviegoer). Knock Down the House eventually got featured on Tucker Carlson like a year after release(I think it coincided with Netflix making it free on youtube). I watched in realtime how the movie critic score kept going down and down and down to where it is now (11%). Dumping the scraped data, I ran a simple analysis and discovered a large portion of the people ranked it with no comments, or simplistic things like how stupid AOC is and many had had no other ratings other than this movie or the only other film is the one featuring Illhan Omar (another politician hated by the right). For Death of a nation, the scores were flipped. A whopping 0% in the critic score(12 reviews) while the user score stood at a respectable 87% (again at the time when I did my scraping yet again we saw tons of 1 movie reviewers). Yeah the movie royally sucked and was painful to watch but 0%? That was a bit fishy. This essentially killed any credibility that I had in Rotten Tomatoes. I started to trust places like /r/movies and /r/AMCsAList only to get burned by that as well when movies recommended in the comments would end up being terrible and then when I went back to criticize the films, I would get criticized and downvoted to non visible status. It was not a definite signal but gave me the feeling that there is a lot of astroturfing going on there as well. Furthermore, these movies promoted on Reddit would typically be in 3/5 range on Rotten tomatoes which further made me think there is no real way to get a real signal if a film is likely to be good or not. What I started to do was not a great metric but has helped cut down on the cruft: Follow specific actors/directors I really liked and ones that I felt were in it to make good films. As an example, actresses like Mary Elizabeth Winstead have turned down big roles in favor of indie films or other interesting scripts to the detriment of her career but the films have been more enjoyable and interesting. In each film I also find other actors to follow and if I start to see more studio promotion of a specific person (for example Anya Taylor Joy after The Queens Gambit) I start to caution away and sometimes just drop that actor from the list. In her case I stuck to films she worked with other people that I determined I liked(like Edgar Wright) before feeling like there is too much promotion going on and just dropping her from the list. Other than this, I fill out my list with franchises I like or subjects that I always give a chance to (science/space, etc.). I know I am leaving out a lot of potential good films but the noise has become unbearable to the point where I don't want to waste my time anymore. A few years later Rotten Tomatoes introduced "Verified Reviews". I thought this will be amazing as now it will only include people with skin in the game (ie. verified to have paid for a ticket)...except now this has been completely hijacked as well. Going back to the example of political films now what the right wing does is they have a billionaire finance a film through some intermediary group then free tickets to the movie are given out at conservative events. People books seats to the movie, promote it on social media, post a "Verified Review" and then often don't show up to the screening. I have discovered sold out screenings to some of these movies but when I went to the theater to see some other film, i'd often peek into the screening of these films only to find they are almost empty. The movie plays regardless of if someone shows up or not. Furthermore some of these films actually have a code that they show at the end of the film to gift a free ticket to a friend so the box office numbers of these films are inflated and its all a bunch of hogwash. Like I said its hopeless. In a way we saw the rise of this new fake world play out in Hollywood before it took over social media and the rest of the internet now. How will it end? People trusting only what they already know they like or from trusted friends and everything else is ignored. This article has got me thinking of an interesting idea though: What if we go and determine which of the critics are known to provide reviews that reflect our tastes (maybe by reading reviews of movies that we enjoyed), then pull only those review data and compute a new Tomato score based only on those critics? We could toss the Rotten Tomatoes tomato meter in the trash and get back to a legitimate review that you could use as a positive signal again.
- seemaze 1y agoI just attend my local independent cinema. Sometimes I’m blown away. Sometimes the film sucks, but there are only 3-4 options at any given time. Simplifies the decision, and at the very least, they serve beer.
- ildon 1y agoI only rely on IMDb scores and they're still reliable if enough time has passed since the movie release. A movie with more than 6 avg is usually enjoyable, while below 6 is not worth watching. Over 7 is usually very good, over 8 is a masterpiece
- kelseydh 1y agoI agree wholeheartedly. What none of the review sites have figured out is how to adjust TV show user scores so that they don't score so much higher than movies.
- defrost 1y agoThere are many metrics that can be sampled, many challenges to normalization. eg: https://ext.to/browse/?sort=seeds&order=desc&cat=1&q=2019 https://ext.to/browse/?sort=seeds&order=desc&cat=1&q=2019 is a listing of 2019 movie torrents ranked by seeds (number of clients holding full copies of a torrent version). A normalization challenge is to group torrent variations (1080p rips and 720p rips and WEB-DL's and BluRay and etc.) and tally up and rank interest in various films over time. Clearly Ne Zha (2019), a Chinese Animation, Fantasy, Adventure movie was a global pirate star of that year .. should it be "normalized" by population of country of origin to smooth out the home team having a billion+ in population "bias" ? One advantage of ranking films by year and pirate copies is it provides a pragmatic measure of "staying power" https://ext.to/browse/?sort=seeds&order=desc&cat=1&q=1964 https://ext.to/browse/?sort=seeds&order=desc&cat=1&q=1964 My Fair Lady, Dr Strangelove, Rudolph the Red-Nosed Reindeer, Mary Poppins, A Fistful of Dollars, and Goldfinger are still being hoarded 60 years after their release. * https://www.themoviedb.org/movie/615453 https://www.themoviedb.org/movie/615453 * https://www.rottentomatoes.com/m/ne_zha https://www.rottentomatoes.com/m/ne_zha
- sirnicolaz 1y agoI still miss the times when I would decide to watch a movie based on the cover at the VHS store or based on a recommendation. Much faster, way more serendipity.
- bmacho 1y agoNetflix shows movie posters and Reddit has recommendation threads.
- sirnicolaz 1y agoExactly the same thing
- neilv 1y ago> To account for this influx of reviewers, Rotten Tomatoes has created a "Top Critic" designation reserved for established media outlets, such as The New York Times and The Atlantic. However, this label has no special bearing on a film's top-line Tomatometer score and is largely incorporated into ancillary aspects of the site. Just last night, I noticed that I could access the two percentage scores for critic reviews. If you go to "https://www.rottentomatoes.com/m/the_dilemma https://www.rottentomatoes.com/m/the_dilemma", and click on the critic reviews percentage (25%), you get a popup that lets you select between seeing the All Critics score (25%) and Top Critics score (28%). (And if I'd thought to check Rotten Tomatoes first, when selecting what looked like a fun light comedy on Netflix, I wouldn't have wasted an hour of my life before I said WTF, checked RT, and continued to be in a bad mood.) Incidentally, I'd love to have the Tomatometer score integrated into my UI for video streaming services. The services seem to instead like to use their own very generous scoring instead. (When they show any score at all. Some like to suppress the ratings for new shows they produced, presumably to avoid shooting down their own poor shows before people watch them by default.) But Rotten Tomatoes is a much better predictor of how I'll like a show than the streaming service scores are. But maybe the streaming services don't want to expose that the majority of the movies and series offered at any time now range from mediocre to outright bad.
- michaeljx 1y agoI am doing my movie selection via Plex, which has both tomatoes' and IMDb scores in the movie description
- brabel 1y agoYou guys are opening yourself to manipulation. Why not just be open for surprises? That’s what I do. Most of the time it’s bad surprises, but the occasional masterpiece that everyone else seems to hate make up for it. I do seem to not have the same taste as most people writing reviews though. I almost always find other people’s opinions ludicrous.
- maximus_01 1y ago
- jackero 1y agoI find RT scores very accurate but not the raw score. What I mean is that a 70% score is meaningless to me. I need to know the movie genre, the audience score, the age of the movie and then I basically do a “lookup table” in my head. And I have that lookup table because I’ve looked up every movie I’ve watched on RT for 15 years so I know how the scores correlate to my own personal opinions. As an example: the author said that critic scores should align with audience scores but no that’s not true at all. Critics tend to care more about plot continuity, plot depth and details while the audience tends to care about enjoyability. Both are important to me so I always look at both scores. That’s why a lot of very funny comedies have a 60-69% critic score but a 90%-100% audience score — because it’s hilarious but the plot makes no fucking sense and has a million holes. And if you see a comedy with 95% critic but 70% audience, it will be thought-provoking and well done but don’t expect more than occasional chuckles.
- Finbel 1y agoAlways wondered why Rings of Power have 84% critics score but just 49% audience.
- moi2388 1y agoBecause critics get paid, whilst audience have to pay.
- dandellion 1y agoI didn't like the idea that my money had paid for such a disservice of my favourite book, so it pushed me to cancel my Prime subscription that had been ongoing for years. I don't buy nearly as much on Amazon these days as a consequence.
- Gareth321 1y agoI rarely get angry about bad content but RoP felt like a personal affront. I love Tolkien's world and the people who put RoP together did so with not just ignorance and incompetence, but some kind of malice. They intentionally butchered Tolkien's writing and world. This stands in such stark contrast with Peter Jackson's position that it is not his right to inject his personal values and narcissistic hubris into the movies. He chose to honour the material as best he could while adapting it. It is, without any shadow of a doubt, the better approach.
- fifticon 1y agoaudience scores, as I view them, is very much an indicator of "did the marketing connect to the intended audience?" A great movie that met the wrong audience, will reasonably get a low score.
- jamiek88 1y agoThat’s a pretty good point. Lots of films find an audience post marketing phase and have a long tail. If the marketing phase had connected better their tail might be shorter.
- Joel_Mckay 1y agoIn my experience of film reviews, Rotten Tomatoes high positive scores are not always representative of how entertaining some content will be.. However, the negative skew is almost always accurate for how bad something will be for all viewers. The bimodal distribution of professional critics versus community opinions obviously describes what is happening behind the data. Recycled AstroTurf for 1980's cult films have little appeal to modern viewers even with maximal pandering for nostalgia. Good Hollywood writers likely starved to death, and were replaced with LLM interpreted Nielsen Media Research data. Most video games offer better writing now... lol =3
- DantesKite 1y agoHighly recommend MovieLens if you have eclectic or niche movie tastes. It can be a bit of a nerd-snipe though. One of my favorites activities is rating movies and watching the recommended ratings (what rating it thinks I'll give a movie) update overnight. It's at the very least, better than average chance at predicting which movies you will like.
- baronblackmore 1y agoSo, the best movie is the one we sell
- rurban 1y ago> Humanity Has Stopped Producing Bad Art: After a century of trial and error, mankind has perfected the art of cinema, as proven by recent masterworks like Cats, Space Jam: A New Legacy, the live-action Snow White, Red One, and Joker: Folie à Deux. Critics, who were once joyless automatons thriving on takedowns of human creativity, now bask in this golden age of moviemaking, lavishing praise upon the timeless artistry of The Walt Disney Company and Warner Bros. Discovery. This really should appear in professional film reviews.
- s_dev 1y agoI've always found IMDBs rating to be far far better. Anything that's 7+ is generally good, anything below that is flawed. The Tomato meter just comes off as random and an unreliable indicator for me.
- bazmattaz 1y agoYes I was going to say something similar. I often use a blend on IMDb score and rotten tomatoes to judge whether I should watch a movie. The only thing I will say is that IMDB scores are also (likely) gamed by movie studios to artificially inflate their scores. You also need to be careful with certain genres and series on IMDB that attract positive ratings from the archetypal IMDB user. This is why marvel movies and such receive such sky high ratings. Basically if the movie is not a mainstream superhero type blockbuster full of cgi you can use the IMDB score if your judgement
- kelseydh 1y agoRotten Tomatoes is the First Past the Post of movie reviews. Incredibly distorted and unreliable in capturing the sentiment of the majority.
- garyclarke27 1y agoInteresting, over the past few years, I've ignored the critics score and just looked at the audience score - this explains why.
- FuturisticLover 1y agoI find IMDb better than RT. RT even though it has an Audience score. It tends to give priority to the Critics' score, which in most cases we know is influential in one way or another. IMDb score doesn't rely on a group of people, but on all users. It may give a little more weightage to the US users, but that's fine. Its top movies and TV shows make a lot more sense, unlike RT.
- cantor_S_drug 1y agoTangentially related : It is possible to deanonymize users from kaggle dataset or netflix competition. https://medium.com/@EmiLabsTech/data-privacy-the-netflix-prize-competition-84330d01cc34 https://medium.com/@EmiLabsTech/data-privacy-the-netflix-pri... Compared to the example of the medical records, Netflix had been very careful not to add any data that could identify a user, like zip-code, birthdate, and of course name, personal IDs, etc. Nevertheless, only a couple of weeks after the release, another PhD student, Arvind Narayanan, announced that they (together with his advisor Vitaly Shmatikov), had been able to connect many of the unique IDs in the Netflix dataset to real people, by cross referencing another publicly available dataset: the movie ratings in the IMDB site, where many users post publicly with their own names. https://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf https://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf https://courses.csail.mit.edu/6.857/2018/project/Archie-Gershon-Katchoff-Zeng-Netflix.pdf https://courses.csail.mit.edu/6.857/2018/project/Archie-Gers...
- Gareth321 1y agoI agree. I stopped relying on RT scores years ago. There are far fewer occasions when the audience score doesn't align with my preferences. Today I am so far away from most critics in taste and preference, it's like they live on another planet.
- mosst 1y ago[dead]
- firtoz 1y agoThere needs to be a "score from people who produce scores like me" for everything. E.g. for coffee shops I really don't care if the staff are rude but I care if there are tables and it's clean and the WiFi is good. I was thinking of having granular scoring e.g. having people rate WiFi, rate cleanliness etc but that won't work because people are lazy. So some kind of Amazon algorithm of "people who buy similar things" but for movie, cafe, etc ratings. To bring it back on topic, for movies I care about good story and visuals but not so much how "woke or not" it is, I prefer if it's intelligent Vs dumb except if it's a comedy then dumb is fine, and so on.
- cubefox 1y agoI would like to see the movies with the largest discrepancy between critic and audience score. Edit: An example is Star Wars - The Last Jedi. https://www.rottentomatoes.com/m/star_wars_the_last_jedi https://www.rottentomatoes.com/m/star_wars_the_last_jedi Critic score: 91% Audience score: 41%
- hermitcrab 1y agoI am very confused by the second chart (the bar chart). Shouldn't that be a scatter plot?
- cainxinth 1y agoI’ve always preferred Metacritic because it assigns a weighted average. RT only tells you the “approval rate,” the percentage of favorable ratings. I don’t care if 100% of critics and fans agreed it was better than total crap.
- jonathaneunice 1y agoMovie reviews (stars) given on Amazon Prime's streaming service always seemed incredibly sus. All the bad movies from the long-tail back catalog all seemed remarkably highly-rated. Typical pattern: Forth-rate sci-fi action movie amalgamation of the 12 other well-known sci-fi action hits? Yes!! Four stars! Just assumed Prime's claimed ratings were untrustworthy and must be based on some corporate "we'll promote your dreck, no problem-o!" data feed. But a bit sadder that Rotten Tomatoes, once at least reasonably legit and crowd-sourced, now poisoned by the same "everything's good! watch it! watch it!" inflation. Not surprising. Anti-surprising, really. Still a bit sad. Thank you for the excellent historical, statistical deep dive.
- generalizations 1y agoSeems like it's been common knowledge for a while that the critic reviews are useless, but the audience reviews are still valuable. You just have to make sure to actually look at the audience score.
- kelseydh 1y agoI completely lost faith in Rotten Tomatoes after consistently seeing Marvel capeshit, including their worst slop, reliably get high audience and critical scores.
- nyeah 1y agoAs a casual user of Rotten Tomatoes, I can't make any sense out of the ratings. They could be random. The 'critic' ratings have no predictive power at all for me. The 'user' ratings have some value but not a whole lot.
- ChrisMarshallNY 1y agoIt's simple: When there's money to be made, the folks with the money to make, will pull out the stops to game the reviews. We have seen this in just about every review system ever (except Consumer Reports). They game it, milk some money, until everyone figures out it sucks, then abandon it. The reviewers count on getting brought. I really miss Mr. Cranky[0, 1]. Here's his review of Battlefield Earth[2]: [0] https://en.wikipedia.org/wiki/Mr._Cranky https://en.wikipedia.org/wiki/Mr._Cranky [1] http://web.archive.org/web/20120503072554/http://www.mrcranky.com/content/mr-cranky-review-archives http://web.archive.org/web/20120503072554/http://www.mrcrank... [2] http://web.archive.org/web/20120926010236/http://www.mrcranky.com/movies/battlefieldearth.html http://web.archive.org/web/20120926010236/http://www.mrcrank...
- rfwhyte 1y agoIts not just Rotten Tomatoes, the vast majority of so called "Reviews" the average consumer sees these days are being manipulated in service of corporate interests. Amazon's full of paid reviews for scam products, so called "Independent" review sites like Wirecutter are basically just advertorial hosting platforms now, or are even secretly owned by the companies who's products are being "Reviewed," 99% of YouTube reviews are nothing more than sponsored content that regurgitates press release talking points from companies who provide the reviewers free products, Google and Yelp reviews of local businesses have an entire manipulation industry built up around them (Just Google "Yelp review service" to see what I'm talking about," and the sad reality we live in now is most anywhere you go, most so called "Reviews" you see these days are either from bots or corporate shills. We officially live in the "No trust" era and its only going to get so, so much worse from here.
- lordhumphrey 1y agoThe idea that something like a movie's "good"-ness (or whatever) could be measured accurately, reliably or even usefully, by experts or laypeople, should be absurd on its face to any right-thinking person. What a movie does or does not do for you will depend on a host of things which couldn't possibly be blamed on the movie itself - there's no objective experience of a movie. The lighting, what you had for dinner, the comfort of your chair, if the baby starts crying, who you're with in the room, how they're feeling, whether you actually really have time for a movie or should be doing something else, if you're 16 and just had your first break up 3 days before, whether you saw the original movie 30 years ago or totally missed the boat, if you've smoked a few spliffs with your friends in the car in the parking lot of the cinema beforehand, etc etc. People who think differently are, I think, simply going along with a very peculiar recent trend. It's a general effect the Internet has had on our aesthetics. We no longer think our experience matters or even really exists, instead we think our take on the thing in its social context amongst all the other people's takes is all that matters and exists. Liking something means liking it exactly how the average person liked it, and sharing a "take" means describing how we deviated, ever so slightly, from the agreed-upon-reading of the agreed-upon-thing. In reality, if you watch a movie ten times, you've had ten different experiences. If one hundred people watch a movie, there have been one hundred different experiences. It's a great shame that we've forgotten this fact, and lost ourselves in a culture of "reacting".
- LinuxAmbulance 1y agoThere's truth to that, but the vast majority of people can figure out that a movie is bad, no matter what they had for dinner or what's going on in their lives. We're not looking for the Platonic form of the movie review, but just a simple "Is this movie trash? Is it amazing?" aggregated reaction from viewers.
- EcommerceFlow 1y agoIMDB is my go-to, especially since users grade on "harsher" terms and anything >8 is considered amazing.
- HK-NC 1y agoThe one and only time I used the site, I discovered "Black Panther" was the number one movie of all time. Thats all the analysis I needed to reach a verdict.