14 ms·

Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Down

• kefka 10y ago
  And not only that, but also removing their existing collections of videos and replacing it with their "Made by Netflix" shows that mostly amount to shovelware. I'm about ready to drop them, just like they've dropped MASH, soon to be X-Files, and many many movies.
  • xatan_dank 10y ago
    Same. They dropped Peep Show and that was about the only show I Liked that was still on. Plus King of the Hill and Spongebob, but they took those off ages ago.
  • rampage101 10y ago
    Why are they dropping shows? Does it save them money somehow?
    • dghf 10y ago
      Lots of assumptions, presumptions, guesses and maybes here, but: - Netflix's licences to stream shows are presumably for a limited period of time. - There's only so much money in the pot, and they want to spend it on the renewals and new acquisitions they think mostly likely to retain existing subscribers and attract new ones. - Sometimes the rights owner may not want to renew Netflix's licence: they may strike an exclusive deal with another platform, or may feel that having the show on Netflix is cannibalising their DVD sales.
    • garethsprice 10y ago
      They originally negotiated licenses super cheap as the studios thought they were just some niche upstart. Now they're eating the market the studios are realizing their mistake and either not renewing licenses or jacking the rates up massively - hence the move into original content which can't be taken away.
  • FussyZeus 10y ago
    I read something (on HN actually) awhile ago about how Amazon seems to be actively trying to kill Netflix, not by going after subscribers but by going after licenses for programming by over-bidding. I think Netflix though is making more off of their in-house produced content these days so I wouldn't say they're in danger, they just might look quite different in a few years.
    • dominotw 10y ago
      I use amazon channel instead of Netflix now. I subscribe to channel for a while and watch their content exclusively till I am ready to move on. I am currently watching the excellet Xive Tv channel on amazon [1]. Netflix is waay overhyped for what they actually have in their catalog. https://www.amazon.com/Instant-Video/b?benefitId=xivetv&node=2858778011 https://www.amazon.com/Instant-Video/b?benefitId=xivetv&node...
      • FussyZeus 10y ago
        They used to have more. As they've built up their own programming they've been losing a lot more of the other content, but like I said, I think they're doing it out of necessity.
    • empath75 10y ago
      The way amazon is competing against one of their largest customers should be concerning for anyone on aws
  • dominotw 10y ago
    > but also removing their existing collections of videos and replacing it with their "Made by Netflix" shows that mostly amount to shovelware. You are not wrong "US Netflix offers only 31 of IMDb's top 250 movies, study shows" "David Wells, Netflix’s chief financial officer, was quoted by Variety stating that the company wants half of its content to be original productions over the next few years." [1] RIP Netflix! 1. https://www.theguardian.com/film/2016/oct/14/us-netflix-imdb-top-250-movies https://www.theguardian.com/film/2016/oct/14/us-netflix-imdb...
    • obmelvin 10y ago
      Netflix vs non Netflix content aside, the low amount of movies is a direct result of viewing data. TV shows are shorter so can be squeezed into smaller spaces for viewing and have longer plot lines to hook viewers. Can't blame them for putting there money where people waste their time.
    • surge 10y ago
      They're becoming their own content creator...like HBO, and thus reducing their dependence on often hostile networks for content and able to offer more content to more international markets, there is nothing wrong with that. I find myself watching more of the stuff they've produced then the licensed content, so for me, it's been an improvement. If I run out of things to watch, maybe I'll care then, but at least I'm not tied to a cable contract, I can just switch to another streaming provider until they fix that issue. At this point though, I can't keep pace.
  • talmand 10y ago
    Sigh. Why do so many not seem to understand that, in most cases, Netflix is likely not "dropping" anything? Netflix does not own the rights to the content and the owners can decide on the terms of the agreement on how long the content will be on Netflix. People threatening to leave, or actually leaving, lowers Netflix's leverage to try to get and keep the content in their system. This is why Netflix is creating their own content and bringing in content that's not normally within your area. So they can keep it up longer and have more control over their own destiny.
    • kefka 10y ago
      I really don't care how they run their business. Not my problem. What is my problem is: I started watching X-Files for the first time (yeah, never go around to it for the initial airing). Started watching in late February. And starting March 1st, is a warning on the upper left of my monitor saying "Show will be removed April 1". From my viewpoint, I'm paying the same, and getting less. Not only less, but specifically something I'm trying to watch. And yes, I tried watching that psychic alien abducted Netflix show. The sex scene at ep1 annoyed me, and had no interest still in ep3. Boring is putting it mildly.
      • imron 10y ago
        > "Show will be removed April 1". Don't worry, the show goes downhill after the first couple of seasons, so it's probably for the best.
        • TallGuyShort 10y ago
          I watched until the T-1000 showed up. Can't see him as an organic lifeform. But in all seriousness it's a huge UI improvement that they give you a few weeks heads up now. I've narrowly missed the ending to some damn good shows and all I needed was a few days to get to a spot with some closure. I get that the rights aren't entirely in their control, so at least it's something.
      • talmand 10y ago
        I'm just pointing out that such complaints are aimed at the wrong target, that's the way it shall be with any service as long as people don't complain to the companies that control the rights to such content.
  • surge 10y ago
    I'd hardly call most of their shows shovelware. A great deal of them are at least decent or comparable to top shows on TV. Also, you do realize they don't set prices on licensing and probably drop the shows after people already went through them or stopped watching them. If you like a show, watch it while its still on there, don't put it in your queue for 2 years then get mad when it disappears before you got around to it.
  • lotsofpulp 10y ago
    I support Netflix out of philosophy. I want to support media which is offered without restriction and most importantly without advertisement. Netflix goes out of its way to make sure it's as watchable as possible (sans being able to download their stuff without DRM, but I can't blame them for that, it's too risky for their business). I have similar reasons to support HBO. However, I will not send money towards FOX/Comcast/Disney/Viacom because they still want to restrict their content and make it as hard as possible to watch it. Of course, the danger is they could just start introducing advertising again by way of product placement in the story itself, but if it goes too far, I guess I'll cancel then.
    • massysett 10y ago
      After you hand Comcast your money, you can also watch their content on your phone or tablet, anywhere, both live and on-demand. You can also use the phone or tablet to access what's on your DVR. Comcast is actually doing a good job of making it quite easy to watch the content on any device.
      • lotsofpulp 10y ago
        I haven't looked it up in a long time, but I don't think it's as easy as Netflix. Also, I want zero ads, not just the ability to stream. It's a business model problem, and the entertainment model media providers offer is just not comparable to what is available on the internet. Currently, I don't think the interest of Comcast's shareholders is the same as mine, they want to protect and grow their existing revenue streams, which depend on ads and outdated licensing contracts. My interest is to watch what I want, when I want, with as little friction as possible. Netflix and HBO provide that, none of the other big media companies do. Also, Comcast has an interest in opposing net neutrality and providing a shitty internet experience so as to support their dying ad and TV business, so I also don't wish to support them for those reasons.
• nailer 10y ago
  YouTube used to have a star based ratings system too. I wonder if they drew the same conclusions.
• Tepix 10y ago
  Bummer. I understand why they do it but I think it's too coarse. I'd much rather know how many people thought a movie was fantastic and not just decent.
  • poof_he_is_gone 10y ago
    The current rating system is not a running average of how all users ranked the movie, but represents users with similar taste to yours. The same move could show a different rating for different users.
  • Spooks 10y ago
    It would be neat if they gave each user a set number super thumbs (obviously with a better name). You can use them so many times per month or year. I think month would be ideal, but if they went with the year route, make it possible to remove your super thumb like to use it on another movie.
    • jfaat 10y ago
      Likes?
      • imron 10y ago
        Super Thumbs!
    • usrusr 10y ago
      On the contrary, I think that superthumbs would be a brilliant name. If you introduce something as dry and complicated as a quota for strong upvotes, you'd better do anything possible to make it more playful. A certain dosage of silly naming might be a good start.
  • surge 10y ago
    You're forgetting all the other metrics that Netflix collects, like which shows get binge watched, finished, at what episode are people hooked or more or less guaranteed to finish a show after, etc.
• frik 10y ago
  What's next. Just a thumb up like on Facebook? Rating is a hard problem. No system works universally well. IMDb (Amazon property) for example uses 10 stars, Amazon uses five stars. Facebook use thumb up ("likes"). Games ratings are often in percent 1-100% (summarized by metacritics.com and others). School systems around the world use A, B, etc or numbers like 1-5 or 1-6 for grades.
  • eli 10y ago
    HN uses upvotes and flags
  • lorenzhs 10y ago
    You're lumping together different kinds of rating systems. Netflix just wants to recommend you stuff you might like, which is a very different problem from scoring things on an absolute scale (grades, IMDb, game critics). Facebook likes have a social interaction component that's not there in Netflix, so it isn't comparable either.
  • jerf 10y ago
    Netflix lacks the issues that lead Facebook not to implement a thumbs-down. Netflix should still want a thumbs-down. You should also distinguish between aggregate ratings and individual ratings. Metacritic and other aggregators normalize to some scale, but not all the individual reviews use the normalized granularity. A common movie case is Metacritic having to normalize a 5-star, in half-star increments, review to a 100-point scale. I believe Metacritic also allows user reviews to use a 10-point scale, but present the average on a 100-point scale (the first ten integers but with one decimal point, technically allowing 101 ratings from 0.0 to 10.0). Aggregate ratings always have more detail than their constituent parts.
  • throwaway91111 10y ago
    I can safely say up/down voting is pretty terrible across all social media. You end up with HN and Reddit: popularity contests. Meanwhile, popular movies are rarely the ones worth watching, and the top 100 movies on metacritic is a grab bag of (great) movies that probably don't fit your taste profile. Meanwhile, I have to like all the things the same amount. This is just going to lead to me not rating the vast majority of mediocre content on the platform. Why not let you choose as a user?
• flaviojuvenal 10y ago
  But the UX of the current rating system sucks. Rating stars are very small and you have to hover them to know you can rate. Only if you watch a movie to the very end you will see a clearer rating functionality, but even then you have few seconds to use. I bet many new users don't even know they can rate. I wonder if people aren't rating much because the UX sucks, not because it's a 5-star system.
  • poof_he_is_gone 10y ago
    The UX on their web app may have this issue, but they have different methods for rating across their platforms. Some are easier than others.
  • Ambroos 10y ago
    What bothers me is that once you rated something you can't unrate it, or see what the public rating is.
    • ZeroGravitas 10y ago
      I believe what they display is just a guess at how much you will like it (based on people with similar viewing/rating histories), not a public rating as such, so once you've actually rated it, there's very little reason to show the other figure.
      • Ambroos 10y ago
        Wait, really? Damn, that explains why I love literally every 5-star-rated thing they serve me!
    • recursive 10y ago
      I don't believe you can ever see a public rating. Before you rate a title, you can see netflix's prediction of what you will rate it, but that's not public.
• danlindley 10y ago
  I can't imagine any film that conveys a message, cinematography, style, and meaning that could be encapsulated with a simple thumbs up or thumbs down. What a disappointing simplification.
  • dazc 10y ago
    I have found with the current 5 star rating system that anything with less than 4 stars isn't worth watching. And anything with 5 stars has a 50/50 chance of being worth watching. So, for me, any simplification is welcome.
  • whistlerbrk 10y ago
    The star system didn't capture that either though
  • ungzd 10y ago
    That's very rare cases. Cinema is the most unhuman conveyor of entertainment content consumption. Modern movies are nice looking moving pictures generated based on big data and emotion exploitation.
• sgloutnikov 10y ago
  I think it's a great move. Maybe on a similar note, one of the reasons I really like and find more use out of Foursquare is the way their rating system works compared to the five-star Yelp rating system. For example, in Yelp I found that all "good" restaurants are in the above 4-star rating, and that's all the information I can get out of that rating. With Foursquare, they have turned the upvote/downvote/neutral rating into a number between 1-10 that tells me more. Above average is 7+, something unique about the place 8+, truly exceptional 9+. It just seems to me it's a simpler way to rate something by a user, and at the same time classify it more adequately.
• kartickv 10y ago
  If you have many reviewers, you don't need to get more information than thumbs up or down from each person. How much people thumbed it up is a good indicator. Star-rating can be too much detail, anyway: if you're comparing two shows, and one has a higher fraction of five-star ratings, but the other has a higher fraction of ratings that are four or above, which is better? Star ratings can be too much detail and cause confusion. If you want more detail from each person, you can ask specific questions with a yes/no answer, like, "Were parts of it boring?" or "Was it violent?" That's probably better than star ratings.
  • surge 10y ago
    That and Netflix has far more data that indicates likability to a far greater degree, such as how many people finished watching the show, over what period of time did they finish watching the show, after what episode are people more likely to get hooked and finish the series or binge on it, etc.
  • k-mcgrady 10y ago
    >> Star ratings can be too much detail and cause confusion The iOS App Store is a good example of this. An unbelievable number of people, for some reason, thing 1 star = great and 5 star = bad.
    • X86BSD 10y ago
      Haha yes! Where the hell do these people learn that?? The other thing that annoys me with ratings in general, specifically places like amazon, is when people pan a perfectly good item because of some third party factor. "Item was great, but it came in a beat up box from the post office, I'm giving it one star." WTH does someone drop kicking your box have to do with the product?
      • jandrese 10y ago
        I have to think that some of those reviews are just trolls. Bad reviewers will always be part of the system. You have to expect it. Netflix has tons of reviews that go "I turned this off before the opening credit sequence finished, worst movie ever!!!! -- 1 star".
• AlexandrB 10y ago
  > The streaming service said it had been testing thumbs up and down ratings “with hundreds of thousands of members” in 2016 – and it led to 200% more ratings being given. I don't understand why engagement is the right metric here. If someone isn't sure how they feel about a movie, why is it a benefit to have them spew their half-formed thoughts into a like/dislike rating?
  • pasiaj 10y ago
    Some reaction from many has more value than thougthgful one from few
    • Sean1708 10y ago
      Did ... did you just answer with a Haiku?
  • deleted 10y ago
    [deleted]
  • xapata 10y ago
    Big data. Seriously. Lots of low fidelity measurement is sometimes better than a smaller amount of higher fidelity measurement. Especially when that higher fidelity measurement has errors.
  • nindalf 10y ago
    Here's the benefit - getting a clearer picture of your tastes, which allows them to make you better recommendations. Their recommendation engine would be based on finding users similar to you and telling you what they liked. More ratings is better than nuanced ratings for this engine because you're more likely to find overlap between users. The goal isn't making a database of the "best" and "worst" movies and TV shows. We already have the International Movie Database for that. This is just about finding out what you like, and finding other stuff like it.
  • thomnottom 10y ago
    >If someone isn't sure how they feel about a movie, why is it a benefit to have them spew their half-formed thoughts into a like/dislike rating? That's pretty harsh. I've watched plenty of movies that I enjoyed and would give a thumbs up but aren't sure whether it should be a 3 or 4 star movie. There's movies I love but consider too flawed to give 5 stars. And I find little difference between 1 and 2 stars other than the level of regret for wasting my time. Overall I rarely bother rating things on Netflix because it just doesn't feel worth it. So they get nothing out of me, whereas with a thumbs up or down I am more likely to chime in.
• instaheat 10y ago
  What does this say about their algorithm? Forget UX/UI. I feel as though this devalues their recommendations to you.
• bjarneh 10y ago
  What I really want to avoid is the mediocre content i.e. 2.0 - 3.5 stars, very good and very bad can both be entertaining.. Now that mediocre content will be scattered across the two categories I really care about; I guess IMDB/Rotten tomatoes will steer clear of this ridiculous binary rating systems.
  • tgb 10y ago
    Isn't Rotten Tomatoes exactly a (aggregated) binary rating system?
    • laurent123456 10y ago
      Yes it is, and it sometime gives some strange ratings like "Ash vs Evil: season 2" having a 100% rating[0]. Though it's probably a good show, it's not absolute perfection like the rating would imply. 0: https://www.rottentomatoes.com/tv/ash_vs_evil_dead/s02 https://www.rottentomatoes.com/tv/ash_vs_evil_dead/s02
    • bjarneh 10y ago
      Aha, I misunderstood, I thought Netflix was going to recommend movies or not based on that rating system, without giving us the background. I would still prefer something more fine grained though.
  • dajohnson89 10y ago
    I've seen so much good content on Netflix rated at 2 stars or less. Movies and TV shows are way too subjective for user reviews to be used with any kind of reliability. Or maybe I just have low standards.
    • bjarneh 10y ago
      I also find the Netflix critics somewhat harsh, perhaps we are more critical from our sofa, then at the cinemas
• widdma 10y ago
  A problem I'd like to see a good solution to is selection bias. Unless there is some kind of reward, only people that really care are likely rate something I think Netflix's move might help this. It certainly lowers the cost rating.
  • ghaff 10y ago
    There's another type of selection bias as well. I'm not going to watch a movie unless I think I'm going to like it (based on reviews, comments, description, etc.) As a result, with the exception of a few real disappointments, I'm generally at least sorta OK with most of the movies I watch. Even most of my disappointments tend to be in the vein of "It was OK I guess but I don't see why people think this is so great."
• khana 10y ago
  The sad and typical path of a young vibrant company going the way of capitalistic follow the money mantra. t f
• j_s 10y ago
  The best way to rate a movie for me would be: when will I watch it again? They kind of already know this though. (How many times have I watched a particular movie on Netflix?)
  • new_hackers 10y ago
    not for me. I rarely watch movies twice, even those I really enjoy. A better metric for me would be "am I glad I watched this movie"
    • j_s 10y ago
      Therefore, any movie you've watched more than once becomes pure gold for Netflix! Spread that across several million subscribers and get enough data to be useful.
• sodapopcan 10y ago
  Well, if no one else is gonna link it I may as well: https://xkcd.com/1098/ https://xkcd.com/1098/ I'm all for this!
• ykler 10y ago
  From the linked Variety article, "However, over time, Netflix realized that explicit star ratings were less relevant than other signals. Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries." But the signal isn't just for Netflix; it is also for users, who might sometimes be in the mood for something silly and sometimes in the mood for something good. Also, people might rather get more suggestions of good movies even if they are more likely to watch bad ones. (Of course, people might also just overrate documentaries.)
  • bryanlarsen 10y ago
    But I want Netflix to surface good documentaries as well as good silly movies. I'm more likely to watch a silly movie, so with this change my queue is more likely to be filled with silly movies, making me more likely to unsubscribe because it appears that Netflix only has silly movies. I watch more silly movies than good documentaries, but I pay for Netflix mostly for the good documentaries. I can get mindless pablum elsewhere easily.
    • fyolnish 10y ago
      To me it sounds like they should instead use categories as hints for how to weigh ratings of different users for different categories. I definitely don't want my feed to be filled with silly comedies all of a sudden.
    • Ntrails 10y ago
      I'd be pretty happy if netflix managed to actually suggest things there's a vague chance I'll be interested in rather than whatever they recently released/bought and desperately want viewers for..
    • galdosdi 10y ago
      It's a good thing netflix created original programming, because I would have long ago been gone from netflix if they didn't have /something/ nobody else had. Their nonoriginal selection has gone to total shit. Even hulu is competitive with netflix's selection excluding originals
• muninn_ 10y ago
  I wish there was a "meh" option. Sometimes things are ok, but not good or bad. This will leave me not rating a bunch of things unless they're good or bad.
  • eyeinthepyramid 10y ago
    I'd bet Netflix uses the watched but not rated signal, especially if you often rate things.
  • TallGuyShort 10y ago
    Special categories would also be nice. "Would probably be funny if I was high". "Don't watch when high - will freak you the hell out".
• wvh 10y ago
  Is this just part of the trend to dumb everything down to a polarising dichotomy? Because the trend to stop people from having a more complex and rational opinion than just picking sides doesn't seem to exactly improve discourse or debate, or inform and educate... Wouldn't it be more beneficial to have fewer people give a more detailed review to describe what they like and dislike about something? What use is "good" or "meh" compared to for example "I rate this product low because the shoes are very narrow for my normal feet" so you can actual relate it to what you actually are looking for? One person's "meh" could be another's "good"... From the article: "This makes sense – giving a five-star rating takes some thought, especially for something like a movie or TV show. A binary “yes or no” option is much easier for viewers to commit to, [...]
  • Schaulustiger 10y ago
    I think it's more a reaction to a usage pattern of their star system. YouTube abandoned their five star rating system citing that most people voted only 1 or 5 stars with almost nothing in between. [1] I'd guess a vast amount of Netflix users do the same. [1] https://youtube.googleblog.com/2009/09/five-stars-dominate-ratings.html https://youtube.googleblog.com/2009/09/five-stars-dominate-r...
    • stromgo 10y ago
      YouTube is partly to blame for this. Since YouTube reported the average stars, if you saw a 4-star video and thought that it should be 3-star, then the optimal strategy was to vote 1-star to pull it down as fast as possible. If instead YouTube had reported the majority option (and publicized that fact) then it would have incentivized people to vote what they truly think.
      • kartickv 10y ago
        Your suggestion has its own problems. Imagine two videos with a majority rating of three stars. Say 60% of raters have rated it that. But one video has the remaining 40% one-star, and the other has the remaining 40% five-star. Your algorithm will ignore this data, showing a completely wrong result that both videos are equally good.
• PhasmaFelis 10y ago
  I don't have Netflix, so I 'm not clear if the rating shown to users is changing. Or does it not show a rating, just use it to decide what movies to surface for you? In any case, simplifying the ratings users can bestow makes a lot of sense, unfortunately. We like to imagine people carefully considering the merits and flaws like a professional reviewer, but the fact is that the vast majority of users only ever use two ratings anyway: if they like it, 5 stars; if they didn't, 1 star. There's maybe a third option where if they're ambivalent they just don't rate it. So star ratings aren't actually very useful for evaluating products. This xkcd https://xkcd.com/1098/ https://xkcd.com/1098/ made me realize that an Amazon product which has a 20% chance of exploding when you open the box, and otherwise works normally but unexceptionally, is gonna average out to a four-star rating for a product you really shouldn't buy. I always look at the one-star reviews before buying stuff now, no matter how few of them there are; knowing a thing's failure modes is much more useful than a bunch of praise.
  • new_hackers 10y ago
    the histogram of the star ratings are much more valuable. I tend to avoid bathtub curves.
• jerf 10y ago
  I think a lot of people are reacting here as if Netflix is moving from the only signal they have being the star and review system to the only signal they have being the up/down system, and thus confused as to why Netflix would throw away the vast majority of their rating ability. But that's not even close to true. In addition to it technically being up/down/no rating, we've got how long people watch the show for, how the shows they watch form a pattern, how all the patterns of what people watch show global preference patterns, whether they rewatch a show, when they watch what sort of shows, what individual scenes are rewatched vs. skipped... Netflix is swimming in a sea of preference data, not sitting here trying to figure out "Gosh, um, if the user likes this movie 3 vs. 4 stars, uh... what does that mean?" It makes perfect sense to me to optimize this one-data-stream-among-many to increase user participation and get more bits of information from more people engaging with the simpler system, rather than trying to squeeze bits out of the few people willing to use the star system and the even fewer willing to write useful reviews. It isn't really even as shocking as it may seem at first. The star system has 6 states, "no rating", 1, 2, 3, 4, and 5 stars. That's 2.6 bits, with some simplifying assumptions [1]. The thumb system has up, down, and no rating; that's 1.6 bits [1]. To make up for the bits, you need only see ~40% increase in participation over the current star system... think that's going to happen? [1]: The simplifying assumption is that all outcomes are equally likely, but that's not true. I don't have the numbers to run a more complete information theory analysis, but it's not hard to imagine the "no star rating" case is so common that it produces such a small fractional bit that a higher-participation-rate yea/nay/no rating (if you puth this UI in their face, "no rating" becomes much more meaningful, too) straight-up produces more bits of information on average, and is thus simply an improvement even before considering the superior UI experience. I rather suspect this is the case, some very sensible assumptions would suggest this, but I lack the ability to prove this; the "assume all outcomes are equally likely" is at least a concrete case I can discuss.
  • bo1024 10y ago
    Yes, but the ratings aren't just for Netflix to use, they're also interesting information for users. This move seems very inward-focused: Netflix is thinking about the information it wants to get from users, but I wonder if they surveyed people about how much they want to see percentage-likes instead of stars.
• adrianlmm 10y ago
  Terrible idea, thumbs up or down rating only promote political reviews, a movie that makes critics of feminism? thumbs down despite the acting or contents inmediatly, a movie showing the good parts of religion? thumbs down by viceral atheists, some one just needs to look at youtube and see how this doesn't work,
• yladiz 10y ago
  I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities. For example, maybe I like the cinematography of a movie but I think the acting is shit, or maybe I think the taste of the beer is nice but it's too acidic or sour and wouldn't work with specific foods because it's overpowering. Airbnb does this pretty much spot on, in that they ask for 1-5 stars for five categories, rather than just "did you enjoy this Airbnb?" I also think that Untappd does this wrong, because in general beers can't be described in just a 1-5 star rating without deeper explanation. I think that moving from a pure star rating to a thumbs up and down rating is better overall, if only because it makes me, as a watcher, not have to think as much and therefore give a rating where I might not have before. If I want to go more in depth, I can explain more too.
  • masklinn 10y ago
    Incidentally, Steam ratings are also simple up/down, with total and sliding window 30 days aggregate, and an Amazon-style helpful/unhelpful/funny meta-rating.
    • Houshalter 10y ago
      Steam ratings have their own problems. Like the creators changing something minor and getting thousands of protest negative reviews from upset people. Which doesn't reflect the quality of the game at all. But does discourage new people from trying the game. My favorite example is Fallout 4. There are tons of negative reviews from people who put 600+ hours into the game. How the hell do you put 25 days of your life into a game you don't like? They claim it "got boring". Not many games will not be boring after 600 hours...
  • curun1r 10y ago
    > I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities. This is only true when those star ratings are taken out of context. Individual reviewers can have completely coherent star ratings that are very useful for making determinations. The golden age of Netflix discoverability, for me, was back when they added social networking features. I connected my account with many friends and family and was really able to get a sense for everyone's taste in movies. When it came time to discover new movies/tv to watch, I'd open up the ratings history for someone who's tastes I felt aligned with the mood that I was in. Some of my friends would only rate high quality content with 5 stars, others would typically rate mass market, mindless entertainment with 5 stars and others tended to include a lot more 5-star reviews within a certain genre. Over time, I got a feel for that and could use it to my advantage. But when you remove that individual context and only aggregate 5 star reviews across a massive population who all view the 5-star scale in their own way, you get that uselessness that you talked about. Some people may reserve 5-star ratings for truly extraordinary content, but Netflix doesn't weight those ratings any more highly than the 5-star ratings of someone who rates nearly everything as 5 stars. This makes the ratings that Netflix displays meaningless. But that's only because Netflix's current methodology makes them meaningless, not because they're inherently meaningless.
• pizzetta 10y ago
  As I posted in the dupe: This saddens me. >Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries That's not incongruous to me. The stars are not about "enjoyment" factor, they are about perceived quality. I may have a go to cheap ice-cream and rate it 3-stars but rate a good affogato 5-stars and only have it once in a while. They are diluting the meaning of quality and instead are opting for a saccharine "enjoyment" factor. This binary choice does not sit well with me and I hope they abandon the idea soon.
  • bluntfang 10y ago
    When I watch netflix, I use the stars as a measure of if I would enjoy something. I don't think most people use stars as a measure of "quality" because when they browse movies, they are looking for something they enjoy, not something that's high quality that they might not enjoy.
    • bmelton 10y ago
      This seems like pure difficulty for me. When I think of thumbs up / down metrics I think in terms of "would I endorse this?" A movie like "Suicide Squad" was (to me at least) highly enjoyable, though I thought it objectively had many problems with plot and delivery. I would probably give it 3 stars, which is relatively neutral for me (or leave it altogether unrated). If it were thumbs up/down, I would probably have to leave it unrated completely. Maybe that's an improvement, but the more discussion I see around this, the more I realize the futility of single-axis metrics for things like this, until and unless you're asking a single question "Did you enjoy this movie" is a much different question from "is this a good movie" -- and I don't really know which one I've been answering on Netflix all these years past.
      • bluntfang 10y ago
        I think you are an anomalous user. I'm not sure why you're OK with rating 1-5 stars but not a thumbsup or down. I think you're just upset because they are changing.
• yitchelle 10y ago
  I have always wanted to get ratings from a selected group of viewers who has similar tastes as me. For example, I like SciFi movies that involves planetary travel, ie Star Trek or similar style movies. If I can get ratings from folks with similar interests as well, then the rating system would make sense for me. Otherwise, it is still a hit and miss for me.
  • apetresc 10y ago
    That's exactly how their recommendation system works. If it just surfaced the highest-rated movies globally, everyone's recs would be identical. Collaborative filtering implicitly weighs the opinions of those similar to you more highly.
• nrki 10y ago
  Should be "five star-rating"..."five-star rating" indicates they are replacing ratings of five stars :)
• mrmondo 10y ago
  Life is not black and white, how you feel about something isn't terrible or brilliant every time. This is idiotic.
• kochandy 10y ago
  Whenever I would browse the Netflix reviews it appeared most people voted either 5 stars or 1 star anyway.
• DaveWalk 10y ago
  Question to the Machine Learning folk: are five-star ratings "better" than thumbs up/down, or is it just a matter of algorithm design? I know ratings/prediction has long been studied by the MovieLens.org scientists.
  • dredmorbius 10y ago
    The technical term for a multi-perference rating is a Likert Scale. There are different schools on this -- an even or odd number of options (even "forces" reviewers to indicate some preference positive or negative, odd allows fence-sitting), how to compare different raters' preferences, etc. I'm finding interesting the rating system I've created and am trying to apply with some consistency to my Pocket archive of a few thousand items. Nominally it runs from 0 to 5, though I may reserve a 6 for an absolutely mind-blowing piece. A 0 is a net negative: you are less informed for having read it, it reduces teh intelligence of its reader. A 1 is, generally, a simple noting of some event. A 2 should be a general news story, without strong insight. A 3 is a news or general interest story with strong insight, or a typical scientific paper, or an undistinguished book (generally nonfiction). A 4 is a particularly good scientific paper, or a typically well-thought-out book. A 5 is a document which establishes a fundamental idea or field. Claude Shannon's original paper on information theory, say. I don't think I've run across a 6 yet, but that might be a work which ties together two or more previously unrelated fields into a common theory. My problem has been in assigning far too many '3' class articles. I've already carved out a list to re-assess and downgrade if appropriate. I'd also like to be able to report on the numbers for each classification, though Pocket's utter lack of quantitative reporting (I cannot even state how many articles I've collected in total) stymies this. I am ... increasingly dissastisfied with Pocket as an information management tool.
  • refrigerator 10y ago
    I think the fact that Netflix saw a 200% increase in "thumbs" ratings vs "stars" in the A/B test makes it much better from a machine learning perspective, even if it might at first seem like the data is of "lower quality". One of the biggest problems for recommender systems is that the data is extremely sparse - most users will have only rated a tiny proportion of films, and most films will only have been rated by a tiny proportion of users. This is just my opinion now, but having studied recommender systems in a decent amount of depth, I don't think the design of the algorithm will need to change. The current techniques that provide the best results use matrix factorisation to simultaneously learn characteristics of films and how much each user likes each characteristic. My intuition would be that the algorithm can learn a lot more about a user from 3 up/down-ratings than from 1 star-rating, and the only reason Netflix are doing this is so that they can provide better recommendations so it's almost certainly the case. TL;DR: All else being equal, 5-star ratings carry more signal for any machine learning algorithm, but the fact that thumbs up/down will result in 200% more data is a lot more significant than the delta in the signal.
• Houshalter 10y ago
  Normalize star ratings. If something has an average rating of 4 stars, but four stars is in the lowest 10 percent, it should be given 1 star. This solves the classic issue with star ratings where anything under 5 stars is below average. Personally I use star ratings badly myself. I only ever rate titles 5 stars. But that's because I mostly only watch stuff I am fairly sure I am going to like.
• andy_ppp 10y ago
  You could come up with an algorithm that instead of showing an average of 5 stars, biases the weighting towards people who rate things in a similar way to you. We all have an internal set of assumptions about what a 5 star rating system means to us; this is a case where the filter bubble leads to better understanding. People would then see the ratings they expected rather than a mix of different methodologies averaged?
• openasocket 10y ago
  I don't have particularly strong opinions about this change, I don't actually rate many things that I watch (I think the recommendation system can go mostly by what I watch). I think the fundamental problem with different rating systems is that they are all one-dimensional. Maybe a two-category system, like 1-5 stars for "enjoyment" and another 1-5 stars for "quality". Or a 1-5 rating paired with a label, like "campy" or "serious", to prevent apples-and-oranges comparisons.
• ggregoire 10y ago
  I wish we could have the IMDB/RT ratings directly in Netflix and a page with all the movies sorted by those ratings. There are articles on Internet like "top 100 Netflix movies of the month" but it's almost always for the US Netflix.
  • eric_the_read 10y ago
    Unfortunately, A Better Queue is dead, but you can use NEnhancer (https://chrome.google.com/webstore/detail/nenhancer/ijanohecbcpdgnpiabdfehfjgcapepbm?hl=en https://chrome.google.com/webstore/detail/nenhancer/ijanohec...) in Chrome to add Rotten Tomatoes and IMDB ratings to Netflix.
  • deleted 10y ago
    [deleted]
• sergiotapia 10y ago
  Amy Schumer's comedy special bombed so hard they're changing their ranking system? Total failure.
  • minimaxir 10y ago
    This system would arguably be more unfavorable to Netflix/Amy Schumer.
    • orthecreedence 10y ago
      Right. The original thumbs down means being put to death.
  • LordKano 10y ago
    If/When Dave Chappelle sets a new record for 5 Star reviews, perhaps they'll consider reversing this decision.
  • crikli 10y ago
    She's blaming the boogeyman of the "alt-right" for the poor reviews. Yeah, it's got to be that, not that she has no actual material beyond "I'm a girl saying gross things, now laugh at me." I'd rather have seen the money wasted on her spent boosting the publicity of female comics that are actually, you know, talented. Start with Tig Notaro.
  • cnnsucks 10y ago
    I read that headline too, but the actual story didn't provide any evidence to support it. Schumer's 'Leather Special' was your usual tedious progressive lecture disguised as comedy with a few laughs and a lot of porn, and was modded appropriately, but that doesn't connect it to the Netflix rating change as far as I can see.
• chrisper 10y ago
  After almost 5 years I finally cancelled my Netflix subscription. I just realized I was watching more movies / shows on Amazon, Youtube, or any of the other paid ones more than Netflix. I used to enjoy it a lot more back then.
  • Markoff 10y ago
    i would recommend FMovies.se or 123movies.gs
• ed_balls 10y ago
  Netflix recommendations are quite bad. You've seen this standup? How about watching it again a day later.
• mr_tristan 10y ago
  At no point do I recall Netflix promoting this rating system as anything but way to get better matches. The fact that so many comments are about losing a system for criticism, just confirms that it was the wrong metaphor to use in the first place. Perhaps they could bring back a "critical review" mechanism, but I'd guess it wouldn't affect your matches at all. And probably be mostly unusable on a TV anyway.
  • throwaway91111 10y ago
    The issue is that their digital library is vanishingly small. I swear half the tv shows on there are Netflix original content from the last five years.
• shdon 10y ago
  I get why they do this and it'll probably work out fine, but for me personally, it's going to mean I will rate a lot fewer movies/shows. In the past, I've rated pretty much every single thing I've ever watched on Netflix. On YouTube, I hit the thumbs-up or thumbs-down buttons on a very small fraction of the videos that I watch. That's just because there is no middle ground (and no 2 or 4 star ratings either). There's plenty of stuff that I enjoyed, but didn't love, plenty of stuff that I disliked, but didn't really hate.
• wj 10y ago
  I'm still disappointed they removed their friends feature. I valued my friends' ratings more than the ratings of the Netflix user base as a whole. I have noticed that Netflix tends to be more accurate with the "users like you" star rating than the whole user base ratings. I no longer see a sum of my ratings but I believe it is well over 2,500. I know others that have rated a lot more (when there was the friends feature you could see your friends' number of ratings). Because of my number of ratings I felt that Netflix did a pretty good job with recommending me content. I will be sad to see this go as I definitely refer to star ratings when adding content to my queue. Others have mentioned that the two factors of quality and enjoyment make the star rating more valuable and I agree. The only time I remember it breaking down for me was the film Rachel Getting Married (though I am sure there were others). I couldn't stand Anne Hathaway's character to the point that I gave it one star but at the same time I recognized that she gave a really strong performance in what was probably a good film. Are they converting ratings to thumbs up/down? What does a three-star rating convert to? Those are typically movies that I enjoyed but wouldn't rewatch or recommend to others.
  • Markoff 10y ago
    why not use TMDb or IMDb for ratings? you can export your ratings from IMDb and import them at TMDb which has also discussions for individual movies
  • fmihaila 10y ago
    > Are they converting ratings to thumbs up/down? What does a three-star rating convert to? It would convert to no rating.
• minimaxir 10y ago
  It's about damn time. 5-star rating systems are broken. A 1/2/3-star rating is effectively a dislike, and a 4/5-rating is effectively a like. In the big data sense, a deviation from a 1-star for dislike and 5-star for like is statistically meaningless. (and this is universal; star ratings on Amazon Products and Yelp Locations have the exact same distribution). The interesting part of this is what Netflix gains from the change, since their recommendation algorithms will become less granular. Maybe they came to the same conclusion?
  • wwarren 10y ago
    > star ratings on Amazon Products and Yelp Locations have the exact same distribution Source?
    • minimaxir 10y ago
      Glad you asked! http://minimaxir.com/2017/01/amazon-spark/ http://minimaxir.com/2017/01/amazon-spark/ http://minimaxir.com/2014/09/one-star-five-stars/ http://minimaxir.com/2014/09/one-star-five-stars/
      • e12e 10y ago
        I actually thought Randy Farmer touched on this in his Google talk on "Building Web Reputation Systems" - but maybe I saw it somewhere else, years ago. Like this: http://www.lifewithalacrity.com/2006/08/using_5star_rat.html http://www.lifewithalacrity.com/2006/08/using_5star_rat.html At any rate I thought it was common knowledge that 5-point ratings have issues - especially as a "social rating system"?
        • minimaxir 10y ago
          Unfortunately, as other comments on this HN submission demonstrate, the 5-star system is sacrosanct and any change, even those made a decade later, is bad.
  • Markoff 10y ago
• coding123 10y ago
  I'm going to guess they looked at things and found that 80% of the ratings were either 5 stars or 1 star. (edit - not that this is a good thing.. I'm guessing they could have carved out the 2-4 star ratings and did its own data analysis that probably provided much more nuanced information)
• ultrasandwich 10y ago
  I've found some solace using Mubi, which has a pretty devoted film fan base that really puts thought into written reviews + star ratings. I almost always read a handful of these before following-through with a viewing. They also have a nice 1-in-1-out movie per day system which lets them theme films by event (Cannes, director birthdays, etc). Not for everyone, but it's out there.
  • dasil003 10y ago
    Glad to see MUBI mentioned here. I spent 9 years of my life building it as co-founder and tech lead. My biased view is that if you are sick of bland streaming selections then you should really support MUBI. The more subscribers we can get the more we are able to fund great unique content. Our approach is fundamentally different from the data-driven, mass-appeal approach that tends to sand all the corners off of Netflix/Amazon content. Currently most MUBI content is licensed with a few exclusive releases sprinkled in, and the US selection is not as good as the UK, but as the subscriber base grows it will get better and better. Netflix et al are a volume game and the unit economics don't work out for them to purchase great indie films. If you care about great film as opposed to episodic content, MUBI is the streaming service that is pushing the envelope there.
    • ultrasandwich 10y ago
      Wow, how random that you are on here too, haha. Thanks for building a great product. It addressed pretty much every gripe I had with the bigger streaming services.
• lukeschlather 10y ago
  Netflix's star rating system is really poorly designed. Of course people clicked on a thumbs up/thumbs down widget more. I used Netflix for months before I even noticed the star rating was interactive. I wish they would invest in a quality star rating system like Amazon's, rather than dumbing down the UI.
• fmihaila 10y ago
  The way I understand their explanation is that users tend to view the act of rating as a stronger endorsement than it actually is. In other words, users think their ratings are more predictive of what they would enjoy in the future than the implicit signals that can be derived from actual behaviour. Netflix concluded that it's better to boost the weak(er) signal by encouraging users to rate more. They will continue to use it on top of the stronger implicit signals to provide personalized ratings, which they will display as a percentage match as long as it's greater than 50%. So their recommendations may actually improve.
• Fej 10y ago
  Interesting. YouTube made the same switch some years ago, I wonder how it's worked out for them.
• Pxtl 10y ago
  In most cases I would agree with the simple up/down rating, because we're seeing the lunacy of "less than 5* is failure" in various platforms. But in the case of Netflix where it's primarily being used to manage my own personal preferences on content, some nuance of "more like this" and "well I kinda enjoyed it but it was badly flawed" and "meh" and "never ever show this to me again" is useful.
• cpeterso 10y ago
  I already push my Netflix ratings higher or lower than I really feel just to more heavily influence Netflix's suggestions. Otherwise, Netflix thinks I would give a 3.5 stars to pretty much all movies, which is not helpful to me. That said, a simple thumbs up/down doesn't allow me to tell Netflix which movies were not bad and which to never show me again.
• Markoff 10y ago
  so how exactly i distinguish between liking something a little and loving something? this seem like pretty retarded decision at least we have TMDb where we can rate and discuss movies, unlike IMDb or Netflix
  • always_good 10y ago
    I think you should distribute your TMDB shilling across a few more accounts, mate. I actually recognized your username from one of your shills I found so bad that I linked it to a buddy of mine. Just some of your recent posts on HN: > why would i participate on dumb backup of database, > done without my consent (OP stole 14 years of my posts), > when I can go to proper movie database with discussions, > imported ratings and watchlist from IMDb, called TMDb (themoviedb.org) Shill Tip: don't link to your shill target when it's googleable like "TMDb" is. It makes it look like you are just dropping a name everyone should know. Love the fake outrage about having your comments "stolen", as if anybody really cares. > he doesn't and he illegally copied 14 years of my posts which > i don't like especially since I moved to TMDb which is proper > movie database where you can also import your IMDb ratings > and watchlist and each movie there has own discussions Another: > i would recommend checking TMDb which launched discussions > for each listed title just yesterday, so don't expect there > will be much content, but at least each movie or TV show had > its own page and own forum, much more neat than reddit Consider leveling the load across more accounts though, and mix in some different strategies. ctrl-f for "TMDb" on each page of your comment history and you'll see the problem.
    • Markoff 10y ago
      it's related to discussion for people who prefer normal star ratings instead of retarded thumbs up, so not sure what's your problem, also not sure what's your definition of shill, but I doubt it's someone who doesn't even bother to provide link for his potential victims and who is just regular user of some website where he was forced to move from IMDb not sure where you got some fake outrage from my post, when some thief is stealing 14 years worth of my content from IMDb judging by your language you are most likely shill from some competitors failed forums which are already forgotten as IMDb alternatives everyone feel free to go through my history to see if I am some account made to spam with TMDb as this expert imply
• oculusthrift 10y ago
  this makes sense to me considering most people only leave bother to use the extreme star amounts anyway. anyway, you're kind of a sap if you take netflix ratings seriously and don't use something like rotten tomatoes
• erikb 10y ago
  I can only wonder why. 5 stars is the normal rating system for movies, and there's a huge difference between each star, at least for me. I basically rate like this: unbearable-cannot-finish (1), bad (2), okay-for-background-noise (3), what-I-expect (4), awesome (5). Of course, some movies are somewhat between 3 and 4 or somewhat between 4 or 5. But if you just give me thumbs up or down there are like 80% of movies I cannot rate at all, because they are neither.
• neuronexmachina 10y ago
  Uh oh, now they're going to need to re-film the Black Mirror "Nosedive" episode (s3e1) using thumbs-up/down instead of 5-star ratings...
• rebootthesystem 10y ago
  What Netflix needs to do is improve their content, not screw around with ratings. The quality of the content has definitely deteriorated over the last few years. It is almost impossible to find good material to watch. I have found myself wanting to watch a good movie only to give up after searching for half an hour. Amazingly enough non-premium TV channels have better movies than Netflix, so we record them and skip the commercials. Frankly, the only reason we keep Netflix is because there are a couple of kids TV series our kids like. We are using them to improve their Spanish language skills. If it weren't for that we would have cancelled a long time ago.
• relaytheurgency 10y ago
  There's always a better option if you like metrics: instantwatcher.com sorts Netflix/Amazon content with a variety of external metrics like IMDB score etc. It's pretty great if you are just browsing (rather than seeking recommendations).
• sugarygrind 10y ago
  My Previous ratings already in Netflix, was the reason for being there. This dumbed down version of up/down voting is ridiculous. Most companies become sophisticated. Netflix is going backwards in time. All in all this is a RIDICULOUS move. I wish I could download all my previous ratings before they remove them!
• derrekl 10y ago
  I've been in favor of up/down for years over x out of 5 or x out of 10. The core reasoning is: it captures your overall emotional response to something very succinctly. Sure you might hate X more than Y or like A more than you like B, but you'll return, revisit, recommend all the things you like and not do so for the things you don't like. When you can rate out of 5 or 10 you get thrown into a sort of tizzy trying to rationalize a system of what is 1 star, 2 stars, etc. Do you subtract a star because you think something is overpriced even though you like it? Do you subtract a star because it's hard to get it (say slow delivery) even though you like it? Basically you want to start to categorize every facet of a product/service which becomes complicated quickly. Thumbs up/down avoids all that and I think over many hundreds or thousands of reviews the ratio of up to down will provide a good probability indicator of liking something or not.
  • echelon 10y ago
    This benefits those that want to data mine such data, not those that want to meticulously catalog their interests. This is one of my biggest gripes with music services such as Google Play. Thumbs up/down ratings are great for Google, but suck for me, the listener, as I wish to construct a detailed, per-song breakdown of my musical tastes.
    • derrekl 10y ago
      I don't believe up/down should replace written reviews, but neither does an x out of 5 or 10 system. Written reviews are the place for that in depth analysis. That said I'm not familiar with Google Play and don't know if they have written reviews or not.
  • michaelmrose 10y ago
    If they captured and displayed to the user how many users watched it vs how many ups and downs it might give users a better indication.
    • derrekl 10y ago
      That's a pretty interesting idea. It would indicate a sort of apathy rating. Not that a product or service with a high apathy rating is bad - might actually be good for certain things.
• vthallam 10y ago
  This is what even Google does now for the movies and tv shows on the search. If you are logged in while searching for a movie or TV show, it gives thumbs up or down and shows the approval rating in total.
• peterjlee 10y ago
  I've always liked Rotten Tomatoes ratings more than IMDB star ratings. Rotten Tomatoes rating is basically, what percentage of critics gave a thumbs up.
• garrickvanburen 10y ago
  I've been a Netflix subscriber for a decade. To me, 3-stars means: this movie has all the characteristics of something you'd enjoy - but we don't think you will. Unfortunately, it's been more and more challenging for me to find 4- & 5- star stuff in their streaming interface. Perhaps the thumbs-up will help expose those things, or at least help me hide the thumbs-down things.
• drivingmenuts 10y ago
  I don't rate things on Netflix (and rarely on any other service). If I made it thru the movie, I probably enjoyed it; if not, I didn't. What I would like to see is a collection of all available movies by genre, and I'll pick what I want to watch. That's it - I don't need or want recommendations. In fact, if it's recommended, my first thought is "who's getting paid for that?"
• yashksagar 10y ago
  Why not keep the rating system, and have an additional thumbs up/down rating? That way you get to vote on the 'objective' quality and the 'subjective' enjoyment factor of the content.
• Viper007Bond 10y ago
  YouTube got rid of star ratings back in 2009 for similar reasons: https://techcrunch.com/2009/09/22/youtube-comes-to-a-5-star-realization-its-ratings-are-useless/ https://techcrunch.com/2009/09/22/youtube-comes-to-a-5-star-...
• eiopa 10y ago
  I understand people who want more expressiveness in their reviewing, but the reality is that the 5-star system just leaves too much ambiguity. It's like how in Uber/Lyft, most people just default to 1/5 star ratings. In that case, the average rating a driver has to maintain gets skewed to a a pretty high number, like 4.5, and people who think "oh, this was a pretty good ride! 4 stars!" end up unintentionally boning the driver.
• jschwartzi 10y ago
  What they should be asking is "would you watch another movie like this?"
• wiseleo 10y ago
  Great. I can't stand 5-star systems. Anything other than 5 stars is negative signal.
• true_tuna 10y ago