3 ms·

> I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities. This is only true when those star ratings ar

by curun1r 10y ago

> I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities. This is only true when those star ratings are taken out of context. Individual reviewers can have completely coherent star ratings that are very useful for making determinations. The golden age of Netflix discoverability, for me, was back when they added social networking features. I connected my account with many friends and family and was really able to get a sense for everyone's taste in movies. When it came time to discover new movies/tv to watch, I'd open up the ratings history for someone who's tastes I felt aligned with the mood that I was in. Some of my friends would only rate high quality content with 5 stars, others would typically rate mass market, mindless entertainment with 5 stars and others tended to include a lot more 5-star reviews within a certain genre. Over time, I got a feel for that and could use it to my advantage. But when you remove that individual context and only aggregate 5 star reviews across a massive population who all view the 5-star scale in their own way, you get that uselessness that you talked about. Some people may reserve 5-star ratings for truly extraordinary content, but Netflix doesn't weight those ratings any more highly than the 5-star ratings of someone who rates nearly everything as 5 stars. This makes the ratings that Netflix displays meaningless. But that's only because Netflix's current methodology makes them meaningless, not because they're inherently meaningless.

bcbrown 10y ago
> Netflix doesn't weight those ratings any more highly than the 5-star ratings of someone who rates nearly everything as 5 stars. Is that fact or speculation? It's a common recommendation technique to find the mean rating for a person, and use that to normalize. Normalized score = (actual_score - mean_score + 2.5) for a 5-point scale, for example. I have no insight into whether or not Netflix does something similar, but I'd be surprised if they don't.
- kartickv 10y ago
  Doesn't normalisation assume that a person who rates everything five-star has low standards? It may just be that that person rates something only if they're really (un)happy. Imagine two people watched 100 movies, and assessed them the same way, say that 20 are five-star, 20 are four-star, 20 are three-star, and so on. The first person rates all 100 movies, but the second one has a habit of writing a review only if he considers a movie excellent. Then your algorithm will wrongly discount the second person's ratings. Is this true in the real world? Have you found different people to have different habits, like rating everything vs rating only good movies vs rating only bad movies?
kartickv 10y ago
First, few people will put in effort into understanding their friends' psychology and habits to use Netflix effectively. We shouldn't optimise for such users. Second, I don't know whether individual ratings should be normalised as you say, but the final rating that's displayed should be normalised. For example, the app stores should have 20% of apps rated five stars, 20% four stars, and so on. This will make the ratings more objective and easy to understand. If the average rating is 4, then an an app rated 4 is merely average, which is counter-intuitive. (Perhaps normalised within a top-level category, like Productivity and Games)

by curun1r 10y ago

bcbrown 10y ago

> Netflix doesn't weight those ratings any more highly than the 5-star ratings of someone who rates nearly everything as 5 stars. Is that fact or speculation? It's a common recommendation technique to find the mean rating for a person, and use that to normalize. Normalized score = (actual_score - mean_score + 2.5) for a 5-point scale, for example. I have no insight into whether or not Netflix does something similar, but I'd be surprised if they don't.

kartickv 10y ago
Doesn't normalisation assume that a person who rates everything five-star has low standards? It may just be that that person rates something only if they're really (un)happy. Imagine two people watched 100 movies, and assessed them the same way, say that 20 are five-star, 20 are four-star, 20 are three-star, and so on. The first person rates all 100 movies, but the second one has a habit of writing a review only if he considers a movie excellent. Then your algorithm will wrongly discount the second person's ratings. Is this true in the real world? Have you found different people to have different habits, like rating everything vs rating only good movies vs rating only bad movies?

kartickv 10y ago

First, few people will put in effort into understanding their friends' psychology and habits to use Netflix effectively. We shouldn't optimise for such users. Second, I don't know whether individual ratings should be normalised as you say, but the final rating that's displayed should be normalised. For example, the app stores should have 20% of apps rated five stars, 20% four stars, and so on. This will make the ratings more objective and easy to understand. If the average rating is 4, then an an app rated 4 is merely average, which is counter-intuitive. (Perhaps normalised within a top-level category, like Productivity and Games)