4 ms·

The problem with star ratings, and this has plagued star rating systems since their introduction, is different people have different rules. There's the 1/5-crow

The problem with star ratings, and this has plagued star rating systems since their introduction, is different people have different rules. There's the 1/5-crowd where they either love it or hate it, and the 3-everything crowd where everything's just meh. There's also those that "save" their 5s for their absolute favourites, and only hand out 4s save for special occasions. If they had a system like "like/dislike" for preference data plus an additional "I hate this, do not show anything like it ever again" and another akin to "I love this, I will cancel my subscription if you don't renew it" you'd get better data. Some rating systems have a "super like" which you could dole out in limited quantities. Maybe Netflix needs one!

overcast 9y ago
I don't see that as a problem, since the rating system was tailoring content for YOU. So whatever your rule set you decided to use for your rating, you would continue to do so in the future, and it would recommend based on that. That star rating you used to see for a movie, that was what Netflix determined YOU would rate the movie if you saw it. Not what everyone else cumulatively rated it.
- astrodust 9y ago
  The problem is normalizing rating data that's all wonky like this to try and tease out any meaningful patterns.
entropie 9y ago
Well these days with ML there should be a way to get around the 5 star problem. I recently learned, think it was on the HN frontpage, that netflix uses ML to show different preview images of movies to difference audiences. (Like, iam a male and see more womans and more skin)
ssalka 9y ago
It's tough to capture subjective features of data - like what a user means by rating something 4 stars - since the interpretation of each star value could mean something different to each user. But there is a (somewhat) simple solution - instead of presuming to know how to map stars to thumbs up/down for every user, let each user specify how the mapping would occur. The easiest way to go about this would be to just put 5 stars on the screen, and a toggle below each to indicate what values should map to what (eg 1,2 = thumb down; 3 = no rating; 4,5 = thumb up), and then apply the thumb ratings to everything the user has previously rated with stars. Maybe have the above settings as the default, and let the user tweak to their liking (eg, make 3 = thumb up). Boom, everyone gets their ratings migrated with little to no hassle.
- BoiledCabbage 9y ago
  Which again doesn't solve the problem of people being more likely to rate high-brow oscar type things with more stars even if they didn't really enjoy watching them, and more hesitant to rate light or slapstick things the same way. It's the same reason why so many people's to watch queue is filled with documentaries and profound movies they feel they should watch, but most of the time when they turn it on they flip on a simple TV show. Whether we want it to or not, like in hotels 5 stars implies a level of quality, not just a level of like. In a scenario, someone was looking for a silly light-hearted movie and I got a silly light-hearted movie. They don't think it deserved "5-stars" so they give it 3 or 4, but in truth it was exactly what they wanted. They want to ensure their data is a pure a signal as possible: did you enjoy it - and having people map things to star ratings adds in noise.
  - overcast 9y ago
    That only spoils their experience. If they want to rate like dummies, then that is their tailored experience. With five stars, the thumbs up / thumbs down people can continue to vote either a 1 or 5. For the rest of us civilized viewers, we can more accurately categorize stuff.

overcast 9y ago

I don't see that as a problem, since the rating system was tailoring content for YOU. So whatever your rule set you decided to use for your rating, you would continue to do so in the future, and it would recommend based on that. That star rating you used to see for a movie, that was what Netflix determined YOU would rate the movie if you saw it. Not what everyone else cumulatively rated it.

astrodust 9y ago
The problem is normalizing rating data that's all wonky like this to try and tease out any meaningful patterns.

entropie 9y ago

Well these days with ML there should be a way to get around the 5 star problem. I recently learned, think it was on the HN frontpage, that netflix uses ML to show different preview images of movies to difference audiences. (Like, iam a male and see more womans and more skin)

ssalka 9y ago

It's tough to capture subjective features of data - like what a user means by rating something 4 stars - since the interpretation of each star value could mean something different to each user. But there is a (somewhat) simple solution - instead of presuming to know how to map stars to thumbs up/down for every user, let each user specify how the mapping would occur. The easiest way to go about this would be to just put 5 stars on the screen, and a toggle below each to indicate what values should map to what (eg 1,2 = thumb down; 3 = no rating; 4,5 = thumb up), and then apply the thumb ratings to everything the user has previously rated with stars. Maybe have the above settings as the default, and let the user tweak to their liking (eg, make 3 = thumb up). Boom, everyone gets their ratings migrated with little to no hassle.

BoiledCabbage 9y ago
Which again doesn't solve the problem of people being more likely to rate high-brow oscar type things with more stars even if they didn't really enjoy watching them, and more hesitant to rate light or slapstick things the same way. It's the same reason why so many people's to watch queue is filled with documentaries and profound movies they feel they should watch, but most of the time when they turn it on they flip on a simple TV show. Whether we want it to or not, like in hotels 5 stars implies a level of quality, not just a level of like. In a scenario, someone was looking for a silly light-hearted movie and I got a silly light-hearted movie. They don't think it deserved "5-stars" so they give it 3 or 4, but in truth it was exactly what they wanted. They want to ensure their data is a pure a signal as possible: did you enjoy it - and having people map things to star ratings adds in noise.
- overcast 9y ago
  That only spoils their experience. If they want to rate like dummies, then that is their tailored experience. With five stars, the thumbs up / thumbs down people can continue to vote either a 1 or 5. For the rest of us civilized viewers, we can more accurately categorize stuff.