4 ms·
If you have many reviewers, you don't need to get more information than thumbs up or down from each person. How much people thumbed it up is a good indicator.
by kartickv 10y ago
If you have many reviewers, you don't need to get more information than thumbs up or down from each person. How much people thumbed it up is a good indicator.
Star-rating can be too much detail, anyway: if you're comparing two shows, and one has a higher fraction of five-star ratings, but the other has a higher fraction of ratings that are four or above, which is better? Star ratings can be too much detail and cause confusion.
If you want more detail from each person, you can ask specific questions with a yes/no answer, like, "Were parts of it boring?" or "Was it violent?" That's probably better than star ratings.
- surge 10y agoThat and Netflix has far more data that indicates likability to a far greater degree, such as how many people finished watching the show, over what period of time did they finish watching the show, after what episode are people more likely to get hooked and finish the series or binge on it, etc.
- k-mcgrady 10y ago>> Star ratings can be too much detail and cause confusion The iOS App Store is a good example of this. An unbelievable number of people, for some reason, thing 1 star = great and 5 star = bad.
- X86BSD 10y agoHaha yes! Where the hell do these people learn that?? The other thing that annoys me with ratings in general, specifically places like amazon, is when people pan a perfectly good item because of some third party factor. "Item was great, but it came in a beat up box from the post office, I'm giving it one star." WTH does someone drop kicking your box have to do with the product?
- jandrese 10y agoI have to think that some of those reviews are just trolls. Bad reviewers will always be part of the system. You have to expect it. Netflix has tons of reviews that go "I turned this off before the opening credit sequence finished, worst movie ever!!!! -- 1 star".
- always_good 10y agoIIRC there was a blog post on HN a while past that complained about it, and it seemed to be people that went with 1-star so that their review was potentially more visible.
- TallGuyShort 10y agoThe other thing you see in ratings: "I've patronized this business every day for 5 years and always loved it. Then the service was kinda rude the other day. 1 star. Can't believe they stay in business."
- sterex 10y agoVery true. There could be socio-cultural reason as well. If a movie hurt the sentiment of a particular section of the society, you can expect 1-star ratings even from people who weren't going to watch the movie anyway.
- svachalek 10y agoSince the early days of Yelp I learned to read restaurant star ratings: 3.5: Good place, check it out 4.0: Great place, impress your friends < 3.3: Double check your health insurance first > 4.2: Fake
- duderific 10y agoHaha totally agree. I've often wished they would put the actual decimal rating (4.12, 3.56 etc.) because we have no idea what their rounding strategy is. Does 3.99 become 4? How about 3.75? 3.51?
- philh 10y agoI do find IMDB's ratings breakdown fascinating to explore, though. For example, you can look at the 50 Shades films or A Dog's Purpose or (especially) Gunday and have a pretty good idea that lots people are giving them bad ratings without having seen the films. (Their weighted average ratings are supposed to combat that, but they don't seem to be doing a very good job.)
- kartickv 10y agoInteresting point. Would polarised reviews make you less or more likely to watch a film (or download an app or whatever)? In other words, two items with the same average, but different distributions.
- sterex 10y agoActually, Netflix is poised in a better position to filter between people who just rated the item vs people rating it AFTER viewing it (for a minimum duration). The latter class of people will have have more weight attached to their voting.
- philh 10y agoThat's true, but it doesn't sound as fun to poke around in. (Obviously, "collect a dataset that philh finds interesting" should not be a business goal of Netflix. I'm not suggesting this is a reason to keep the star ratings. It's just a thing that I like about having ratings more detailed than yes-no.)
- subroutine 10y agoThe goal isn't to broadly determine which movies are good and bad; the goal is to make accurate predictions about which titles a particular person will enjoy. One good way to accomplish this is to pool users into cohorts of similar tastes, then suggest movies enjoyed by their cohort but not yet watched by a given user. The question is whether doubling the frequency of rating a title is more valuable than a slightly more nuanced understanding of what each person likes. 'More valuable' meaning they can build you a better cohort from which to make title suggestions.
- kartickv 10y agoEverything you've said makes sense, but I don't see it being contradictory to what I said. If you decide you want a more nuanced understanding of what a given reviewer likes, I think it's better to evaluate different dimensions (Was it funny? Was it violent? etc) than shades of grey* of the same one. * pun not intended :)