5 ms·
> 10-star rating system (More nuance than 5 stars) Does one really get anything meaningful out of saying this was a 6-star book vs a 7-star book? Personally I
by magicalhippo 2y ago
> 10-star rating system (More nuance than 5 stars)
Does one really get anything meaningful out of saying this was a 6-star book vs a 7-star book?
Personally I think 4 levels is sufficient. Either it's rather bad, not bad but not good, good but not great or it's great.
Anything beyond that will have to be written in words.
- nerdponx 2y agoNo, and that's why Netflix switched to thumbs up/down instead of 5 stars.
- jfengel 2y agoAnd they mostly ignore that. They have access to what you actually watch, rather than what you say you like. They know what you start, what you finish, and how quickly you jump on a new one. It doesn't matter how many times you down vote Mexican soap operas or singing talent shows. If you keep watching they're gonna keep suggesting them.
- watwut 2y agoAnd result is stupid. It is pretty much impossible to discover something new on Netflix. It puts you into a quick box and no matter how much I try to find a comedy, I can't because it decided the same crime shows are my thing. Netflix recommendation system just does not work. It does not allow me to find movies I can like, it allows me to see the same thing I seen once before. I am not in the mood for serious documentary evey day of course, it takes more concentration. But when I am in mood for one, I should be able to find it.
- stevage 2y agoYeah and I hate this scheme. Facebook keeps showing me clips that I do not want to watch and yet fined myself drawn to watching like some compulsive behaviour. Urgh.
- skeeter2020 2y agoIt's debatable this was the motivation for Netflix. More stars ==> more nuanced ==> more qualitative reviews ==> much more effort and time to decide if you should consume. Netflix is long(er) form TicTok and wants to optimize for continued consumption without friction. I wouldn't be surprised if they drop ratings all together and only offer a personalized AI curator stream. They could do this just based on viewing time and engagement and avoid even the minor disruption of "up/down". Don't make the sheep think.
- vasanthk1125 2y agoThey switched because it took more effort from users to rate on a 5 vs just saying good or bad. Because Netflix is a streaming service, casual users don't want to put in that much effort when they're chilling. Tracking is fundamentally different, where you go in with the expectation of organizing your library.
- lostmsu 2y agoLast time I saw a movie there there was "super thumbs up" in addition to normal one. For clarity I'd replace rating systems with "was it a good spend of my time?" yes/no question. Then just show percentages. Could not be clearer.
- hansvm 2y agoI could see "proportion of 10s" vs "proportion of 5s" (the max score in either case) being meaningful.
- bmacho 2y agoRating is a mess. Still I think people are used to express their opinions on a 0-10 scale, and they will appreciate if they can just keep doing that.
- dexterdog 2y ago0 to 10 is a mess as well because some people give 10s very often and others only give a 10 for the very best things. And of course the bots all rate everything 1 or 10.
- krykp 2y agoI was once playing around with a 3-point ranking system. Think thumbs-down, thumbs-up, and double-thumbs-up. The thumbs up and down would basically function as expected, while the latter would be weighted heavier for the recommendation algorithm. Basically a `recommend me more of this, this is high quality content` action. There is a general problem with a 5 or 10 star voting system, consider a [malicious] user who only gives a 1 or 10 star vote, thus ending up with more voice than one that votes in the range of 4-6 which would be what the majority of the content deserve. Therein lies another problem too, while the scale would imply 5.5 to be average [out of 1-10 with no 0 option], most people tend to consider 7-7.5 to be average instead, there's a very natural bias on the scale. This idea isn't actually uncommon however, as platforms tend to work with a thumbs-up, thumbs-down, and a `favorite` action of sorts. Some platforms tend to respect favorites in recommendations and some don't. I have found that YouTube doesn't care all that much about my... let alone favorites, it doesn't even care about my votes. TikTok however did this well, I had downloaded it one day and at the end of the day my feed consisted of neat programming tricks and lessons on color theory. Which kind of revealed something my own prejudice too, as I had expected TikTok to show me the worst content and it was the platform that respected my choice the most. That said these things change a lot so it wouldn't surprise me if the same test shows the opposite results a year from now on.
- philistine 2y agoWhat you're describing is fully observed in education. The scale of your evaluation has to have an odd total of numbers and a limited number of choices. 1-2-3, for example. That means that each digit sends a strong signal. 1-2-3-4 means the 3 is the non-controversial choice. Average, so to speak, and you don't want average in your evaluation. You need to adequately grade stuff here, and giving most things an average grade is a weak signal that prevents you from differentiating. When you grade by competency (not by knowledge), you also assign a written description for each grade. That helps a lot. I think those platforms are keenly aware of those facts I just described, and are trying to boil them down to simple actions for users, that impart large signal, and that respect the cultural norms of evaluation. That's why Letterboxd has a 5-star with half-stars rating system, but also has a like button.
- vasanthk1125 2y agoIt just feels satisfying to me—organizing my library neatly by rating, based on how much I like each book. A four-level system just doesn’t have the same impact. Personal preference, I guess.
- ijustlovemath 2y agoIf you added half stars and stuck to a 5-star system, there's less user surprise for functionally the same thing.
- vasanthk1125 2y agoI did go back and forth on this. I just like 10 stars better. IMDB, MAL, Mangadex all have 10 stars. So it's not really uncommon. But yeah, if a lot of people feel 5 is enough, we'll just return it back to the 5 star system.
- jbrr25 2y agoAnilist let's you switch your rating system. You can do 5 stars, 10 points, 100 points or a simple 3 points reprrsented with emojis. I quite like the idea, although I think it makes it a little more complicated to calculate ratings for all people.
- ijustlovemath 2y agoYou could do a simple weighting, probably adjusting for sample distributions.
- oneeyedpigeon 2y agoJust ranking everything in order is the ideal rating system. And it's easy to convert to whatever poorly-chosen system someone else wants you to use. It does mean the scores for your first few ratings will fluctuate quite a bit, though!
- 7bit 2y ago
- darthrupert 2y agoShould be Positive / Neutral / Negative and anything beyond that described in text. Reasoning: number ratings are subjective. My 4/5 is not the same as yours... or even the same as mine 2 years ago.
- latexr 2y agoAgreed in general that 10 and even 5 is too much, and that 4 is a good compromise. Though personally I prefer thumbs up and thumbs down, plus a separate starring option. The first two signify “would I recommend this to anyone else” while the latter means “this has something interesting I’d like to revisit at a later date”. Something losing its star rating is par for the course, but the recommendation status is less likely to change (though it can happen). And yes, it is possible to give something a thumbs down and favourite it, e.g. when you don’t think something is particularly good or competent but it still had something which you recognise as meaningful to yourself specifically. I don’t think this system is right for everyone, but I like it. Depending on the platform I may even use a rating system of 1, which represents the starring and everything else is just read/watched.
- stevage 2y agoI informally often use a -1 to 2 scale. Bad, fine, good, great. The difference between 1 and 2 on a 5 point scale is not useful.
- d1sxeyes 2y agoThis is a great scale to use when you’ve got a group of people because it’s easy to teach without needing to do calibration: -1 is bad, not meeting expectations, 0 is “not good, but meets expectations”, 1 is “good, better than expectations” and 2 is outstanding. It’s hard to come up with a nice visual for it though, you just have to use the numbers themselves (or rather ugly emojis)
- latexr 2y ago> It’s hard to come up with a nice visual for it though Suggestion off the top of my head: down arrow / thumb down; circle / horizontal dash; up arrow / thumb up; star / heart. The circle / horizontal dash could be ambiguous in isolation but should be clear in context.
- croisillon 2y agofor me there are 3 levels: - books i wish i hadn't lost time with - books i've read and were probably ok - books i would give/recommend
- gniv 2y agoHaving worked on a rating system, I think 10 levels are useful (or 5 levels with half-stars). You can create better averages/recs using 10 levels, since ratings of 0 or 10, which are often spammy, can be down-weighed.
- magicalhippo 2y agoI can absolutely understand that it makes sense to have a much more fine-grained average, but personally I struggle to give meaningful ratings beyond the four I mentioned. That said perhaps multiple binary dimensions would be better. Good story yes/no, interesting/unique premise yes/no, overall good acting yes/no, good cinematography yes/no etc etc.
- magicalhippo 2y agoI was interrupted, so when I picked up my brain continued with movie recommendation mode, I watch more movies than read books, but the binary multi-dimensional rating could of course be applied to books as well.
- jay_kyburz 2y agoI once had an interesting conversation with my mother in law who was telling us about how much thought and effort she puts into scoring shows on IMDB. She was shocked to learn that I want my opinion to have more impact, so If I like a show I give 10 and if I didn't it gets 1.
- gniv 2y agoYes, but sadly a 10 is often used by fans of a lead actor to pump the ratings, and a 0/1 is used by the moral police as a protest vote.
- kmfrk 2y agoGoodfilms (goodfil.ms), rest in peace, had a great two-rating system with Quality and Rewatchability, because the latter turned out to be a really useful metric. https://news.ycombinator.com/item?id=3082241 https://news.ycombinator.com/item?id=3082241 One of the main frustrations I have with Goodreads is how limited the API is nowadays, and how there appear to be no measures against brigading and other campaigns. One of the core issues with ratings services. Personally I'm hoping Open Library by the Internet Archive grows more in popularity, given how most websites come and go: https://openlibrary.org https://openlibrary.org
- vasanthk1125 2y agoI'm curious—what would you like to do with the API?
- kmfrk 2y agoIronically, mainly to check for irregular voting patterns like brigading. Not dissimilar to what Steam implemented, which is basically Bollinger bands for ratings.
- SamBam 2y agoHow does one use an average Rewatchability score to determine whether one should watch a movie? If I'm trying to pick a movie, I don't care what its score for rewatching is, I care about what its score is for watching it the first time. And once I've watched a movie, I don't care about whether other people say I should watch it again, I care about whether I want to watch it again. A movie is different from buying a board game. If I'm shelling out $50 for a game, I'll want to know if it's still fun the twentieth time I play. But that isn't a consideration when picking a new movie to watch, the experience may be worth it even if I never watch it again. And ditto with books. I'm probably not going to read that 800-page book again, but that shouldn't stop me reading it once.
- hiAndrewQuinn 2y agoTo present an opposite viewpoint, I try to only engage with media I suspect future me will be able to revisit and pull enjoyment out of at least once. I rarely actually do so, but I've found it to be a remarkably effective quality filter. It's also a genre independent quality metric. That's not to be underplayed. Some examples of films that successfully passed it for me: Casablanca; Portrait of a Lady on Fire; Hereditary; Under the Skin; My Neighbor Totoro; The Fifth Element. I'm pretty sure most people would agree at least half of these movies are good. It turns out most of the things I consider worth revisiting at least once are also things other people would consider that way. So for me a Rewatchability rating is a positive signal.
- lgl 2y agoSure, but can we really deny that the 5 star rating is pretty decent and simple, having an exact middle point and then two levels for each side.
- magicalhippo 2y agoWell yeah, but I feel the neutral midpoint is problematic because I feel it's too easy to pick it. Removing the midpoint forces one to decide which side it leans towards. If one really feels the need for the "meh" category, I'd say go for a 3-level system: bad, meh, great.
- 7bit 2y agoBlack Panther has a rating of 96% at Rotten Tomatoes. So whatever the scale is, nothing is really meaningful. I've been in the cinema with about 60 other people and a friend. statistically one of us should have enjoyed it but we both found the movie shit. So no, 10 stars is not more meaningful as 5 stars. 2-4 should be enough to still be wrong about taste once in a while.
- gamblor956 2y ago96% means that 96% of critics liked the movie enough to recommend it. It also means that 4% of critics did not recommend the movie. In a theater of 60 people, you and your friend would fit into that 4%. So there's nothing wrong with RottenTomatoes.
- BeetleB 2y ago> Does one really get anything meaningful out of saying this was a 6-star book vs a 7-star book? I rate for myself, and not others. And for over 20 years I've used a 10 point system. 10 = Easily amongst my favorite 9 = Awesome, but not in all time favorites 8 = Really liked it, and would recommend 7 = Liked it, was worth my time, but not so much that I would happily recommend to others 6 = Liked it, but wasn't worth my time 5 = Neutral And below 5 I don't distinguish. I randomly pick to indicate I didn't like it.
- fcarraldo 2y agoIt kind of seems like you use a five-point system, then?
- BeetleB 2y agoThe difference being that I don't want someone to think that because I rated something 2 out of 5 that I didn't like it because it's less than 2.5. And for most people, a 3/5 doesn't mean "yes, I heartily recommend". But they can believe it for 8/10 (Actually, 7 points as someone else pointed out - by my point stands even with a 7 point system).
- cAtte_ 2y agoyou use a 7 point system
- longdustytrail 2y agoI agree with a lot of your top 5 so I’ll try to continue 5: I enjoyed sections of this book but as a whole I didn’t like it 4: had some cool ideas and there were moments when I got excited but the execution wasn’t there. Basically an amateur with a good idea 3: readable but unsatisfying. I finished it but was roasting it in my head the whole time 2: garbage. Bad story idea and bad writing. Nothing good to say except that it seemed like the author was trying 1: offensive. Celebrity cash grabs, polemics, etc. no artistic value whatsoever, author was not trying to write a good book. “Book” is just a format here
- dimava 2y ago
- mvkel 2y agoThe 4-star system is indeed wonderful. A rating that is truly "neutral" doesn't exist in a world of human subjectivity
- phaedrus441 2y agoI completely agree. I've never read a 1-star (to me) book because that implies it's unreadable, and anything good enough to keep my attention room is generally 4-stars and rarely 5-stars. I bet if I look at my Goodreads it's 60% 4s, 30% 3s, and 10% 5s
- jay_kyburz 2y agoI don't rate books on Goodreads, but I do look at the average rating when deciding to read a book. I won't start anything less than about 3.7 or 3.8.
- qznc 2y agoRating books on an absolute scale makes little sense to me. The actual questions is: Whom can you recommend this book? Even mediocre books can be very useful for the right people.
- barotalomey 2y agoIf there's no voting data, it doesn't matter what scale is used.
- stock_toaster 2y agoI typically like 7 point scales (like this[1] post from jgc outlines), but your classification also seems pretty good. [1]: https://blog.jgc.org/2007/12/seven-point-scale.html https://blog.jgc.org/2007/12/seven-point-scale.html
- teeray 2y agoMy stars are as follows: 5 - This book was so good that it’s life-changing 4 - This is a really good book 3 - I enjoyed this book, it was good. 2 - It’s alright. 1 - I hated this book with every fiber of my being, because it somehow tricked me into finishing it despite my hatred of it.
- kaushikt 2y agoYouTube changed to thumbs up/down from 5-star rating system many years ago like Netflix. They learnt most people either give it 1 star or 5 stars. It’s also hard to understand the meaning of 2,3, and 4 stars in the context of a video.
- eviks 2y agoNo, that's meaningless precision. meaningful precision would be stars by category, not a generic overall assessment
- magicalhippo 2y agoI don't feel it's entirely meaningless, but I do agree multidimensional rating would be more informative and better to guide others.
- kqr 2y agoI think one of the most robust rating systems I've worked with is based on comparisons, i.e. "did you like this more or less than X". This is more computationally intensive, but it can be made to work even with intransitive and unreliable judgments.
- stared 2y agoTo me, even 5-star system is broken by design, with a pressure to rate everything 4 and 5, especially if these are known to be classic books. I rarely give 1 or 2 - in vast majority of cases it means I stop reading them, out of respect for my time. What is nice, but underused (since most platforms want us to be excited, because of sales and adverts) is some kind of slider with mean at 0, for expected quality. Even better, tags to choose from "awesome", "insightful", "well-researched", "funny", "cringe", "inaccurate" etc. I mean, there are tags, but I mean ones explicitly displayed next to rating.
- slightwinder 2y agoI think for a personal level, everything with more than 3 rating-states is overrated. Either you like it, dislike it, or don't care. Nothing more necessary. But on a social site, more nuance can be valuable to get a more fine-tuned result without polishing algorithms.
- SergeAx 2y agoI can definitely say there is a difference between 7-star and 6-star movies on IMDb. A 7-star movie on IMDb is worth watching most of the time, while a 6-star movie is a 50/50 chance. A 5-star and less movie is a hard pass.
- bwb 2y agoYa i do love, like, meh, dislike :). I want personal ratings not a weird public scale that everyone uses differently.