7 ms·
Similarly - one of my biggest complaints about almost every rating system in production is how just absolutely lazy they are. And by that, I mean everyone seem
by nlh 1y ago
Similarly - one of my biggest complaints about almost every rating system in production is how just absolutely lazy they are. And by that, I mean everyone seems to think "the object's collective rating is an average of all the individual ratings" is good enough. It's not.
Take any given Yelp / Google / Amazon page and you'll see some distribution like this:
User 1: "5 stars. Everything was great!"
User 2: "5 stars. I'd go here again!"
User 3: "1 star. The food was delicious but the waiter was so rude!!!one11!! They forgot it was my cousin's sister's mother's birthday and they didn't kiss my hand when I sat down!! I love the food here but they need to fire that one waiter!!"
Yelp: 3.6 stars average rating.
One thing I always liked about FourSquare was that they did NOT use this lazy method. Their score was actually intelligent - it checked things like how often someone would return, how much time they spent there, etc. and weighted a review accordingly.
- theendisney 1y agoWith averages: to have 5 stars you need a hudred 5 star ratings for each one star rating. If one would normalize the ratings they could change without doing anything. A former customer may start giving good ratings elsewhere making yours worse or give poor ones inproving yours. Maybe the relevance of old ratings should decline.
- kayson 1y agoThe normalization doesn't have to be "live". You could apply the factor at time of rating and then not change it.
- theendisney 1y agoThen could let everyone start with 100 one star ratings. If they rate their first thing it counts as 1/101 vote. If they start with a one star rating it will be their highest ever. Alternatively you could apply the same rating to the customer and display it next to their user name along with their own review counter. What also seems a great option is to simply add up all the stars :) Then the grumpy people wont have to do anything.
- ajmurmann 1y agoIs that actually bad? What happened is that we learned more about the customer's rating system. I might never have had Cuban food and love it the first time I try it on Miami but then keep eating it and it turns out the first restaurant was actually not as good as I thought, I just really like Cuban food. This actually somewhat goes into another pet peeve of mine with rating systems. I'd like to see ratings for how much I will like it. An extreme but simple example might be that the ratings of a vegan customer of a steak house might be very relevant to other vegans but very irrelevant to non-vegans. More subtle versions are simply about shared preferences. I'd love to see ratings normalized and correlated to other users to create a personalized rating. I think Netflix used to do stuff like this back in the day and you could request your personal predicted score via API but now that's all hidden and I'm instead shown different covers off the same shows over and over
- theendisney 1y agoSounds like a fun startup. Vegan restaurant rating. Could do quite a few variations of it for people who care about a specific thing. Also, yes it matters that one in a hundred ratings leaves such a large mark on your business. I know one where they go out of their way to deliver quality. They get maybe two ratings per week. The competitor left only four fake ratings. It would take 200 weeks or 4 years to get back to 5 stars.
- Hizonner 1y agoI buy a lot of "technical things", and you constantly see one or two star ratings from people who either don't know what the thing is actually supposed to do, or don't know how to use it. My favorites: A power supply got one star for not simultaneously delivering the selected limit voltage and the selected limit current into the person's random load. In other words, literally for not violating the laws of physics. An eccentric-cone flare tool got one star for the cone being off center. "Eccentric" is in the name, chum....
- stevage 1y agoOr worse, a 1 star rating for a product they loved but there was a problem with delivery.
- derefr 1y agoI take this not as people being dumb, but as a clear conflict of interest: people want to be able to rate the logistics provider separately from the product, but marketplaces don't want to give people the option to do that — as that would reveal that the marketplace will sometimes decide to use "the known-shitty provider" for some orders. (And make no mistake, the marketplace knows that that provider is awful!)
- esperent 1y ago> for not violating the laws of physics. I would personally frame that as a review for poor documentation. A device shouldn't expect users to know laws of physics to understand it's limitations.
- Hizonner 1y agoIf you don't know that particular law of physics, you have no business messing with electricity. You'll very likely damage something, and quite possibly damage someone. We're talking about a general-purpose device meant to drive a circuit you create yourself. I'm not sure what a good analogy would be. Expecting the documentation for a saw to tell you you have to cut all four table legs the same length?
- kazinator 1y ago-1 stars! They forgot it was my cousin's sister's mother's birthday, and the obnoxious waiter snarkily pointed out that my cousin's sister is just another cousin, and her mother is just my aunt.
- zzo38computer 1y agoI think that numeric ratings (especially if only one number can be specified) (and then averaging or making or other types of statistics) are not as useful as actually reading the reviews in order to determine whether or not it addresses your concerns with it, and if they have specific complaints or specific things they say are good, to judge them by yourself according to your own intentions.
- nlh 1y agoAgreed. And particularly with the advent of LLMs, this is something that could be done quite easily. Don't even give users an option of giving a numeric/star rating - just allow people to write a sentence or two (or 10) and let the LLM do the sentiment analysis and aggregate a score.