4 ms·

We barely need a Yelp for restaurants. I think it is time for the internet to acknowledge that the pointless drivel in star rating systems is just not helping a

We barely need a Yelp for restaurants. I think it is time for the internet to acknowledge that the pointless drivel in star rating systems is just not helping anyone. If you’re going to start a system for rating things, give it more nuance and value.

sn41 6y ago
Yup. Recently bought an affordable mechanical keyboard without consulting any review websites. Am very happy with it for about a month. Then I went to Amazon to see the reviews, out of curiosity. They are very negative for the product I am very satisfied with. Had I done due diligence, I would have lost a bunch of cash for a mere marginal improvement which none but connoisseurs can discern. I think there are a few things in life we must decide carefully - like a partner, which doctor to consult etc. Reading reviews for most of the other stuff is overthinking, and pointless.
fastball 6y ago
I actually have a plan for this! I want to build a ratings website where you rate things in the same way they generally structure psychological tests, by responding to affirmative statements with something between "strongly agree" and "strongly disagree". As an example, if you were rating films, questions could be: "This film was enjoyable." "This film had an engaging plot." "The acting was believable / appropriate for the film." etc. Ideally you'd have about 5 standard statements that you respond to for a particular thing you want to rate (whether its films, books, food, or a university course). Better than a star rating system, because the problem with stars is that you end up with scores that always hover from 5-8 (if out of 10) in a very not useful way (see IMDB). It's also better than what RT / Netflix / many others have moved to (just voting with up/down) as those require a lot of data in order to really give interesting results, where as with this system you can start to find interesting insights almost immediately, or within a single person's own collection.
- jkinudsjknds 6y ago
  People not answering questions correctly is precisely the reason many companies switch to thumbs up thumbs down. The precision of the measurement just isn't there for good / bad. Collaborative filtering with just these simple metrics works decently well, the issue is most sites like netflix have difficult UIs to actually use it as a poweruser. The average consensus of people on how good a thing is on some sort of scale, is pretty worthless. The consensus of people who seem to be like you on previous works is kind of useful. Strongly disagree, disagree, neutral, agree, strongly agree is just a 5 star system with a different name.
- abakker 6y ago
  I like this - in general, I’m a fan of this (my job is survey research). There are many different approaches to getting better results than most reviews get. Personally, I’d love to see a max-diff approach to reviews.
  - fastball 6y ago
    Feel free to steal the idea! I'm a bit busy building out another idea I had haha.
kerkeslager 6y ago
I agree with you that Yelp for restaurants is generally a waste of the electricity used to run it. However, I think the problem isn't stars or a lack of nuance, it's that you get what you pay for. If you don't pay for reviews, you mostly get two kinds of reviews, which are at the extremes: 1. I'm angry enough that I went online and wrote a review. 2. I own the business or I'm a true believer in the business. Neither case is actually what you want from a review. But people don't post middle-star reviews because they don't feel strongly enough about it to feel an incentive to do that. The nuance comes when you pay someone to write an opinion they don't feel strongly about, which is the only time they can maintain anything resembling an objective opinion. I've found very good products by paying for Consumer Reports, for example. "But it doesn't scale!" says HN. Sure. Lots of working solutions don't scale, and lots of scalable ideas don't solve any real problem. Bad things happen to good people.
tvanantwerp 6y ago
My wife and I have learned the hard way: figure out first if the people leaving reviews value what you value, then look at the number of stars.
- gowld 6y ago
  The stats are nearly completely meaningless. Once you've read the review, the stars say nothing relevant to you.

sn41 6y ago

Yup. Recently bought an affordable mechanical keyboard without consulting any review websites. Am very happy with it for about a month. Then I went to Amazon to see the reviews, out of curiosity. They are very negative for the product I am very satisfied with. Had I done due diligence, I would have lost a bunch of cash for a mere marginal improvement which none but connoisseurs can discern. I think there are a few things in life we must decide carefully - like a partner, which doctor to consult etc. Reading reviews for most of the other stuff is overthinking, and pointless.

fastball 6y ago

I actually have a plan for this! I want to build a ratings website where you rate things in the same way they generally structure psychological tests, by responding to affirmative statements with something between "strongly agree" and "strongly disagree". As an example, if you were rating films, questions could be: "This film was enjoyable." "This film had an engaging plot." "The acting was believable / appropriate for the film." etc. Ideally you'd have about 5 standard statements that you respond to for a particular thing you want to rate (whether its films, books, food, or a university course). Better than a star rating system, because the problem with stars is that you end up with scores that always hover from 5-8 (if out of 10) in a very not useful way (see IMDB). It's also better than what RT / Netflix / many others have moved to (just voting with up/down) as those require a lot of data in order to really give interesting results, where as with this system you can start to find interesting insights almost immediately, or within a single person's own collection.

jkinudsjknds 6y ago
People not answering questions correctly is precisely the reason many companies switch to thumbs up thumbs down. The precision of the measurement just isn't there for good / bad. Collaborative filtering with just these simple metrics works decently well, the issue is most sites like netflix have difficult UIs to actually use it as a poweruser. The average consensus of people on how good a thing is on some sort of scale, is pretty worthless. The consensus of people who seem to be like you on previous works is kind of useful. Strongly disagree, disagree, neutral, agree, strongly agree is just a 5 star system with a different name.
abakker 6y ago
I like this - in general, I’m a fan of this (my job is survey research). There are many different approaches to getting better results than most reviews get. Personally, I’d love to see a max-diff approach to reviews.
- fastball 6y ago
  Feel free to steal the idea! I'm a bit busy building out another idea I had haha.

kerkeslager 6y ago

I agree with you that Yelp for restaurants is generally a waste of the electricity used to run it. However, I think the problem isn't stars or a lack of nuance, it's that you get what you pay for. If you don't pay for reviews, you mostly get two kinds of reviews, which are at the extremes: 1. I'm angry enough that I went online and wrote a review. 2. I own the business or I'm a true believer in the business. Neither case is actually what you want from a review. But people don't post middle-star reviews because they don't feel strongly enough about it to feel an incentive to do that. The nuance comes when you pay someone to write an opinion they don't feel strongly about, which is the only time they can maintain anything resembling an objective opinion. I've found very good products by paying for Consumer Reports, for example. "But it doesn't scale!" says HN. Sure. Lots of working solutions don't scale, and lots of scalable ideas don't solve any real problem. Bad things happen to good people.

tvanantwerp 6y ago

My wife and I have learned the hard way: figure out first if the people leaving reviews value what you value, then look at the number of stars.

gowld 6y ago
The stats are nearly completely meaningless. Once you've read the review, the stars say nothing relevant to you.