3 ms·

> We as a society need to start thinking about how we rate people. Right now we all shop for that 5-star rating and punish anyone with 4. We need to mature beyo

by sp527 8y ago

> We as a society need to start thinking about how we rate people. Right now we all shop for that 5-star rating and punish anyone with 4. We need to mature beyond that as it grows increasingly impossible to maintain perfect ratings. I couldn't disagree more with this sentiment. The answer is not to intentionally sabotage competitive market forces that are driving better outcomes for consumers. That's the equivalent of a jobs program. If you can't compete, you don't compete. Businesses are inherently risky and some people will lose. If an inordinate number of sellers are failing or struggling, that reflects a failed assessment of market risk and ill-informed decision-making. The answer is to improve the decision-making capability of participants or otherwise improve the social safety net for people who are facing new realities of economic obsolescence.

jolmg 8y ago
I didn't downvote, but I think cptaj's point was to "unsabotage" the rating system. Ideally, the 5-star rating system should work like: - 1-star means Awful - 2-star means Bad - 3-star means OK - 4-star means Good - 5-star means Excellent That means no one should have any problem using something rated 3 or above. However, by "[shopping] for that 5-star rating and [punishing] anyone with a 4", the rating system ends up meaning: - 1-star means Couldn't be worse - 2-star means Awful - 3-star means Bad - 4-star means There was a problem - 5-star means OK Because we all know that's how the rating is treated, no one gives a 3 when the experience was normal and tries to reserve that 5-star for when a service was absolutely phenomenal. We just give a 5 when a service was OK. Being more accepting of 4-star or 3-star ratings is not sabotaging competitive market forces; it's trying to use the 5-star rating system like it should be.
- notafraudster 8y ago
  This does not follow. Even in a system where 3 stars means excellent, 4 stars means once in a lifetime, and 5 stars means Platonic Ideal, I'm still going to look for restaurants (or drivers or network routers or whatever the thing is) that are as close to 5* as possible. Do I subdivide between 4.45 and 4.47 stars? No. But in general I'm going to sort my results by some measure of quality. I'm also optimizing over other variables, but to the extent quality is one of those variables, there's no reason to settle for OK when Excellent is available. I'm definitely not going to voluntarily get worse food so I can protest the idea of ratings. And as long as that's the case, there will always be pressure for people and services that aren't "Excellent" to go for the 5 star anyway, and as long as they're going for it, there'll be a social pressure to provide it. The problem isn't linguistic, it's game theoretic.
  - deleted 8y ago
    [deleted]
  - xoa 8y ago
    I don't think you've thought this through entirely, primarily because your analysis here though misses out on a cost function for stars. >I'm definitely not going to voluntarily get worse food so I can protest the idea of ratings. Yeah, but most people (maybe including you) certainly will "voluntarily get worse food" if you consider the price difference sufficient. In a system where 3 stars means good or excellent it can be expected there will be a non-linear increase in cost and skill requirements to hit 4 or 5 stars. They will be rarer, likely less convenient, and certainly more expensive. For a special occasion sure, maybe you go for the very best. But in general many people will be perfectly happy with a lower starred establishment that is more affordable and convenient and still perfectly decent. Sometimes people just want to fill their stomachs for the purpose of continued survival in a reasonably pleasant fashion, not have a magical gourmet experience. Scale compression removes visibility for this basic tradeoff right? I mean, we can even find a direct example of this in the food world right now: Michelin Stars. It's a 1 to 3 scale and even a single one is a big deal. But the standards necessary to qualify and maintain it are not cheap for the restaurant. I remember reading of a very nice restaurant in a more run down area of France that had earned a star, and were very proud of it, but ultimately decided to give it up because it resulted in them being priced out of the general range of locals. Conversely there is nothing bad about going to even a 1 Michelin star restaurant, it'll still be likely to be very good.
    - dcole2929 8y ago
      Read this story in December when it happened and was absolutely shocked but it makes so much sense. https://www.eater.com/2017/12/27/16821648/chef-returns-michelin-star-le-france-jerome-brochot https://www.eater.com/2017/12/27/16821648/chef-returns-miche...
- amerkhalid 8y ago
  There is another big negative about using ratings when dining out that no one talks about. My friends and I were recently discussing how we haven't had a bad experience in any restaurant in awhile. But now we are so used to really good restaurants that it is extremely rare to get amazed by any restaurant. If we compare this to our experiences in pre-Yelp era, it was so exciting to discover an amazing restaurant. Sure we ate at a lot of mediocre restaurants but joy of finding something amazing was greater back then. I am not sure what the solution is. Stop looking up ratings?
  - snoman 8y ago
    I don't know. Sounds like the ratings might just be working.
- logfromblammo 8y ago
  Deciding what star level to accept should not be the rating-reader's problem. The reviewers need to have their own ratings statistically analyzed, so that their median rating is adjusted to 3 stars out of five for the purposes of averaging across multiple reviewers. If you only rate things as 5 stars, you have no dynamic range. I want your reviews to all be 3 stars in the average I see. Another optimization could be to change the aggregation function such that reviewers that rated well the same things I rate well, and who rate poorly the same things I rate poorly, and those reviewers I explicitly mark as having helpful reviews, would be weighted more heavily in the number I see. That would effectively filter out bots that rate their brand as 5 stars and competitors as 1 star, because that pattern is so dissimilar from normal consumer behavior. The bots see cheap white-label crap as 5-star because the recommendations of their fellow ratings-gaming spam-bots dominate their own aggregate rating. If you rate a microwave oven as 1-star because it shipped in 3 days rather than 2, I don't really want your reviews polluting my averages. If you rate a cheap-ass USB cable as 5-stars because it "works good i had no problems", I don't want your reviews polluting my averages. If you rate a USB charger as 1-star because it "Violates USB 3.0 spec and will probably burn your house down; I am an electrical engineer and here's how I figured that out..." then you're danged right I want that reviewer's reviews to have more weight. I don't see how a system like that would be any more complex than Google's ranking algorithms for search. If they can discover the most useful sites and weed out SEO gaming attempts by analysis of links and traffic, so too could Amazon put an actually useful rating value on the products it ships, and so too could Uber show you what drivers and riders are really like.

by sp527 8y ago

jolmg 8y ago

I didn't downvote, but I think cptaj's point was to "unsabotage" the rating system. Ideally, the 5-star rating system should work like: - 1-star means Awful - 2-star means Bad - 3-star means OK - 4-star means Good - 5-star means Excellent That means no one should have any problem using something rated 3 or above. However, by "[shopping] for that 5-star rating and [punishing] anyone with a 4", the rating system ends up meaning: - 1-star means Couldn't be worse - 2-star means Awful - 3-star means Bad - 4-star means There was a problem - 5-star means OK Because we all know that's how the rating is treated, no one gives a 3 when the experience was normal and tries to reserve that 5-star for when a service was absolutely phenomenal. We just give a 5 when a service was OK. Being more accepting of 4-star or 3-star ratings is not sabotaging competitive market forces; it's trying to use the 5-star rating system like it should be.

notafraudster 8y ago
This does not follow. Even in a system where 3 stars means excellent, 4 stars means once in a lifetime, and 5 stars means Platonic Ideal, I'm still going to look for restaurants (or drivers or network routers or whatever the thing is) that are as close to 5* as possible. Do I subdivide between 4.45 and 4.47 stars? No. But in general I'm going to sort my results by some measure of quality. I'm also optimizing over other variables, but to the extent quality is one of those variables, there's no reason to settle for OK when Excellent is available. I'm definitely not going to voluntarily get worse food so I can protest the idea of ratings. And as long as that's the case, there will always be pressure for people and services that aren't "Excellent" to go for the 5 star anyway, and as long as they're going for it, there'll be a social pressure to provide it. The problem isn't linguistic, it's game theoretic.
- deleted 8y ago
  [deleted]
- xoa 8y ago
  I don't think you've thought this through entirely, primarily because your analysis here though misses out on a cost function for stars. >I'm definitely not going to voluntarily get worse food so I can protest the idea of ratings. Yeah, but most people (maybe including you) certainly will "voluntarily get worse food" if you consider the price difference sufficient. In a system where 3 stars means good or excellent it can be expected there will be a non-linear increase in cost and skill requirements to hit 4 or 5 stars. They will be rarer, likely less convenient, and certainly more expensive. For a special occasion sure, maybe you go for the very best. But in general many people will be perfectly happy with a lower starred establishment that is more affordable and convenient and still perfectly decent. Sometimes people just want to fill their stomachs for the purpose of continued survival in a reasonably pleasant fashion, not have a magical gourmet experience. Scale compression removes visibility for this basic tradeoff right? I mean, we can even find a direct example of this in the food world right now: Michelin Stars. It's a 1 to 3 scale and even a single one is a big deal. But the standards necessary to qualify and maintain it are not cheap for the restaurant. I remember reading of a very nice restaurant in a more run down area of France that had earned a star, and were very proud of it, but ultimately decided to give it up because it resulted in them being priced out of the general range of locals. Conversely there is nothing bad about going to even a 1 Michelin star restaurant, it'll still be likely to be very good.
  - dcole2929 8y ago
    Read this story in December when it happened and was absolutely shocked but it makes so much sense. https://www.eater.com/2017/12/27/16821648/chef-returns-michelin-star-le-france-jerome-brochot https://www.eater.com/2017/12/27/16821648/chef-returns-miche...
amerkhalid 8y ago
There is another big negative about using ratings when dining out that no one talks about. My friends and I were recently discussing how we haven't had a bad experience in any restaurant in awhile. But now we are so used to really good restaurants that it is extremely rare to get amazed by any restaurant. If we compare this to our experiences in pre-Yelp era, it was so exciting to discover an amazing restaurant. Sure we ate at a lot of mediocre restaurants but joy of finding something amazing was greater back then. I am not sure what the solution is. Stop looking up ratings?
- snoman 8y ago
  I don't know. Sounds like the ratings might just be working.
logfromblammo 8y ago
Deciding what star level to accept should not be the rating-reader's problem. The reviewers need to have their own ratings statistically analyzed, so that their median rating is adjusted to 3 stars out of five for the purposes of averaging across multiple reviewers. If you only rate things as 5 stars, you have no dynamic range. I want your reviews to all be 3 stars in the average I see. Another optimization could be to change the aggregation function such that reviewers that rated well the same things I rate well, and who rate poorly the same things I rate poorly, and those reviewers I explicitly mark as having helpful reviews, would be weighted more heavily in the number I see. That would effectively filter out bots that rate their brand as 5 stars and competitors as 1 star, because that pattern is so dissimilar from normal consumer behavior. The bots see cheap white-label crap as 5-star because the recommendations of their fellow ratings-gaming spam-bots dominate their own aggregate rating. If you rate a microwave oven as 1-star because it shipped in 3 days rather than 2, I don't really want your reviews polluting my averages. If you rate a cheap-ass USB cable as 5-stars because it "works good i had no problems", I don't want your reviews polluting my averages. If you rate a USB charger as 1-star because it "Violates USB 3.0 spec and will probably burn your house down; I am an electrical engineer and here's how I figured that out..." then you're danged right I want that reviewer's reviews to have more weight. I don't see how a system like that would be any more complex than Google's ranking algorithms for search. If they can discover the most useful sites and weed out SEO gaming attempts by analysis of links and traffic, so too could Amazon put an actually useful rating value on the products it ships, and so too could Uber show you what drivers and riders are really like.