4 ms·

Disclaimer: I don't work for Yelp but I have created other user-ratings-based sites before. It's not entirely fair to criticize Yelp for under-emphasizing nega

Disclaimer: I don't work for Yelp but I have created other user-ratings-based sites before. It's not entirely fair to criticize Yelp for under-emphasizing negative reviews. One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting. You work harder to get positive reviews than negative ones. If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied. This should be the motivation for preferencing good reviews over lots of one-off bad reviews. BUT: Yelp made the cardinal sin of "objective" ratings sites: their profit comes from the entities they are rating. Ergo everything on their site is questionable.

dfxm12 14y ago
If this bias occurs across all businesses on Yelp, then still, wouldn't some places will be rated higher than others (even if "higher rated" comes to mean "has fewer bad reviews")? Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.
bloggergirl 14y ago
"people are most motivated to write reviews when they are outraged. ...your site runs the risk of becoming a soap box for ranting." There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you. No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.
- Kesty 14y ago
  People are way more willing to take the time to write a bad review if they feel that something wrong was done to them than take the same time to leave a positive review, even if you ask them to. The only one that will leave a good review will be Yelp users that rates everything. And maybe people that you did them a big favor and they feel they have to repay you someway.
pbreit 14y ago
I'm not sure I can agree with that. Negative reviews are almost always far more helpful than positive reviews. Review sites should be careful about referencing positive reviews. The gold standard is Consumers Reports which is much more balanced than the average interent review site (where everything gets 4 stars) and thus much more helpful.
- bbrtyth 14y ago
  Speaking as an app developer, sometimes people get furious over things that are clearly stated before they download it. It really is easier to get complaints, I can see usage stats that show many more people using my apps daily.
  - drcube 14y ago
    So I read the complaint, and that thing that was clearly stated before is now reiterated. If it doesn't bother me, I go ahead and buy your app. Good reviews are worthless to me as a consumer. What I want are details, good and/or bad, that I can use to evaluate my purchase ahead of time. "This is great!!!", "I would use it again!" are devoid of information. If negative reviews are more informative, then I assure you, you should welcome them over a bunch of five-star "great!"s. Besides being informative for me, the user, they should be valuable to you, the business owner. Nobody maintains an "attaboy" database, but you better keep track of bugs and fix them if you expect to grow or even survive in this economy.
    - bbrtyth 14y ago
      You are more willing to read than most, I guess. Legitimate complaints are welcome, I never implied they were not.
steve8918 14y ago
I do agree that a lot of valid reviews are being thrown out with the bathwater. There should be a better mechanism to show all unfiltered reviews. The thing that Yelp that makes it stand above other review sites is that they incentive positive reviews. Like Yelp or not, the idea of "Yelp Elite" is actually a great idea, because it is able to draw out a lot of medium to great reviews, stuff you would rarely see on other sites. When you incentive reviews, either good or bad, then you do tend to eliminate this "one-and-done" bias that tends to focus on the negative. That being said, my wife is/was a Yelp Elite for several years with over 800 reviews, but quit this year because she thinks that Yelp as a whole has started to go downhill, and doesn't like the pretentiousness of most people on there now.
- jonknee 14y ago
  > I do agree that a lot of valid reviews are being thrown out with the bathwater. There should be a better mechanism to show all unfiltered reviews. What's tricky about it now? Click "X filtered" reviews and you can read to your heart's content.
  - ceejayoz 14y ago
    After you fill out a totally unnecessary captcha that seems to only be there to make some users give up at that point.
    - jonknee 14y ago
      The captcha is there so people don't scrape the filtered reviews (be it Google, a browser extension, spammers, etc)
      - ceejayoz 14y ago
        That's absurd. Google's not going to see the filtered reviews unless they start doing a lot more JavaScript execution than they've been doing, and why would Yelp care if a browser extension or spammers see filtered reviews?
starpilot 14y ago
> One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting. This is the risk we take listening to ANYTHING depending on voluntary responses. Burying negative reviews to attempt some intrinsic rating dismisses the value in actual user input, with the paternalistic view that the review aggregator somehow knows better. Who decides which and how many negative reviews to hide? You? How about letting the user figure it out, knowing that trusting any opinion is a risk and learning from experience to calibrate reviews he reads to himself? It's the best we can hope for without resorting to the review site, in effect, clobbering consensus views to get something it "knows to be more accurate." In that case the users no longer review, but the site, which is not a user, does. > If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied. You absolutely do not know this. It could be that 70% were dissatisfied. You can't assume this without taking the role of a reviewer, which you cannot legitimately do without experiencing the item being reviewed.

dfxm12 14y ago

If this bias occurs across all businesses on Yelp, then still, wouldn't some places will be rated higher than others (even if "higher rated" comes to mean "has fewer bad reviews")? Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.

bloggergirl 14y ago

"people are most motivated to write reviews when they are outraged. ...your site runs the risk of becoming a soap box for ranting." There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you. No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.

Kesty 14y ago
People are way more willing to take the time to write a bad review if they feel that something wrong was done to them than take the same time to leave a positive review, even if you ask them to. The only one that will leave a good review will be Yelp users that rates everything. And maybe people that you did them a big favor and they feel they have to repay you someway.

pbreit 14y ago

I'm not sure I can agree with that. Negative reviews are almost always far more helpful than positive reviews. Review sites should be careful about referencing positive reviews. The gold standard is Consumers Reports which is much more balanced than the average interent review site (where everything gets 4 stars) and thus much more helpful.

bbrtyth 14y ago
Speaking as an app developer, sometimes people get furious over things that are clearly stated before they download it. It really is easier to get complaints, I can see usage stats that show many more people using my apps daily.
- drcube 14y ago
  So I read the complaint, and that thing that was clearly stated before is now reiterated. If it doesn't bother me, I go ahead and buy your app. Good reviews are worthless to me as a consumer. What I want are details, good and/or bad, that I can use to evaluate my purchase ahead of time. "This is great!!!", "I would use it again!" are devoid of information. If negative reviews are more informative, then I assure you, you should welcome them over a bunch of five-star "great!"s. Besides being informative for me, the user, they should be valuable to you, the business owner. Nobody maintains an "attaboy" database, but you better keep track of bugs and fix them if you expect to grow or even survive in this economy.
  - bbrtyth 14y ago
    You are more willing to read than most, I guess. Legitimate complaints are welcome, I never implied they were not.

steve8918 14y ago

I do agree that a lot of valid reviews are being thrown out with the bathwater. There should be a better mechanism to show all unfiltered reviews. The thing that Yelp that makes it stand above other review sites is that they incentive positive reviews. Like Yelp or not, the idea of "Yelp Elite" is actually a great idea, because it is able to draw out a lot of medium to great reviews, stuff you would rarely see on other sites. When you incentive reviews, either good or bad, then you do tend to eliminate this "one-and-done" bias that tends to focus on the negative. That being said, my wife is/was a Yelp Elite for several years with over 800 reviews, but quit this year because she thinks that Yelp as a whole has started to go downhill, and doesn't like the pretentiousness of most people on there now.

jonknee 14y ago
> I do agree that a lot of valid reviews are being thrown out with the bathwater. There should be a better mechanism to show all unfiltered reviews. What's tricky about it now? Click "X filtered" reviews and you can read to your heart's content.
- ceejayoz 14y ago
  After you fill out a totally unnecessary captcha that seems to only be there to make some users give up at that point.
  - jonknee 14y ago
    The captcha is there so people don't scrape the filtered reviews (be it Google, a browser extension, spammers, etc)
    - ceejayoz 14y ago
      That's absurd. Google's not going to see the filtered reviews unless they start doing a lot more JavaScript execution than they've been doing, and why would Yelp care if a browser extension or spammers see filtered reviews?

starpilot 14y ago

> One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting. This is the risk we take listening to ANYTHING depending on voluntary responses. Burying negative reviews to attempt some intrinsic rating dismisses the value in actual user input, with the paternalistic view that the review aggregator somehow knows better. Who decides which and how many negative reviews to hide? You? How about letting the user figure it out, knowing that trusting any opinion is a risk and learning from experience to calibrate reviews he reads to himself? It's the best we can hope for without resorting to the review site, in effect, clobbering consensus views to get something it "knows to be more accurate." In that case the users no longer review, but the site, which is not a user, does. > If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied. You absolutely do not know this. It could be that 70% were dissatisfied. You can't assume this without taking the role of a reviewer, which you cannot legitimately do without experiencing the item being reviewed.