6 ms·

An exploration of Yelp's own filtered reviews

86 points by jhdavids8 15y ago

michaelcampbell 15y ago
The captcha looked quite clearly to me to be "inctory". These captcha's are not necessarily real words (I believe to thwart dictionary based crackers?), and I also am pretty sure the service they are using is not tied to Yelp itself.
- jhdavids8 15y ago
  The first word wasn't the difficult one, it was the last word (it's even tougher to decipher when zoomed in for normal view. Zoomed out, it appears more like 'Law'). I agree though, I know the service probably isn't tied to Yelp, I just found it weird that I had to decipher a blob of ink on page 52.
  - rottencupcakes 15y ago
    If one word is completely illegible, it often isn't being used as a captcha. The captcha company may be trying to have you manually recognize text their OCR software could not. You can often make it through after painfully screwing up the hard to read word.
    - jrockway 15y ago
      This is how I do reCaptcha. I guess the easy-looking word correctly, and then type garbage for the hard-looking word. This results in approximately 99% correctness and hopefully screws up their "make jrockway do work for us for free" scam.
      - icebraining 15y ago
        What scam? Google offers the reCaptcha service for free in exchange for having words OCRed; seems fair to me. To you as the user, it's not only the same work as any other captcha (which you would have to fill in anyway), but you get free OCRed books and magazines from Google Books. If you don't like the reCaptcha ($deity know why, but it's your right), complain to the website, don't fuck it up for everyone else.
      - michaelcampbell 15y ago
        How proud you must be.
    - stan_rogers 15y ago
      That's exactly what's going on with ReCAPTCHA. One of the elements presented is a "known" that you have to get right, and the other is (usually) an unknown or ambiguous OCR result. Google is crowdsourcing their scans. And yes, sometimes it is just a blob of ink -- on other occasions, it's an inline differential equation or something similar that can't be properly entered into the text box (even with the whole of Unicode at your disposal). There's no penalty for getting the graecum est part wrong, but if it's something you can read but can't enter properly, do be a gem and don't pollute the pool of results -- hit the refresh icon.
pauljonas 15y ago
I stopped using Yelp after I discovered that reviews I logged to the site were not showing if I was not "logged in" to Yelp. What is the point of using a social review repository if the reviews are only viewable by me, when I log in to the site? I'd be better served by putting it in a text file/directory.
- bkbleikamp 15y ago
  It's not a democracy - Yelp pushes the best reviews to the top. Most people read a couple of reviews, it serves Yelp's interest to make sure they read the best ones.
  - derwiki 15y ago
    Why the downvotes? I'm sure everyone thinks highly of their own reviews, but let's be honest -- not everyone is Walt Whitman.
  - pauljonas 15y ago
    So, in other words, I as the user contributing reviews for which makes Yelp valuable, are treated as a de facto ruffian, my time and effort expended to "share" considered folly. So Yelp "filters" these and beyond wasting my time (as I should just collect in my own file of reviews or "share" on my own site) is it not a great conflict of interest going on here?
- Egregore 15y ago
  If you reviewed something you might refer to it later, for example - who was that carpenter we used last year?
jrockway 15y ago
"I don't like Yelp, so here are some random unsubstantiated complaints about the reviews about Yelp itself." The author seems to think people hate Yelp, but I'm not sure that's the case. Everyone I know in real life uses it regularly when trying to find some place to go, and the results are largely acceptable. Review sites are always going to trend negative because people who have had average or good experiences aren't going to be driven by rage to write a nasty review. Everyone has their own star scale and expectations of service ("I had to sit in economy class on my $10 ticket! I'm never flying United again!"). This leads to useless star ratings, but this is no fault of Yelp itself. It's the fault of relying on non-professionals to do professional-quality work. But, if you read for content, you can usually figure out whether a place is good or not. For example, a review like "I went during the dinner rush on Saturday night and it took 5 minutes to get a table! 1 star! Oh yeah, the food was good." is a positive review, even though the reviewer only gave one star. So anyway, don't hate on Yelp, hate on the clueless people clueless writing reviews.
- jhdavids8 15y ago
  Unsubstantiated complaints? I simply wrote what I saw, besides a few opinions here and there. And I know a lot of people do indeed dislike Yelp, but like some other sites with huge user bases, there's little other options to turn to, so you're stuck with using Yelp. I actually use Yelp quite a bit myself. Anyway, the evidence is clear for there to be good reason to hate on Yelp (see my article). But like you said, there's also good reason to hate on clueless reviewers. Why can't we do both until either issue is fixed?
  - jrockway 15y ago
    Unsubstantiated complaints? You claim that Yelp is writing their own 5 star reviews: Personally, I wouldn't be shocked if Yelp is putting in its own 5 star reviews for itself in order to offset the barrage of negative reviews. Common perception of Yelp is not good, far from their current 3-star rating. Just glancing over the first few 4 and 5-star reviews for Yelp, all but two came from an 'Elite' member. Of course they review Yelp highly, they get thrown crazy parties. And I wouldn't be shocked if you lose your Elite status for a negative Yelp review. This is pure FUD. You don't know if you lose elite status for a negative review. Not everyone gets to go to crazy parties. If this were Wikipedia, every statement would be tagged [weasel words] or [citation needed]. You've just made a bunch of stuff up. If this is just entertainment, fine, but if you think you're writing something useful that is worthy of HN, then you're a little off. Your article boils down to links to filtered reviews not being obvious enough (even though the link is the same as all the other links on the page), complaints about reCaptcha, which Yelp doesn't even control, and then a bunch of made-up facts/anecdotes that you have done no research to prove true or false. Look, I don't even like Yelp. But this is simply sloppy journalism, and it reads like it was written by a fifth grader who got a bunch of one-star reviews on Yelp.
progolferyo 15y ago
Interesting article. I don't understand the filtered review system at all. Beyond the 'he said / she said' complaints that occasionally come out, there are things about their system that simply don't make any sense unless Yelp is incompetent or slimy. For example: - When you post a review, you as a reviewer think its unfiltered forever. When you revisit the page as a logged in user and read a place that has your review, your review is visible. When you log out or log in as another user, the review is filtered and hidden. At the very least, it should tell you your review is filtered, I see no reason to pretend the review is not filtered when the review is legitimate. - When you view unfiltered results, the per page number mysteriously changes to 10 per page. I don't see any reason why this should change. Plus the results are pretty slow to load, quite slower than the results for filtered reviews. - Why do you need to enter in a captcha to view the unfiltered reviews? Why would they care if you were a bot only for the unfiltered reviews and not the normal reviews? I don't see the difference, unless they want to prevent people from writing scripts to pull in unfiltered review data. Plus the captcha is fucking horrible, literally half the captcha's I get are not readable and I need to refresh. - The filter algorithm seems to be clearly flawed and simply catches way too many reviews that should not be filtered. For example, take this user: http://www.yelp.com/user_details?userid=tZlbsUVo-8wtnR7oMa-3xg http://www.yelp.com/user_details?userid=tZlbsUVo-8wtnR7oMa-3... . The guy has 11 reviews, 1 1-star review, 1 2-star review and nothing out of the ordinary and yet his review about Yelp was filtered. Why? His points in the review seemed legitimate. He seems to be a normal user, not a new user and posts reviews across the board (more good reviews than bad in fact). They should either fix the algorithm or be more transparent about why reviews are filtered because I can't understand why a review like that is filtered.
- pbz 15y ago
  "Why do you need to enter in a captcha to view the unfiltered reviews?" Maybe they don't want Google to see them...
  - SoftwareMaven 15y ago
    That's what robots.txt is for, not silly captchas. You put captchas for the people who ignore that file.
deleted 15y ago
[deleted]
- deleted 15y ago
  [deleted]
mathattack 15y ago
Organizations like Yelp live and die on their accuracy and predictive ability. If Netflix recommends movies you don't like, you will use it less. If Zagat allows too much personal preference, it will no longer be the gold (Maroon?) standard of crowd sourced restaurant reviews. If half the things suggested about Yelp are true, they risk their integrity at great peril.
- cHalgan 15y ago
  Actually, Yelp can play game like this and be profitable. If business have a good rating on Yelp, then that pretty much mean that business is ok. However, a bad rating might be because business was not nice with Yelp sales people. That is ok because they target small, unestablished, businesses which you will never discover that they are actually excellent business because they have just one star review. In other words, advertising at Yelp does not guarantee you a good review. However, not being nice with their sales people might get you a bad review if you are a small business. And this is absolutely nothing new.
ZipCordManiac 15y ago
I've posted about 6 reviews in total on yelp, 2 of which were immediately filtered for no reason. Funny how my positive reviews went through just fine and the rest got binned. If you're going to run a review site, at least let people give their honest opinions don't filter anything other then profanity. People should be able to share an experience be it good or bad.
- wallawe 15y ago
  Although I wish it were that simple, it's more complicated than that. The reason for this simply is, if I own a business, I could create multiple fake accounts and have friends and and associates do the same. We would then be able to give multiple (fake) positive reviews for our own business, while giving one star feedback to our competition.
derwiki 15y ago
Yelp has more information on their web site about the review filter, soliciting reviews, false positives, and more: https://biz.yelp.com/support/common_questions https://biz.yelp.com/support/common_questions It's easy to jump on the bandwagon and say "Yelp is evil," but having been a near-constant point of controversy, it's something they do their best to address.
jprobert 15y ago
I think Yelp has always been a good service but obviously there is a disconnect between the business end of Yelp and the service end. The Chicago Tribune article is not the first to complain about the business practices of Yelp. Additionally, advertising on Yelp is some of the most expensive real estate on the web with CPM rates of $100-$300. Ultimately I think that directory sites such as Yelp will become obsolete as social search becomes more refined and relevant. My company is actually working on this and we thing that we have a good product so far (in alpha). http://www.cliqsearch.com http://www.cliqsearch.com
tlb 15y ago
Yelp's main defense is "It's done by an algorithm". That's not a valid defense for a bad business practice. They are accused of hiding positive reviews for businesses that don't pay them. You can perfectly well write this logic in Python for an filtering algorithm: def filter(review): ... if review.business.contacted_by_sales_count >= 3 and not review.business.paying_yelp and review.stars >= 4 and random() > 0.8: # 80% chance return true # yes, filter it ... Even if it's not written so explicitly they have to take responsibility for their algorithms.
- derwiki 15y ago
  The sales org only has access to Salesforce copy of Yelp's internal data, and numbers like "contacted_by_sales" never ends up back in the main Yelp database. It's not something the algorithm could take into account because it doesn't know about the sales org at all. Disclaimer: I used to work on the Yelp engineering team, at one point on the Salesforce data refresh project.
- pbreit 15y ago
  I'm sure most already understand this but when Yelp says the filter algorithmic, it's implied that a reasonable algorithm is being used.
eurohacker 15y ago
so are you saying that yelp is basicly an online extortion business, and judicial system and all the yelp users accept it ..?

86 points by jhdavids8 15y ago

The captcha looked quite clearly to me to be "inctory". These captcha's are not necessarily real words (I believe to thwart dictionary based crackers?), and I also am pretty sure the service they are using is not tied to Yelp itself.

jhdavids8 15y ago
The first word wasn't the difficult one, it was the last word (it's even tougher to decipher when zoomed in for normal view. Zoomed out, it appears more like 'Law'). I agree though, I know the service probably isn't tied to Yelp, I just found it weird that I had to decipher a blob of ink on page 52.
- rottencupcakes 15y ago
  If one word is completely illegible, it often isn't being used as a captcha. The captcha company may be trying to have you manually recognize text their OCR software could not. You can often make it through after painfully screwing up the hard to read word.
  - jrockway 15y ago
    This is how I do reCaptcha. I guess the easy-looking word correctly, and then type garbage for the hard-looking word. This results in approximately 99% correctness and hopefully screws up their "make jrockway do work for us for free" scam.
    - icebraining 15y ago
      What scam? Google offers the reCaptcha service for free in exchange for having words OCRed; seems fair to me. To you as the user, it's not only the same work as any other captcha (which you would have to fill in anyway), but you get free OCRed books and magazines from Google Books. If you don't like the reCaptcha ($deity know why, but it's your right), complain to the website, don't fuck it up for everyone else.
    - michaelcampbell 15y ago
      How proud you must be.
  - stan_rogers 15y ago
    That's exactly what's going on with ReCAPTCHA. One of the elements presented is a "known" that you have to get right, and the other is (usually) an unknown or ambiguous OCR result. Google is crowdsourcing their scans. And yes, sometimes it is just a blob of ink -- on other occasions, it's an inline differential equation or something similar that can't be properly entered into the text box (even with the whole of Unicode at your disposal). There's no penalty for getting the graecum est part wrong, but if it's something you can read but can't enter properly, do be a gem and don't pollute the pool of results -- hit the refresh icon.

pauljonas 15y ago

I stopped using Yelp after I discovered that reviews I logged to the site were not showing if I was not "logged in" to Yelp. What is the point of using a social review repository if the reviews are only viewable by me, when I log in to the site? I'd be better served by putting it in a text file/directory.

bkbleikamp 15y ago
It's not a democracy - Yelp pushes the best reviews to the top. Most people read a couple of reviews, it serves Yelp's interest to make sure they read the best ones.
- derwiki 15y ago
  Why the downvotes? I'm sure everyone thinks highly of their own reviews, but let's be honest -- not everyone is Walt Whitman.
- pauljonas 15y ago
  So, in other words, I as the user contributing reviews for which makes Yelp valuable, are treated as a de facto ruffian, my time and effort expended to "share" considered folly. So Yelp "filters" these and beyond wasting my time (as I should just collect in my own file of reviews or "share" on my own site) is it not a great conflict of interest going on here?
Egregore 15y ago
If you reviewed something you might refer to it later, for example - who was that carpenter we used last year?

jrockway 15y ago

"I don't like Yelp, so here are some random unsubstantiated complaints about the reviews about Yelp itself." The author seems to think people hate Yelp, but I'm not sure that's the case. Everyone I know in real life uses it regularly when trying to find some place to go, and the results are largely acceptable. Review sites are always going to trend negative because people who have had average or good experiences aren't going to be driven by rage to write a nasty review. Everyone has their own star scale and expectations of service ("I had to sit in economy class on my $10 ticket! I'm never flying United again!"). This leads to useless star ratings, but this is no fault of Yelp itself. It's the fault of relying on non-professionals to do professional-quality work. But, if you read for content, you can usually figure out whether a place is good or not. For example, a review like "I went during the dinner rush on Saturday night and it took 5 minutes to get a table! 1 star! Oh yeah, the food was good." is a positive review, even though the reviewer only gave one star. So anyway, don't hate on Yelp, hate on the clueless people clueless writing reviews.

jhdavids8 15y ago
Unsubstantiated complaints? I simply wrote what I saw, besides a few opinions here and there. And I know a lot of people do indeed dislike Yelp, but like some other sites with huge user bases, there's little other options to turn to, so you're stuck with using Yelp. I actually use Yelp quite a bit myself. Anyway, the evidence is clear for there to be good reason to hate on Yelp (see my article). But like you said, there's also good reason to hate on clueless reviewers. Why can't we do both until either issue is fixed?
- jrockway 15y ago
  Unsubstantiated complaints? You claim that Yelp is writing their own 5 star reviews: Personally, I wouldn't be shocked if Yelp is putting in its own 5 star reviews for itself in order to offset the barrage of negative reviews. Common perception of Yelp is not good, far from their current 3-star rating. Just glancing over the first few 4 and 5-star reviews for Yelp, all but two came from an 'Elite' member. Of course they review Yelp highly, they get thrown crazy parties. And I wouldn't be shocked if you lose your Elite status for a negative Yelp review. This is pure FUD. You don't know if you lose elite status for a negative review. Not everyone gets to go to crazy parties. If this were Wikipedia, every statement would be tagged [weasel words] or [citation needed]. You've just made a bunch of stuff up. If this is just entertainment, fine, but if you think you're writing something useful that is worthy of HN, then you're a little off. Your article boils down to links to filtered reviews not being obvious enough (even though the link is the same as all the other links on the page), complaints about reCaptcha, which Yelp doesn't even control, and then a bunch of made-up facts/anecdotes that you have done no research to prove true or false. Look, I don't even like Yelp. But this is simply sloppy journalism, and it reads like it was written by a fifth grader who got a bunch of one-star reviews on Yelp.

progolferyo 15y ago

Interesting article. I don't understand the filtered review system at all. Beyond the 'he said / she said' complaints that occasionally come out, there are things about their system that simply don't make any sense unless Yelp is incompetent or slimy. For example: - When you post a review, you as a reviewer think its unfiltered forever. When you revisit the page as a logged in user and read a place that has your review, your review is visible. When you log out or log in as another user, the review is filtered and hidden. At the very least, it should tell you your review is filtered, I see no reason to pretend the review is not filtered when the review is legitimate. - When you view unfiltered results, the per page number mysteriously changes to 10 per page. I don't see any reason why this should change. Plus the results are pretty slow to load, quite slower than the results for filtered reviews. - Why do you need to enter in a captcha to view the unfiltered reviews? Why would they care if you were a bot only for the unfiltered reviews and not the normal reviews? I don't see the difference, unless they want to prevent people from writing scripts to pull in unfiltered review data. Plus the captcha is fucking horrible, literally half the captcha's I get are not readable and I need to refresh. - The filter algorithm seems to be clearly flawed and simply catches way too many reviews that should not be filtered. For example, take this user: http://www.yelp.com/user_details?userid=tZlbsUVo-8wtnR7oMa-3xg http://www.yelp.com/user_details?userid=tZlbsUVo-8wtnR7oMa-3... . The guy has 11 reviews, 1 1-star review, 1 2-star review and nothing out of the ordinary and yet his review about Yelp was filtered. Why? His points in the review seemed legitimate. He seems to be a normal user, not a new user and posts reviews across the board (more good reviews than bad in fact). They should either fix the algorithm or be more transparent about why reviews are filtered because I can't understand why a review like that is filtered.

pbz 15y ago
"Why do you need to enter in a captcha to view the unfiltered reviews?" Maybe they don't want Google to see them...
- SoftwareMaven 15y ago
  That's what robots.txt is for, not silly captchas. You put captchas for the people who ignore that file.

deleted 15y ago

[deleted]

deleted 15y ago
[deleted]

mathattack 15y ago

Organizations like Yelp live and die on their accuracy and predictive ability. If Netflix recommends movies you don't like, you will use it less. If Zagat allows too much personal preference, it will no longer be the gold (Maroon?) standard of crowd sourced restaurant reviews. If half the things suggested about Yelp are true, they risk their integrity at great peril.

cHalgan 15y ago
Actually, Yelp can play game like this and be profitable. If business have a good rating on Yelp, then that pretty much mean that business is ok. However, a bad rating might be because business was not nice with Yelp sales people. That is ok because they target small, unestablished, businesses which you will never discover that they are actually excellent business because they have just one star review. In other words, advertising at Yelp does not guarantee you a good review. However, not being nice with their sales people might get you a bad review if you are a small business. And this is absolutely nothing new.

ZipCordManiac 15y ago

I've posted about 6 reviews in total on yelp, 2 of which were immediately filtered for no reason. Funny how my positive reviews went through just fine and the rest got binned. If you're going to run a review site, at least let people give their honest opinions don't filter anything other then profanity. People should be able to share an experience be it good or bad.

wallawe 15y ago
Although I wish it were that simple, it's more complicated than that. The reason for this simply is, if I own a business, I could create multiple fake accounts and have friends and and associates do the same. We would then be able to give multiple (fake) positive reviews for our own business, while giving one star feedback to our competition.

derwiki 15y ago

Yelp has more information on their web site about the review filter, soliciting reviews, false positives, and more: https://biz.yelp.com/support/common_questions https://biz.yelp.com/support/common_questions It's easy to jump on the bandwagon and say "Yelp is evil," but having been a near-constant point of controversy, it's something they do their best to address.

jprobert 15y ago

I think Yelp has always been a good service but obviously there is a disconnect between the business end of Yelp and the service end. The Chicago Tribune article is not the first to complain about the business practices of Yelp. Additionally, advertising on Yelp is some of the most expensive real estate on the web with CPM rates of $100-$300. Ultimately I think that directory sites such as Yelp will become obsolete as social search becomes more refined and relevant. My company is actually working on this and we thing that we have a good product so far (in alpha). http://www.cliqsearch.com http://www.cliqsearch.com

tlb 15y ago

Yelp's main defense is "It's done by an algorithm". That's not a valid defense for a bad business practice. They are accused of hiding positive reviews for businesses that don't pay them. You can perfectly well write this logic in Python for an filtering algorithm: def filter(review): ... if review.business.contacted_by_sales_count >= 3 and not review.business.paying_yelp and review.stars >= 4 and random() > 0.8: # 80% chance return true # yes, filter it ... Even if it's not written so explicitly they have to take responsibility for their algorithms.

derwiki 15y ago
The sales org only has access to Salesforce copy of Yelp's internal data, and numbers like "contacted_by_sales" never ends up back in the main Yelp database. It's not something the algorithm could take into account because it doesn't know about the sales org at all. Disclaimer: I used to work on the Yelp engineering team, at one point on the Salesforce data refresh project.
pbreit 15y ago
I'm sure most already understand this but when Yelp says the filter algorithmic, it's implied that a reasonable algorithm is being used.

eurohacker 15y ago

so are you saying that yelp is basicly an online extortion business, and judicial system and all the yelp users accept it ..?