13 ms·

Yelp, You Cost Me $2000 by Suppressing Genuine Reviews. Here’s How You Fix It

330 points by jv22222 15y ago

AznHisoka 15y ago
Yelp doesn't care because it doesn't affect their bottom line. If you're not a customer, you're the product.
- bloggergirl 15y ago
  "The customer is the product." - Don Draper :)
pg_bot 15y ago
Honestly the author is over thinking his solution to the problem. The way to solve this problem is to give people the option of viewing reviews that Yelp believes are of dubious origin. Add another section to the reviews page, and let the users decide how to process the extra information but frame it as possibly being untrustworthy with an explanation of why it was tagged as untrustworthy.
- colomon 15y ago
  They already allow you to click through to the reviews they have filtered out. How about having two average ratings, one for only "filtered in" reviews and one for all reviews?
  - dcpdx 15y ago
    And the font is about 10px and 30%K buried in the bottom corner. I can understand that would be desirable if the algorithm was more accurate in filtering bogus reviews, but that's clearly not the case so it just seems unscrupulous.
AlexBlom 15y ago
Wow. Surely Yelp can pick up on a continuous, consistent pattern of negative reviews. The hard part is staying one step ahead of less-legitimate accounts. There's always somebody trying to cheat the system, and stopping these users often results in a tradeoff.
- twoodfin 15y ago
  > Surely Yelp can pick up on a continuous, consistent pattern of negative reviews. But a continuous, consistent pattern of one-shot negative reviews is also not particularly difficult for a competing business (or anyone with a grudge) to reproduce. This is a hard problem, and I don't think the credit check-style identity verification questions suggested in the post are going to solve it. For one thing, I think an overwhelming number of people would find it creepy. They're trying to tell people about a bad moving company, not get a car loan. Putting more of your "valuable" online identity at stake seems likely to help. I hate to say it, but "Log in using your Facebook account" suddenly gives Yelp a lot more data to exploit to decide just how much trust they should put in your reviews.
  - jv22222 15y ago
    Why would they find it creepy? You wouldn't need to say your using credit history. It would be as simple as asking the user a question "Please validate your a real person by answering these few questions". As far as the user is concerned it's just a few questions and answers.
    - masterzora 15y ago
      Have you seen the questions they use? They range from "which of these addresses have you previously lived at" to "which of these companies do you have a loan with" to "how much is your loan payment" among others. They're pretty creepy and made even more so by the fact that these are the same questions you answer to prove your identity to, say, the credit bureaus or your creditors. It's not the sort of stuff one should be answering lightly.
    - egypturnash 15y ago
simonw 15y ago
The proposed solution sounds incredibly easy to game - as a moving company, I could create a bunch of fake, one-off profiles with complaints about one of my competitors and destroy their Yelp reputation. Unfortunately, I don't think it's possible for an algorithm to detect the difference between a bunch of angry one-time posters and a coordinated reputation attack. This is why I've gone off sites like Yelp for recommendations, and instead stick to places like Foursquare which rely much more heavily on the behaviour of the people that I know. Seeing that several of my friends have been somewhere more than once is a much stronger recommendation signal than a bunch of strangers giving something five stars.
- jv22222 15y ago
  You need a valid, fully qualified, unique credit history for each new account. How is that easy to game? Are you assuming that the moving company will get all their employees to to signup to Yelp and certify their credit history against fake sock puppet accounts? They maybe able to do it for a few accounts but pretty soon they would run out of real people to use up... in the mean time real reviews will be submitted by an endless pool of real people with valid credit history. It will be quite easy to tell which reviews are real and which are not. At the very least it will make the company look questionable because 50% of reviews will be 5 star and 50% will be one star. The main issue is we need to get valid reviews out of the filter so that Yelp users can more easily make their mind up. As it stands a lot of real reviews are hidden and a lot of fake reviews are exposed. Simply by exposing real reviews you level the playing field.
  - hncommenter13 15y ago
    Say what? Yelp is pulling credit histories of people who join the site? If they are, that is way out of bounds for what users typically expect when joining an online review site. As a simple reviewer, I'm not engaging in any business with them, they're not offering me credit, and pulling a credit report on some anonymous internet user would not only be expensive but would require some serious disclosure (as having credit reports pulled actually impacts one's credit score). Not only that, but if they are doing it, it doesn't seem to be working very well. Besides, I never supplied sufficient information for Yelp to connect me to my credit report without doing some seriously ethically questionable data mining. Why do you believe that Yelp does this? If so, that would be more surprising to most users than the pay-for-play behavior we're seeing here.
dhyasama 15y ago
The hiding of reviews has made Yelp virtually useless. I like the identity verification idea. I implemented the Equifax version and it's very simple to code. I believe it cost us in the neighborhood of $1/verification. That may outweigh the value of a verified Yelp user though, but I have no idea of the Yelp economics.
mrgreenfur 15y ago
Yelp has been routinely hiding bad reviews and fixing good reviews to the top. It is a total scam and a total extortion scheme for local businesses.
deleted 15y ago
[deleted]
ed209 15y ago
It would be interesting to see a crowd-sourced recommendation site based only on positive recommendations. Something like a Net Promoter score to tell me where is good. i.e. you can only leave feedback for a place if you would actually recommend it to someone. Every business can have bad days, and a negative review can put a prospective customer off even if actually 99% of the time that's a good business. This isn't really a comparative recommendation site like yelp, the scenario would be something like a user saying what they want to do (like eat burgers on friday) then people would only recommend place to go. Kind of like Quora for places. Eventually you could build a decent picture of business that are recommended multiple times to start making suggestions beyond the crowd sourced suggestions.
- spotting 15y ago
  I work for a startup in Sunnyvale that is trying to do something like that. Would you mind looking at (the web site listed in my profile) and describing what is still missing? edit: removed link
  - pbhjpbhj 15y ago
    Why not link it, http://spotlikes.com/ http://spotlikes.com/? FWIW I watched the video (all the women have moustaches!) but am no more informed as to how it works. I also looked at the business sign up page but couldn't find any thing there to help work out the mechanics of the process. It looks like you integrate with FB closely?
    - spotting 15y ago
      Thanks for your feedback! I have gotten so little. :) I was unsure of the etiquette here at HN about pimping one's own site in the body of a post. I saw that others chose to put their site URLs in their profiles, but very few were posting them in comment text. As 'iamgilesbowkett mentions above, maybe it would be more informative to search over an aggregation of different opinions rather than hash mountains of conflicting reviews into a single byte digest. For the consumer, our site offers a way to casually discover nearby businesses based on how well they are liked by specific circles of people. All activity is contextualized by location; the target of attention is geo-tagged by lat-long, not the user. We try to reduce the effort required to browse through large amounts of business listing and business review data. The user should be able to quickly explore a specific area for businesses that match a search term, or are physically close to another (perhaps unrelated) business. Please feel free to surf around using the business search. On the site, business search itself is open to the world, but the core social parts require an account to participate. You can enter an object term plus a location. Search URLs can look like this: http://spotlikes.com/discover/places/noisebridge/san-francisco-CA http://spotlikes.com/discover/places/noisebridge/san-francis... Due to the founders' interest in Napa and Sonoma counties, we have lots of comments describing San Francisco North Bay Area small businesses. One entry point for browsing might be someplace like: http://spotlikes.com/business/benziger-family-winery-glen-ellen-CA-13804004 http://spotlikes.com/business/benziger-family-winery-glen-el... During after-release QA, sometimes I try to do a kind of major metropolitan area hopscotch, visiting all the largest US cities using the fewest clicks. For the SMB owner, our site offers businesses a place to make a simple listing - a page where they can post their basic details (phone number, URL, logo, etc) - and then sit back and watch the users stream that small referral link out in their activities. Yes, the site is tightly integrated with Facebook. User comments on business listings can flow automatically into social media sites. If you authorize it, your "likes" will flow onto your FB wall/timeline. We have similar Twitter and Foursquare integration. We have probably overlooked all kinds of stuff. We move pretty fast. If you would like to call out anything in particular, I am keenly interested in the feedback.
mbesto 15y ago
Question based identity verification can be very painless. For you, but not the vendor. See PayPal. Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian. And that leads to friction. Yelp got it's massive user base from frictionless technology. I don't like Yelp's business practices just as much as the next person, but this is a human interaction issue that isn't solved by a simple algorithm, nor is it solved purely by (mind you, costly) human being based committees.
antidaily 15y ago
I only use Yelp to see whats nearby.
timdorr 15y ago
So, the solution is replacing the CAPTCHA with my SSN? All to post a review? Yeah, that doesn't seem reasonable to me...
- pavel_lishin 15y ago
  Yelp wouldn't get your SSN; they'd only get a boolean response as to whether you're an actual human being or not.
  - steve8918 15y ago
    Unless, of course, Yelp or whoever else used the service also ran a man-in-the-middle and captured the information being passed back and forth.
    - jv22222 15y ago
      SSN would not need to be involved at any point during a question based verification process.
      - timdorr 15y ago
        How else would they know what credit report to look up and what questions to ask?
tocomment 15y ago
This sounds terrible. I wonder if this space is ripe for a new startup to gradaully take over? If you guys made a competitor to Yelp, how would you improve on it?
- spotting 15y ago
  We have put a lot of time and effort into making a local recommendation engine that is primarily powered by your real friends and acquaintances. I'm not sure how it compares to other sites, but you can see the URL in my profile.
gzervas 15y ago
True, Yelp got the filtering completely backwards in this instance. But, do you base any economic decision on two 5-star reviews from two Yelpers with 0 friends and 4 reviews combined?
- jv22222 15y ago
  Not everyone is a sophisticated Yelp user with an understanding of social networks. A lot of people will simply see two 5-star reviews, zero negative reviews, and think that looks like a safe bet.
  - gzervas 15y ago
    Are you saying it's Yelp's responsibility to protect unsophisticated users from themselves? Also, I am not quite sure "a lot of people" will behave as you describe.
    - suresk 15y ago
      If one has to become a sophisticated Yelp user and do a substantial amount of detective work in order to determine if Yelp reviews are legit in order to get any value out of it (or at least not get actively misled by it), then what value does Yelp provide?
lazerwalker 15y ago
Worth mentioning is Yelp's long (alleged) history of using review filtering as a de facto racket to 'encourage' businesses to sign up for their paid services. A company I used to freelance for found that after they stopped paying Yelp for a premium account, they started seeing more negative reviews and a number of their positive reviews were filtered out. They got almost daily phone calls from Yelp marketing types essentially suggesting that if they renewed their subscription, the negative reviews would 'disappear'. This was a few years ago, so maybe things have changed since then. Either way, I wouldn't hold your breath for a better review verification system.
- jvm 15y ago
  I can confirm that they still do this, my barber shop says they get hassled all the time.
  - patricklynch 15y ago
    Well, there goes my ability to trust Yelp reviews as impartial. But if I don't trust the reviews, Yelp effectively degrades into a complicated remix of Google Maps.
    - pavel_lishin 15y ago
      That's the only thing I use them for. It's more convenient to search for "bar" and see a list of actual boozeries, vs. searching Google Maps and finding metal manufacturers, and various businesses that happen to have "bar" in their name.
    - mortenjorck 15y ago
      I use Yelp mostly for its metadata, such as restaurants' '$/$$/$$$' price categories, or a quick, easy-to-read menu sampling. I give the reviews approximately zero weight at this point in Yelp's evolution.
      - zem 15y ago
        search by "near me", sort by "open now" is an invaluable feature
3am 15y ago
There's no technical workaround for having good references from trusted source for something as important as movers, a dentist, a skilled tradesperson, a doctor, et al. (edit: I don't care, it's true. Even Angie's List isn't great)
underwater 15y ago
I don't know if he's just being coy by saying this is an accident on Yelp's part. The actual reviews are here: http://www.yelp.com/biz/royal-transportation-moving-and-storage-valley-village http://www.yelp.com/biz/royal-transportation-moving-and-stor.... It's pretty obvious there is something fishy about the unfiltered reviews. Each reviewer first left a one line review "Amazing!!! one of a kind..one of the best sushi place" and "the best of the best" before their very detailed, glowing review of the movers. If that's what it takes to fool the spam filter then something is very wrong. By contrast one of the filtered reviewers has ten measured, thorough reviews but his review has been banned for ToS violations. Yelp really needs to disclose which vendors have a business relationship with them. They should also make the filtered reviews easier to see (place the link next to the unfiltered review count) and remove the damn CAPTCHA. I regularly try to check the filtered reviews but can't because the CAPTCHA is too difficult to read.
- jisaacstone 15y ago
  Damn, I never new there was a CAPTCHA. really really really makes me suspicious. Just did a check to make sure that google does not integrate yelp reviews (it does not) since that is what I mostly use. also worth noting that "bob" liked the company so much he cross-posted his review to google. Rather interesting to see his google review history, as long as we are speculating . . . http://maps.google.com/maps/user?uid=213714492309658467573&hl=en-US&gl=US&ved=0CCgQhQo&sa=X&ei=reCWT7_FLIXLObjPqLkI http://maps.google.com/maps/user?uid=213714492309658467573&#... _____ edit: also a bit of fun to compare these two reviews: http://www.mymovingreviews.com/movingreviews/myself-and-my-family-recommend-american-eagle-movers-2776 http://www.mymovingreviews.com/movingreviews/myself-and-my-f... http://www.yelp.com/biz/royal-transportation-moving-and-storage-valley-village#hrid:gJrVrNhglnb74dW52QQvyQ http://www.yelp.com/biz/royal-transportation-moving-and-stor...
- colomon 15y ago
  Maybe I'm doing something wrong, but I was unable to find the sushi reviews here... Is Yelp changing the reviews as we read them?
  - 15y ago
edw519 15y ago
1995 Q: How can you verify the quality of a < mover / restaurant / hairdresser / babysitter / dentist / chiropractor / handyman / etc.>? A: Ask your < friend / relative / neighbor / workmate / schoolmate / etc.>. 2009 Q: Same question. A: Don't need no stinkin' people. We have the internets! 2012 Q: Internet solution doesn't work. How do we fix it? A: Use the 1995 solution.
- grogs 15y ago
  The Internet should not replace the 1995 solution. You should always use a hybrid approach. If a friend recommends Avengers Assemble, I go see it. If I end up at the cinema out of boredem, I'll let imdb help me. I want some website to come along which solves this for me, and neither Facebook nor Hunch nor IMDB do really.
- mcknz 15y ago
  1995 corollary: the < mover / restaurant / hairdresser / babysitter / dentist / chiropractor / handyman / etc.> is invariably a brother- or sister-in-law.
  - TylerE 15y ago
    Not always a bad thing. All but the worst of the worst are going not want to screw over a relative, even a distant one.
    - JeffBlauser 15y ago
      That's why they say, "Always do business with family."
    - aperrien 15y ago
      Not always true. People look guard and abuse their own first. "Exploitation begins at home"
- gdilla 15y ago
  Maybe a shamless plug, but I work on mobile apps at Fodor's Travel. We've been doing print travel guides for 75 years. We now have apps, in addition to books. We're professional curators. That means every restaurant review (and shops, hotels, etc) has been written by an actual human professional who's opinion we hope you trust (we're a little more upscale than Lonely Planet, but we have our hip moments). So in a way, we're the anti-UGC. When you have crowds of people, all the ratings become a noisey 3-4 stars. Including Burger King. That's why I think Fodor's content is a good alternative. We're in the travel space, not local, but, it's a refreshing alternative. Oh, and all our apps are FREE. http://www.fodors.com/mobile-apps/ http://www.fodors.com/mobile-apps/
mamasays 15y ago
Yelp has been having problems for a long time, especially with small business owners. The site is killing small business owners, just do a Google search and you will find endless complaints about the site. Truth is Yelp cannot control the reviews, they can't guarantee them and their Filter... if it works only 50% of the time then it is worthless.
bobbydavid 15y ago
Disclaimer: I don't work for Yelp but I have created other user-ratings-based sites before. It's not entirely fair to criticize Yelp for under-emphasizing negative reviews. One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting. You work harder to get positive reviews than negative ones. If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied. This should be the motivation for preferencing good reviews over lots of one-off bad reviews. BUT: Yelp made the cardinal sin of "objective" ratings sites: their profit comes from the entities they are rating. Ergo everything on their site is questionable.
- dfxm12 15y ago
  If this bias occurs across all businesses on Yelp, then still, wouldn't some places will be rated higher than others (even if "higher rated" comes to mean "has fewer bad reviews")? Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.
- bloggergirl 15y ago
  "people are most motivated to write reviews when they are outraged. ...your site runs the risk of becoming a soap box for ranting." There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you. No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.
  - Kesty 15y ago
    People are way more willing to take the time to write a bad review if they feel that something wrong was done to them than take the same time to leave a positive review, even if you ask them to. The only one that will leave a good review will be Yelp users that rates everything. And maybe people that you did them a big favor and they feel they have to repay you someway.
ckdarby 15y ago
I have no pity for this guy...seriously he only went by Yelp's reviews and did no further research. This article just proves how stupid consumers can be when making important decisions. When I read the article I didn't see the guy mentioning looking up the company on the BBB or even simply Google'ing "[Company] scams" or "[Company] scammers". Yes, Yelp could implement better solutions to protect consumers but ultimately the consumer needs to do adequate research before purchasing.
- bcrescimanno 15y ago
  So there's a resource out there that claims to let the customer do a good portion of that research by aggregating real customer reviews and you berate him for using that research? Calling the guy "stupid" for believing a bunch of supposedly satisfied real customers while at least 10 extremely negative customer experiences were hidden from him is bullshit.
Symmetry 15y ago
This seems like pretty much the exact problem that social networking was designed to solve.
teen 15y ago
I have a friend that works as a veterinarian. Yelp called his business often to try to get an advertising deal. After he declined, many of his 5 star reviews disappeared from the site. After losing a lot of business, he signed up for the advertising. The next day all of the 5 star reviews returns. This was just a few months ago.
- glanch 15y ago
  This has been confirmed from tons of people. How on earth has Yelp not been dismantled by the courts and their CEO put in jail? This is insane. Businesses now have to pay "protection" money to the Yelp mafia. WHY HAS YELP NOT BEEN SHUT DOWN? I FEEL LIKE I'M TAKING CRAZY PILLS!
- derwiki 15y ago
  Then he should sue. Every case brought against Yelp has been thrown out so far. I'd imagine that if this racket stuff happened, at least -one- case would come to fruition.
Otto42 15y ago
Wait, people actually use Yelp? Seriously? I find that to be the most shocking statement in the article.
- NZ_Matt 15y ago
  Of course they do, why else would it be so popular?
tatsuke95 15y ago
Over and over and over again I read negative articles about Yelp. These kind of places are ripe for scammers. These businesses know that you're randomly searching for a service, don't know you and don't plan on ever seeing you again. Yelp's system is, apparently, very easy to game. They've been accused of extortion. And yet, billion dollar valuation. It's laughable. We're so out of touch that people are talking about how this space is ripe for disruption and a new startup. Umm, Yelp is an extremely young company. I guess once you're public you're seen as old hat? Pump and dump, and move on to the next thing. Unbelievable.
pitdesi 15y ago
Is there a startup opportunity for an identity verification service whereby you do the ID verification once and then each service that wants to use it gets charged a smaller amount? Example: consumer goes to idverificationsite.com, registers there for free- they ask out of wallet questions to verify identity, potentially do a minimal background check too. You then use this login with Yelp, AirBnB, Getaround, etc. Yelp, Getaround, Airbnb pay idverificationsite.com some small amount per verified person. Removes some of the friction of ID verification/spreads cost around. Sidenote: the BBB is the worst offender of these protection rackets - http://feefighters.com/blog/bbb-accreditation/ http://feefighters.com/blog/bbb-accreditation/ Also - I've just decided to start posting reviews to Google rather than Yelp, they lost me as a product after reading this thread.
- Lewisham 15y ago
  Google actually used to do verification on people with Knol; you'd type in a credit card number and it would look up your name on the CC and match it against the name in your profile. I think that functionality got retired. It would be nice if this service was available more globally.
- Philadelphia 15y ago
  There was an interesting article in Inc. magazine a while back about a company that tried that. They found out that no sites wanted to use the service, even though they saw how it might solve their problems, because it would immediately make their non-verified users less valuable. http://www.inc.com/magazine/201112/case-study-josh-levy-and-ross-cohen-of-beenverified.html http://www.inc.com/magazine/201112/case-study-josh-levy-and-...
- robatsu 15y ago
  There are at least two startup opportunities for something like this. One is the creation of the credential/verified identity certification and the second is the transmission/exchange of the credential. These are two very disparate set of competencies. Several years ago, I was working with Honesty Online, doing exactly this identity verification startup prop and where it became very evident that this is really two businesses. Now we, in another startup, sigkat.com, are very early stage in the second business - creation/exchange/transmission of any sort of credentials so that vertical apps like identity verification can focus on their domain.
roguecoder 15y ago
We've seen this result before, from Wikipedia: http://thedartmouth.com/2007/10/22/news/wiki http://thedartmouth.com/2007/10/22/news/wiki They found that there were two patterns of content contribution: people who wrote one review and people who wrote lots. For people who wrote one review, anonymous contributions were higher quality than signed, and those single reviews were higher quality than people who contributed multiple reviews. Chances are good your most valuable reviews are coming from the long tail and your least accurate reviews are coming from people who are posting lots, but potentially with sockpuppet accounts. Tossing out your most accurate reviews in an attempt to suppress the latter seems counterproductive.
- shangrila 15y ago
  This needs to be higher. This is exactly the reason why what the article calls "Assumption X" is flawed.
pbreit 15y ago
I generally defend Yelp against a lot of mis-informed opinions but this is pretty bad. It's clear Yelp is hiding a lot of legit reviews. There's a decent case Yelp should be erroring on the side of showing not hiding. At a minimum, Yelp clearly needs to improve its algorithms. As the OP describes, a bad experience with a service provider could easily lead to a 1 star review from a first time reviewer. And these are very helpful reviews.
ChuckMcM 15y ago
Yelp is the poster child of the disingenuous Internet problem. In a nutshell the problem is that people who are not verifiable by "the service" provide information or data which is essential to the operation of the "the service." Further the beneficiaries of the service (customers) are not paying for the service, and the people who can be harmed or benefit from the service (establishments) have every motivation to game the system to increase business and little downside. So your business which has modestly thin margins to begin with, grows slowly, and the more popular it gets the more it tries to get gamed and the more margins are squeezed if you're paying people to filter those reviews. There aren't really any 'good' answers. But there are other examples of businesses like this that have done well. The ones that do well however use the 'review' portal to push more customers to the 'value' area where higher rates of returns are to be had. I hope someone writes up Yelp, Ripoffreport, TripAdvisor, Expedia, and AngiesList up for a Harvard Business Review group study. There have been a lot of experiments in this space taking different tacks and it would be interesting to compare their effectiveness.
- SomeCallMeTim 15y ago
  I swear by AngiesList for services like moving companies. They just called us up a couple weeks ago and sat on the phone for over a half hour (because we were willing) taking reviews from us on a bunch of companies that we had used. This is why it costs money; they actually provide a service. If they don't have enough reviews, they hire people to call you up and write up the reviews for you.
  - vbl 15y ago
    Yea, but they were the first ones to restart advertising with Rush Limbaugh. And their system is just as game-able, albeit at a slight cost.
    - muhfuhkuh 15y ago
      It's a pretty expensive undertaking. You have to be a subscriber (at non-trivial cost) to be a member/reviewer. The vendor would have to create multiple puppet accounts at $20/pop to shill up reviews. But, what of all the other actual reviewers that will write bad ones? You tend to see patterns in the reviews and, when a shining review of a vendor leaves me skeptical, I always look at the members' other reviews to see if they left at least 2 or more. If they did, then I suspect less that they're in cahoots with the one vendor. The Limbaugh thing is unfortunate, that's true.
bbrtyth 15y ago
Remember, everyone, if you aren't paying for it, you might be the product. In this case, a Yelp user is the product that Yelp sells to businesses by featuring 5-star reviews and burying the 1-star review. Featuring the 1-stars and burying the 5-stars would also be quite the incentive for a business to pay Yelp, but it would be cynical to assume they are doing that, too.
- mrobataille 15y ago
  I agree with you completely.
debacle 15y ago
I'm not surprised at all. Hasn't Yelp been considered kind of shady for yearS?
Aloisius 15y ago
Personally I just Yelp would have some sort of collaborative filtering engine. They kind of have a social one, but my friends' opinions are often as bad as a random stranger's. What people review on Yelp is subjective and Yelp makes it hard to filter out people's biases because of their one dimensional rating system. When people review a restaurant, they review the service, the food, the atmosphere and the overall value. For some people, paying $200 on a meal is outrageous regardless of how epic the food might be. For others, they don't even look at the check when they pay and value doesn't factor into their ratings. Yelp doesn't ask you to rank the businesses you like on a scale of best to worst, so people rate things based on some internal weighting scale that probably shifts slightly from review to review. I may have a 5-star taco trunk right next to a 5-star review for the French Laundry. They aren't equal. What people review is also temporal. God help a business if there is say, some construction going on across the street, the patron was in a bad mood and becomes argumentative or the chef is sick and food quality dips for a night. Or maybe the waiter just broke up with his girlfriend. Maybe they run out of the one thing the patron really wants to eat. Any of these things can make someone walk away and feel the business is poor. Businesses and people also change over time, I have reviews on Yelp for 5 stars from 5 years ago for places that have either gone downhill or simply don't match my current taste. People also tend to only review things they either really liked/loved or hated which results in most businesses (especially in SF) as being rated 3.5-4 stars making Yelp pretty useless without looking at the underlying reviews. A collaborative filtering engine could fix a lot of these problems. At the very least I'll match people who have my own biases and normalize people's ratings. Of course it still doesn't fix the temporal issues, but I don't think that's insurmountable.
- cloudkj 15y ago
  That's an interesting idea. Is the main motivation of your suggestion to find a way to enable you to better discover things you might like without relying purely on reviews and ratings? In general, I agree that ratings and reviews have inherent biases that are hard to smooth out. There are so many more dimensions to it, such as how much weight you apply to a review written by someone that is trying a certain type of food for the first time. Do you think that you would see better results if Yelp was able to use a collaborative filtering approach to somehow personalize their results such that they more closely match your tastes and preferences - based on other users' data?
int3rnaut 15y ago
Would anyone be interested in disrupting this space? My friend and I have been working on something we quite like that deals with subjectivity/ethics concerns, but it's a tough space to be in, and we'd love some help. If you want to get in touch, email us at questionasker[at]liveDotCA
droithomme 15y ago
The problem with the proposed solution of verifying ID is that people who leave honest bad reviews often enough get harassment sued by angry dishonest business owners, who it turns out are more likely to harass and sue companies than the honest companies are. The main problem here is that Yelp is a racket as well, this has been known for some time. Those using it, well caveat emptor. There are other review sites out there that are not as morally compromised as Yelp due to their business model being modeled on a mafia protection racket of "Pay us and we can help these bad reviews go away."
IanDrake 15y ago
Any review on Yelp that says "We" or mentions the name of the person that assisted them at company is most certainly fake. Example: "We were really worried about where to stay on Las Vegas, but then we called hotel X and Margo sounded so nice and welcoming on the phone that we just knew it was the right choice."
robatsu 15y ago
Identity verification and such are only part of the problem. Even if it is a bona fide person writing the review (and ignoring the problems w/Yelp filtering practices), a truly valid review only comes from one who the readers can be assured really was a customer of Royal Movers in addition to some of the other, lower identity bars. So how could that work?
pauljonas 15y ago
I stopped using Yelp after I noted that reviews I submitted were only visible to me, if I was logged into the site. That defeats the purpose of an online review site -- if the review is only visible to me, why am I posting to Yelp, and giving them valuable data they just will sit on, as opposed to logging in my own local host `reviews.txt`? I noticed also that a number of favorable reviews, seemingly written by authentic reviewers were also suppressed in like fashion. See Amazon for a site that gets posting reviews mostly correct.
garrett_smith 15y ago
Justin, as a business owner, you are not Yelps customer. Yelp customer, or at least the one they care about, is really the people who use the site.
darkblob32 15y ago
I use Yelp to get ideas about what places to eat, but recommendations and confirmations from friends is usually the final say. I really like Yelp, but things like this keep coming up and it's hard to trust that their heart is in the right place. Not to mention how slow they are at staying competitive with other services. Check-in feature seemed to take forever, and I still can't believe they haven't tried to implement customer appreciation cards a la Stampt.
JohnnyFlash 15y ago
This isn't really Yelp's fault. Sites like this always have fake reviews mingled in with those that are real. The solution? Assume people with a 4-5 star review are competent. GET MULTIPLE QUOTES. Doesn't matter if they have 5*, if you feel something is off with them they you avoid. You also will find out when a quote is too cheap/too expensive and reduce your risk to getting conned. Don't blame the internets or Yelp. Blame youself for not doing due diligence.
schiffern 15y ago
So, you want Angie's List. Is anyone surprised that a recommendation site that gets its money from the businesses is less than honest? Is anyone surprised that when they get popular they turn into a protection racket? Say it with me. "If you're not paying, you're the product." Or in this case, the club.
derwiki 15y ago
For the record: the sales organization at Yelp has a read-only copy of business data in Salesforce. Salespeople are not able to fiddle with reviews in the main database. They're two completely different systems. There's no possible way a salesperson can hide/show reviews to try and sell advertising. The salesperson can _claim_ to be able to do it, but they're lying.
- ceejayoz 15y ago
  Does Yelp policy forbid sales people from being friends with customer service / database / etc. folks?
exue 15y ago
Despite all its flaws, Yelp is the best solution I've had for finding good restaurants and other services. I love being able to read reviews by locals I might randomly see, browse through their profiles and see what else they like. In the vast majority of cases the crowdsourced reviews are awesome. Unfortunately their verification and filtering system is weak, and needs major improvement (an identity verification program would be awesome). Yelp is still a great service overall, and the fact that I use it over ZAGAT or anything else tells you a lot about how effective it is.
Sivart13 15y ago
What's most shocking at this point is that this story has been up for seven hours and Yelp hasn't bothered to fix the page in question, the wrongness of which could be observed by any human with eyes: http://www.yelp.com/biz/royal-transportation-moving-and-storage-valley-village http://www.yelp.com/biz/royal-transportation-moving-and-stor...
- jv22222 15y ago
  I couldn't agree more. It seems amazing that with thousands of eyeballs on this there has been no effort made to fix it.
- dcpdx 15y ago
  Wow, those 1-star reviews are pretty scathing too. I had no idea that negative reviews were filtered like that; that pretty much ruins my trust in the review component of the site. From now on, I'll be using Yelp for contact info. and price points...
rush-tea 15y ago
Like what everyone is saying, Yelp review system is very easy to game. Just create a one off profile, and hype the business. Yelp algorithm only allows good reviews getting posted while the bad ones are buried. Why? Because good reviews generate traffic to Yelp which in turn help Yelp getting money from advertiser. Five stars review system is flawed.
mapster 15y ago
I agree, though the example gripe is problematic as the moving industry is largely unregulated. It is full of outfits that are subcontracted by the big movers. Extortion is commonplace (moving truck is near your new home. they stop and call on phone asking for $2k more before they arrive and unload your property), and is considered a contract dispute and the police will not get involved, citing such.
Quizzy 15y ago
Let's talk about the mobile only world: would a native mobile app be self-verifying in that only a mobile phone could download and use the app - so every review requires a real smartphone to post the review. Although "reputation" companies could game this situation by buying 1000 smartphones, but then you would have 1000 smartphones with the same pattern of up-posting such that it would be easier for an algorithm to recognize these "planted" phones.

330 points by jv22222 15y ago

AznHisoka 15y ago

Yelp doesn't care because it doesn't affect their bottom line. If you're not a customer, you're the product.

bloggergirl 15y ago
"The customer is the product." - Don Draper :)

pg_bot 15y ago

Honestly the author is over thinking his solution to the problem. The way to solve this problem is to give people the option of viewing reviews that Yelp believes are of dubious origin. Add another section to the reviews page, and let the users decide how to process the extra information but frame it as possibly being untrustworthy with an explanation of why it was tagged as untrustworthy.

colomon 15y ago
They already allow you to click through to the reviews they have filtered out. How about having two average ratings, one for only "filtered in" reviews and one for all reviews?
- dcpdx 15y ago
  And the font is about 10px and 30%K buried in the bottom corner. I can understand that would be desirable if the algorithm was more accurate in filtering bogus reviews, but that's clearly not the case so it just seems unscrupulous.

AlexBlom 15y ago

Wow. Surely Yelp can pick up on a continuous, consistent pattern of negative reviews. The hard part is staying one step ahead of less-legitimate accounts. There's always somebody trying to cheat the system, and stopping these users often results in a tradeoff.

twoodfin 15y ago
> Surely Yelp can pick up on a continuous, consistent pattern of negative reviews. But a continuous, consistent pattern of one-shot negative reviews is also not particularly difficult for a competing business (or anyone with a grudge) to reproduce. This is a hard problem, and I don't think the credit check-style identity verification questions suggested in the post are going to solve it. For one thing, I think an overwhelming number of people would find it creepy. They're trying to tell people about a bad moving company, not get a car loan. Putting more of your "valuable" online identity at stake seems likely to help. I hate to say it, but "Log in using your Facebook account" suddenly gives Yelp a lot more data to exploit to decide just how much trust they should put in your reviews.
- jv22222 15y ago
  Why would they find it creepy? You wouldn't need to say your using credit history. It would be as simple as asking the user a question "Please validate your a real person by answering these few questions". As far as the user is concerned it's just a few questions and answers.
  - masterzora 15y ago
    Have you seen the questions they use? They range from "which of these addresses have you previously lived at" to "which of these companies do you have a loan with" to "how much is your loan payment" among others. They're pretty creepy and made even more so by the fact that these are the same questions you answer to prove your identity to, say, the credit bureaus or your creditors. It's not the sort of stuff one should be answering lightly.
  - egypturnash 15y ago

simonw 15y ago

The proposed solution sounds incredibly easy to game - as a moving company, I could create a bunch of fake, one-off profiles with complaints about one of my competitors and destroy their Yelp reputation. Unfortunately, I don't think it's possible for an algorithm to detect the difference between a bunch of angry one-time posters and a coordinated reputation attack. This is why I've gone off sites like Yelp for recommendations, and instead stick to places like Foursquare which rely much more heavily on the behaviour of the people that I know. Seeing that several of my friends have been somewhere more than once is a much stronger recommendation signal than a bunch of strangers giving something five stars.

jv22222 15y ago
You need a valid, fully qualified, unique credit history for each new account. How is that easy to game? Are you assuming that the moving company will get all their employees to to signup to Yelp and certify their credit history against fake sock puppet accounts? They maybe able to do it for a few accounts but pretty soon they would run out of real people to use up... in the mean time real reviews will be submitted by an endless pool of real people with valid credit history. It will be quite easy to tell which reviews are real and which are not. At the very least it will make the company look questionable because 50% of reviews will be 5 star and 50% will be one star. The main issue is we need to get valid reviews out of the filter so that Yelp users can more easily make their mind up. As it stands a lot of real reviews are hidden and a lot of fake reviews are exposed. Simply by exposing real reviews you level the playing field.
- hncommenter13 15y ago
  Say what? Yelp is pulling credit histories of people who join the site? If they are, that is way out of bounds for what users typically expect when joining an online review site. As a simple reviewer, I'm not engaging in any business with them, they're not offering me credit, and pulling a credit report on some anonymous internet user would not only be expensive but would require some serious disclosure (as having credit reports pulled actually impacts one's credit score). Not only that, but if they are doing it, it doesn't seem to be working very well. Besides, I never supplied sufficient information for Yelp to connect me to my credit report without doing some seriously ethically questionable data mining. Why do you believe that Yelp does this? If so, that would be more surprising to most users than the pay-for-play behavior we're seeing here.

dhyasama 15y ago

The hiding of reviews has made Yelp virtually useless. I like the identity verification idea. I implemented the Equifax version and it's very simple to code. I believe it cost us in the neighborhood of $1/verification. That may outweigh the value of a verified Yelp user though, but I have no idea of the Yelp economics.

mrgreenfur 15y ago

Yelp has been routinely hiding bad reviews and fixing good reviews to the top. It is a total scam and a total extortion scheme for local businesses.

deleted 15y ago

[deleted]

ed209 15y ago

It would be interesting to see a crowd-sourced recommendation site based only on positive recommendations. Something like a Net Promoter score to tell me where is good. i.e. you can only leave feedback for a place if you would actually recommend it to someone. Every business can have bad days, and a negative review can put a prospective customer off even if actually 99% of the time that's a good business. This isn't really a comparative recommendation site like yelp, the scenario would be something like a user saying what they want to do (like eat burgers on friday) then people would only recommend place to go. Kind of like Quora for places. Eventually you could build a decent picture of business that are recommended multiple times to start making suggestions beyond the crowd sourced suggestions.

spotting 15y ago
I work for a startup in Sunnyvale that is trying to do something like that. Would you mind looking at (the web site listed in my profile) and describing what is still missing? edit: removed link
- pbhjpbhj 15y ago
  Why not link it, http://spotlikes.com/ http://spotlikes.com/? FWIW I watched the video (all the women have moustaches!) but am no more informed as to how it works. I also looked at the business sign up page but couldn't find any thing there to help work out the mechanics of the process. It looks like you integrate with FB closely?
  - spotting 15y ago
    Thanks for your feedback! I have gotten so little. :) I was unsure of the etiquette here at HN about pimping one's own site in the body of a post. I saw that others chose to put their site URLs in their profiles, but very few were posting them in comment text. As 'iamgilesbowkett mentions above, maybe it would be more informative to search over an aggregation of different opinions rather than hash mountains of conflicting reviews into a single byte digest. For the consumer, our site offers a way to casually discover nearby businesses based on how well they are liked by specific circles of people. All activity is contextualized by location; the target of attention is geo-tagged by lat-long, not the user. We try to reduce the effort required to browse through large amounts of business listing and business review data. The user should be able to quickly explore a specific area for businesses that match a search term, or are physically close to another (perhaps unrelated) business. Please feel free to surf around using the business search. On the site, business search itself is open to the world, but the core social parts require an account to participate. You can enter an object term plus a location. Search URLs can look like this: http://spotlikes.com/discover/places/noisebridge/san-francisco-CA http://spotlikes.com/discover/places/noisebridge/san-francis... Due to the founders' interest in Napa and Sonoma counties, we have lots of comments describing San Francisco North Bay Area small businesses. One entry point for browsing might be someplace like: http://spotlikes.com/business/benziger-family-winery-glen-ellen-CA-13804004 http://spotlikes.com/business/benziger-family-winery-glen-el... During after-release QA, sometimes I try to do a kind of major metropolitan area hopscotch, visiting all the largest US cities using the fewest clicks. For the SMB owner, our site offers businesses a place to make a simple listing - a page where they can post their basic details (phone number, URL, logo, etc) - and then sit back and watch the users stream that small referral link out in their activities. Yes, the site is tightly integrated with Facebook. User comments on business listings can flow automatically into social media sites. If you authorize it, your "likes" will flow onto your FB wall/timeline. We have similar Twitter and Foursquare integration. We have probably overlooked all kinds of stuff. We move pretty fast. If you would like to call out anything in particular, I am keenly interested in the feedback.

mbesto 15y ago

Question based identity verification can be very painless. For you, but not the vendor. See PayPal. Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian. And that leads to friction. Yelp got it's massive user base from frictionless technology. I don't like Yelp's business practices just as much as the next person, but this is a human interaction issue that isn't solved by a simple algorithm, nor is it solved purely by (mind you, costly) human being based committees.

antidaily 15y ago

I only use Yelp to see whats nearby.

timdorr 15y ago

So, the solution is replacing the CAPTCHA with my SSN? All to post a review? Yeah, that doesn't seem reasonable to me...

pavel_lishin 15y ago
Yelp wouldn't get your SSN; they'd only get a boolean response as to whether you're an actual human being or not.
- steve8918 15y ago
  Unless, of course, Yelp or whoever else used the service also ran a man-in-the-middle and captured the information being passed back and forth.
  - jv22222 15y ago
    SSN would not need to be involved at any point during a question based verification process.
    - timdorr 15y ago
      How else would they know what credit report to look up and what questions to ask?

tocomment 15y ago

This sounds terrible. I wonder if this space is ripe for a new startup to gradaully take over? If you guys made a competitor to Yelp, how would you improve on it?

spotting 15y ago
We have put a lot of time and effort into making a local recommendation engine that is primarily powered by your real friends and acquaintances. I'm not sure how it compares to other sites, but you can see the URL in my profile.

gzervas 15y ago

True, Yelp got the filtering completely backwards in this instance. But, do you base any economic decision on two 5-star reviews from two Yelpers with 0 friends and 4 reviews combined?

jv22222 15y ago
Not everyone is a sophisticated Yelp user with an understanding of social networks. A lot of people will simply see two 5-star reviews, zero negative reviews, and think that looks like a safe bet.
- gzervas 15y ago
  Are you saying it's Yelp's responsibility to protect unsophisticated users from themselves? Also, I am not quite sure "a lot of people" will behave as you describe.
  - suresk 15y ago
    If one has to become a sophisticated Yelp user and do a substantial amount of detective work in order to determine if Yelp reviews are legit in order to get any value out of it (or at least not get actively misled by it), then what value does Yelp provide?

lazerwalker 15y ago

Worth mentioning is Yelp's long (alleged) history of using review filtering as a de facto racket to 'encourage' businesses to sign up for their paid services. A company I used to freelance for found that after they stopped paying Yelp for a premium account, they started seeing more negative reviews and a number of their positive reviews were filtered out. They got almost daily phone calls from Yelp marketing types essentially suggesting that if they renewed their subscription, the negative reviews would 'disappear'. This was a few years ago, so maybe things have changed since then. Either way, I wouldn't hold your breath for a better review verification system.

jvm 15y ago
I can confirm that they still do this, my barber shop says they get hassled all the time.
- patricklynch 15y ago
  Well, there goes my ability to trust Yelp reviews as impartial. But if I don't trust the reviews, Yelp effectively degrades into a complicated remix of Google Maps.
  - pavel_lishin 15y ago
    That's the only thing I use them for. It's more convenient to search for "bar" and see a list of actual boozeries, vs. searching Google Maps and finding metal manufacturers, and various businesses that happen to have "bar" in their name.
  - mortenjorck 15y ago
    I use Yelp mostly for its metadata, such as restaurants' '$/$$/$$$' price categories, or a quick, easy-to-read menu sampling. I give the reviews approximately zero weight at this point in Yelp's evolution.
    - zem 15y ago
      search by "near me", sort by "open now" is an invaluable feature

3am 15y ago

There's no technical workaround for having good references from trusted source for something as important as movers, a dentist, a skilled tradesperson, a doctor, et al. (edit: I don't care, it's true. Even Angie's List isn't great)

underwater 15y ago

I don't know if he's just being coy by saying this is an accident on Yelp's part. The actual reviews are here: http://www.yelp.com/biz/royal-transportation-moving-and-storage-valley-village http://www.yelp.com/biz/royal-transportation-moving-and-stor.... It's pretty obvious there is something fishy about the unfiltered reviews. Each reviewer first left a one line review "Amazing!!! one of a kind..one of the best sushi place" and "the best of the best" before their very detailed, glowing review of the movers. If that's what it takes to fool the spam filter then something is very wrong. By contrast one of the filtered reviewers has ten measured, thorough reviews but his review has been banned for ToS violations. Yelp really needs to disclose which vendors have a business relationship with them. They should also make the filtered reviews easier to see (place the link next to the unfiltered review count) and remove the damn CAPTCHA. I regularly try to check the filtered reviews but can't because the CAPTCHA is too difficult to read.

jisaacstone 15y ago
Damn, I never new there was a CAPTCHA. really really really makes me suspicious. Just did a check to make sure that google does not integrate yelp reviews (it does not) since that is what I mostly use. also worth noting that "bob" liked the company so much he cross-posted his review to google. Rather interesting to see his google review history, as long as we are speculating . . . http://maps.google.com/maps/user?uid=213714492309658467573&hl=en-US&gl=US&ved=0CCgQhQo&sa=X&ei=reCWT7_FLIXLObjPqLkI http://maps.google.com/maps/user?uid=213714492309658467573&#... _____ edit: also a bit of fun to compare these two reviews: http://www.mymovingreviews.com/movingreviews/myself-and-my-family-recommend-american-eagle-movers-2776 http://www.mymovingreviews.com/movingreviews/myself-and-my-f... http://www.yelp.com/biz/royal-transportation-moving-and-storage-valley-village#hrid:gJrVrNhglnb74dW52QQvyQ http://www.yelp.com/biz/royal-transportation-moving-and-stor...
colomon 15y ago
Maybe I'm doing something wrong, but I was unable to find the sushi reviews here... Is Yelp changing the reviews as we read them?
- 15y ago

edw519 15y ago

1995 Q: How can you verify the quality of a < mover / restaurant / hairdresser / babysitter / dentist / chiropractor / handyman / etc.>? A: Ask your < friend / relative / neighbor / workmate / schoolmate / etc.>. 2009 Q: Same question. A: Don't need no stinkin' people. We have the internets! 2012 Q: Internet solution doesn't work. How do we fix it? A: Use the 1995 solution.

grogs 15y ago
The Internet should not replace the 1995 solution. You should always use a hybrid approach. If a friend recommends Avengers Assemble, I go see it. If I end up at the cinema out of boredem, I'll let imdb help me. I want some website to come along which solves this for me, and neither Facebook nor Hunch nor IMDB do really.
mcknz 15y ago
1995 corollary: the < mover / restaurant / hairdresser / babysitter / dentist / chiropractor / handyman / etc.> is invariably a brother- or sister-in-law.
- TylerE 15y ago
  Not always a bad thing. All but the worst of the worst are going not want to screw over a relative, even a distant one.
  - JeffBlauser 15y ago
    That's why they say, "Always do business with family."
  - aperrien 15y ago
    Not always true. People look guard and abuse their own first. "Exploitation begins at home"
gdilla 15y ago
Maybe a shamless plug, but I work on mobile apps at Fodor's Travel. We've been doing print travel guides for 75 years. We now have apps, in addition to books. We're professional curators. That means every restaurant review (and shops, hotels, etc) has been written by an actual human professional who's opinion we hope you trust (we're a little more upscale than Lonely Planet, but we have our hip moments). So in a way, we're the anti-UGC. When you have crowds of people, all the ratings become a noisey 3-4 stars. Including Burger King. That's why I think Fodor's content is a good alternative. We're in the travel space, not local, but, it's a refreshing alternative. Oh, and all our apps are FREE. http://www.fodors.com/mobile-apps/ http://www.fodors.com/mobile-apps/

mamasays 15y ago

Yelp has been having problems for a long time, especially with small business owners. The site is killing small business owners, just do a Google search and you will find endless complaints about the site. Truth is Yelp cannot control the reviews, they can't guarantee them and their Filter... if it works only 50% of the time then it is worthless.

bobbydavid 15y ago

Disclaimer: I don't work for Yelp but I have created other user-ratings-based sites before. It's not entirely fair to criticize Yelp for under-emphasizing negative reviews. One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting. You work harder to get positive reviews than negative ones. If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied. This should be the motivation for preferencing good reviews over lots of one-off bad reviews. BUT: Yelp made the cardinal sin of "objective" ratings sites: their profit comes from the entities they are rating. Ergo everything on their site is questionable.

dfxm12 15y ago
If this bias occurs across all businesses on Yelp, then still, wouldn't some places will be rated higher than others (even if "higher rated" comes to mean "has fewer bad reviews")? Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.
bloggergirl 15y ago
"people are most motivated to write reviews when they are outraged. ...your site runs the risk of becoming a soap box for ranting." There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you. No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.
- Kesty 15y ago
  People are way more willing to take the time to write a bad review if they feel that something wrong was done to them than take the same time to leave a positive review, even if you ask them to. The only one that will leave a good review will be Yelp users that rates everything. And maybe people that you did them a big favor and they feel they have to repay you someway.

ckdarby 15y ago

I have no pity for this guy...seriously he only went by Yelp's reviews and did no further research. This article just proves how stupid consumers can be when making important decisions. When I read the article I didn't see the guy mentioning looking up the company on the BBB or even simply Google'ing "[Company] scams" or "[Company] scammers". Yes, Yelp could implement better solutions to protect consumers but ultimately the consumer needs to do adequate research before purchasing.

bcrescimanno 15y ago
So there's a resource out there that claims to let the customer do a good portion of that research by aggregating real customer reviews and you berate him for using that research? Calling the guy "stupid" for believing a bunch of supposedly satisfied real customers while at least 10 extremely negative customer experiences were hidden from him is bullshit.

Symmetry 15y ago

This seems like pretty much the exact problem that social networking was designed to solve.

teen 15y ago

I have a friend that works as a veterinarian. Yelp called his business often to try to get an advertising deal. After he declined, many of his 5 star reviews disappeared from the site. After losing a lot of business, he signed up for the advertising. The next day all of the 5 star reviews returns. This was just a few months ago.

glanch 15y ago
This has been confirmed from tons of people. How on earth has Yelp not been dismantled by the courts and their CEO put in jail? This is insane. Businesses now have to pay "protection" money to the Yelp mafia. WHY HAS YELP NOT BEEN SHUT DOWN? I FEEL LIKE I'M TAKING CRAZY PILLS!
derwiki 15y ago
Then he should sue. Every case brought against Yelp has been thrown out so far. I'd imagine that if this racket stuff happened, at least -one- case would come to fruition.

Otto42 15y ago

Wait, people actually use Yelp? Seriously? I find that to be the most shocking statement in the article.

NZ_Matt 15y ago
Of course they do, why else would it be so popular?

tatsuke95 15y ago

Over and over and over again I read negative articles about Yelp. These kind of places are ripe for scammers. These businesses know that you're randomly searching for a service, don't know you and don't plan on ever seeing you again. Yelp's system is, apparently, very easy to game. They've been accused of extortion. And yet, billion dollar valuation. It's laughable. We're so out of touch that people are talking about how this space is ripe for disruption and a new startup. Umm, Yelp is an extremely young company. I guess once you're public you're seen as old hat? Pump and dump, and move on to the next thing. Unbelievable.

pitdesi 15y ago

Is there a startup opportunity for an identity verification service whereby you do the ID verification once and then each service that wants to use it gets charged a smaller amount? Example: consumer goes to idverificationsite.com, registers there for free- they ask out of wallet questions to verify identity, potentially do a minimal background check too. You then use this login with Yelp, AirBnB, Getaround, etc. Yelp, Getaround, Airbnb pay idverificationsite.com some small amount per verified person. Removes some of the friction of ID verification/spreads cost around. Sidenote: the BBB is the worst offender of these protection rackets - http://feefighters.com/blog/bbb-accreditation/ http://feefighters.com/blog/bbb-accreditation/ Also - I've just decided to start posting reviews to Google rather than Yelp, they lost me as a product after reading this thread.

Lewisham 15y ago
Google actually used to do verification on people with Knol; you'd type in a credit card number and it would look up your name on the CC and match it against the name in your profile. I think that functionality got retired. It would be nice if this service was available more globally.
Philadelphia 15y ago
There was an interesting article in Inc. magazine a while back about a company that tried that. They found out that no sites wanted to use the service, even though they saw how it might solve their problems, because it would immediately make their non-verified users less valuable. http://www.inc.com/magazine/201112/case-study-josh-levy-and-ross-cohen-of-beenverified.html http://www.inc.com/magazine/201112/case-study-josh-levy-and-...
robatsu 15y ago
There are at least two startup opportunities for something like this. One is the creation of the credential/verified identity certification and the second is the transmission/exchange of the credential. These are two very disparate set of competencies. Several years ago, I was working with Honesty Online, doing exactly this identity verification startup prop and where it became very evident that this is really two businesses. Now we, in another startup, sigkat.com, are very early stage in the second business - creation/exchange/transmission of any sort of credentials so that vertical apps like identity verification can focus on their domain.

roguecoder 15y ago

We've seen this result before, from Wikipedia: http://thedartmouth.com/2007/10/22/news/wiki http://thedartmouth.com/2007/10/22/news/wiki They found that there were two patterns of content contribution: people who wrote one review and people who wrote lots. For people who wrote one review, anonymous contributions were higher quality than signed, and those single reviews were higher quality than people who contributed multiple reviews. Chances are good your most valuable reviews are coming from the long tail and your least accurate reviews are coming from people who are posting lots, but potentially with sockpuppet accounts. Tossing out your most accurate reviews in an attempt to suppress the latter seems counterproductive.

shangrila 15y ago
This needs to be higher. This is exactly the reason why what the article calls "Assumption X" is flawed.

pbreit 15y ago

I generally defend Yelp against a lot of mis-informed opinions but this is pretty bad. It's clear Yelp is hiding a lot of legit reviews. There's a decent case Yelp should be erroring on the side of showing not hiding. At a minimum, Yelp clearly needs to improve its algorithms. As the OP describes, a bad experience with a service provider could easily lead to a 1 star review from a first time reviewer. And these are very helpful reviews.

ChuckMcM 15y ago

Yelp is the poster child of the disingenuous Internet problem. In a nutshell the problem is that people who are not verifiable by "the service" provide information or data which is essential to the operation of the "the service." Further the beneficiaries of the service (customers) are not paying for the service, and the people who can be harmed or benefit from the service (establishments) have every motivation to game the system to increase business and little downside. So your business which has modestly thin margins to begin with, grows slowly, and the more popular it gets the more it tries to get gamed and the more margins are squeezed if you're paying people to filter those reviews. There aren't really any 'good' answers. But there are other examples of businesses like this that have done well. The ones that do well however use the 'review' portal to push more customers to the 'value' area where higher rates of returns are to be had. I hope someone writes up Yelp, Ripoffreport, TripAdvisor, Expedia, and AngiesList up for a Harvard Business Review group study. There have been a lot of experiments in this space taking different tacks and it would be interesting to compare their effectiveness.

SomeCallMeTim 15y ago
I swear by AngiesList for services like moving companies. They just called us up a couple weeks ago and sat on the phone for over a half hour (because we were willing) taking reviews from us on a bunch of companies that we had used. This is why it costs money; they actually provide a service. If they don't have enough reviews, they hire people to call you up and write up the reviews for you.
- vbl 15y ago
  Yea, but they were the first ones to restart advertising with Rush Limbaugh. And their system is just as game-able, albeit at a slight cost.
  - muhfuhkuh 15y ago
    It's a pretty expensive undertaking. You have to be a subscriber (at non-trivial cost) to be a member/reviewer. The vendor would have to create multiple puppet accounts at $20/pop to shill up reviews. But, what of all the other actual reviewers that will write bad ones? You tend to see patterns in the reviews and, when a shining review of a vendor leaves me skeptical, I always look at the members' other reviews to see if they left at least 2 or more. If they did, then I suspect less that they're in cahoots with the one vendor. The Limbaugh thing is unfortunate, that's true.

bbrtyth 15y ago

Remember, everyone, if you aren't paying for it, you might be the product. In this case, a Yelp user is the product that Yelp sells to businesses by featuring 5-star reviews and burying the 1-star review. Featuring the 1-stars and burying the 5-stars would also be quite the incentive for a business to pay Yelp, but it would be cynical to assume they are doing that, too.

mrobataille 15y ago
I agree with you completely.

debacle 15y ago

I'm not surprised at all. Hasn't Yelp been considered kind of shady for yearS?

Aloisius 15y ago

Personally I just Yelp would have some sort of collaborative filtering engine. They kind of have a social one, but my friends' opinions are often as bad as a random stranger's. What people review on Yelp is subjective and Yelp makes it hard to filter out people's biases because of their one dimensional rating system. When people review a restaurant, they review the service, the food, the atmosphere and the overall value. For some people, paying $200 on a meal is outrageous regardless of how epic the food might be. For others, they don't even look at the check when they pay and value doesn't factor into their ratings. Yelp doesn't ask you to rank the businesses you like on a scale of best to worst, so people rate things based on some internal weighting scale that probably shifts slightly from review to review. I may have a 5-star taco trunk right next to a 5-star review for the French Laundry. They aren't equal. What people review is also temporal. God help a business if there is say, some construction going on across the street, the patron was in a bad mood and becomes argumentative or the chef is sick and food quality dips for a night. Or maybe the waiter just broke up with his girlfriend. Maybe they run out of the one thing the patron really wants to eat. Any of these things can make someone walk away and feel the business is poor. Businesses and people also change over time, I have reviews on Yelp for 5 stars from 5 years ago for places that have either gone downhill or simply don't match my current taste. People also tend to only review things they either really liked/loved or hated which results in most businesses (especially in SF) as being rated 3.5-4 stars making Yelp pretty useless without looking at the underlying reviews. A collaborative filtering engine could fix a lot of these problems. At the very least I'll match people who have my own biases and normalize people's ratings. Of course it still doesn't fix the temporal issues, but I don't think that's insurmountable.

cloudkj 15y ago
That's an interesting idea. Is the main motivation of your suggestion to find a way to enable you to better discover things you might like without relying purely on reviews and ratings? In general, I agree that ratings and reviews have inherent biases that are hard to smooth out. There are so many more dimensions to it, such as how much weight you apply to a review written by someone that is trying a certain type of food for the first time. Do you think that you would see better results if Yelp was able to use a collaborative filtering approach to somehow personalize their results such that they more closely match your tastes and preferences - based on other users' data?

int3rnaut 15y ago

Would anyone be interested in disrupting this space? My friend and I have been working on something we quite like that deals with subjectivity/ethics concerns, but it's a tough space to be in, and we'd love some help. If you want to get in touch, email us at questionasker[at]liveDotCA

droithomme 15y ago

The problem with the proposed solution of verifying ID is that people who leave honest bad reviews often enough get harassment sued by angry dishonest business owners, who it turns out are more likely to harass and sue companies than the honest companies are. The main problem here is that Yelp is a racket as well, this has been known for some time. Those using it, well caveat emptor. There are other review sites out there that are not as morally compromised as Yelp due to their business model being modeled on a mafia protection racket of "Pay us and we can help these bad reviews go away."

IanDrake 15y ago

Any review on Yelp that says "We" or mentions the name of the person that assisted them at company is most certainly fake. Example: "We were really worried about where to stay on Las Vegas, but then we called hotel X and Margo sounded so nice and welcoming on the phone that we just knew it was the right choice."

robatsu 15y ago

Identity verification and such are only part of the problem. Even if it is a bona fide person writing the review (and ignoring the problems w/Yelp filtering practices), a truly valid review only comes from one who the readers can be assured really was a customer of Royal Movers in addition to some of the other, lower identity bars. So how could that work?

pauljonas 15y ago

I stopped using Yelp after I noted that reviews I submitted were only visible to me, if I was logged into the site. That defeats the purpose of an online review site -- if the review is only visible to me, why am I posting to Yelp, and giving them valuable data they just will sit on, as opposed to logging in my own local host `reviews.txt`? I noticed also that a number of favorable reviews, seemingly written by authentic reviewers were also suppressed in like fashion. See Amazon for a site that gets posting reviews mostly correct.

garrett_smith 15y ago

Justin, as a business owner, you are not Yelps customer. Yelp customer, or at least the one they care about, is really the people who use the site.

darkblob32 15y ago

I use Yelp to get ideas about what places to eat, but recommendations and confirmations from friends is usually the final say. I really like Yelp, but things like this keep coming up and it's hard to trust that their heart is in the right place. Not to mention how slow they are at staying competitive with other services. Check-in feature seemed to take forever, and I still can't believe they haven't tried to implement customer appreciation cards a la Stampt.

JohnnyFlash 15y ago

This isn't really Yelp's fault. Sites like this always have fake reviews mingled in with those that are real. The solution? Assume people with a 4-5 star review are competent. GET MULTIPLE QUOTES. Doesn't matter if they have 5*, if you feel something is off with them they you avoid. You also will find out when a quote is too cheap/too expensive and reduce your risk to getting conned. Don't blame the internets or Yelp. Blame youself for not doing due diligence.

schiffern 15y ago

So, you want Angie's List. Is anyone surprised that a recommendation site that gets its money from the businesses is less than honest? Is anyone surprised that when they get popular they turn into a protection racket? Say it with me. "If you're not paying, you're the product." Or in this case, the club.

derwiki 15y ago

For the record: the sales organization at Yelp has a read-only copy of business data in Salesforce. Salespeople are not able to fiddle with reviews in the main database. They're two completely different systems. There's no possible way a salesperson can hide/show reviews to try and sell advertising. The salesperson can _claim_ to be able to do it, but they're lying.

ceejayoz 15y ago
Does Yelp policy forbid sales people from being friends with customer service / database / etc. folks?

exue 15y ago

Despite all its flaws, Yelp is the best solution I've had for finding good restaurants and other services. I love being able to read reviews by locals I might randomly see, browse through their profiles and see what else they like. In the vast majority of cases the crowdsourced reviews are awesome. Unfortunately their verification and filtering system is weak, and needs major improvement (an identity verification program would be awesome). Yelp is still a great service overall, and the fact that I use it over ZAGAT or anything else tells you a lot about how effective it is.

Sivart13 15y ago

What's most shocking at this point is that this story has been up for seven hours and Yelp hasn't bothered to fix the page in question, the wrongness of which could be observed by any human with eyes: http://www.yelp.com/biz/royal-transportation-moving-and-storage-valley-village http://www.yelp.com/biz/royal-transportation-moving-and-stor...

jv22222 15y ago
I couldn't agree more. It seems amazing that with thousands of eyeballs on this there has been no effort made to fix it.
dcpdx 15y ago
Wow, those 1-star reviews are pretty scathing too. I had no idea that negative reviews were filtered like that; that pretty much ruins my trust in the review component of the site. From now on, I'll be using Yelp for contact info. and price points...

rush-tea 15y ago

Like what everyone is saying, Yelp review system is very easy to game. Just create a one off profile, and hype the business. Yelp algorithm only allows good reviews getting posted while the bad ones are buried. Why? Because good reviews generate traffic to Yelp which in turn help Yelp getting money from advertiser. Five stars review system is flawed.

mapster 15y ago

I agree, though the example gripe is problematic as the moving industry is largely unregulated. It is full of outfits that are subcontracted by the big movers. Extortion is commonplace (moving truck is near your new home. they stop and call on phone asking for $2k more before they arrive and unload your property), and is considered a contract dispute and the police will not get involved, citing such.

Quizzy 15y ago

Let's talk about the mobile only world: would a native mobile app be self-verifying in that only a mobile phone could download and use the app - so every review requires a real smartphone to post the review. Although "reputation" companies could game this situation by buying 1000 smartphones, but then you would have 1000 smartphones with the same pattern of up-posting such that it would be easier for an algorithm to recognize these "planted" phones.