26 ms·

Bay Area woman is on a crusade to prove Yelp reviews can’t be trusted

531 points by gnicholas 3y ago

irskep 3y ago
Edit: I would have loved to add to the conversation, but people do love their pitchforks and I should be working instead of commenting on HN. :-) Admitting you worked for Yelp on HN: not even once
- BonoboIO 3y ago
  What can you say about these accusations brought up in this thread? https://news.ycombinator.com/item?id=36329248 https://news.ycombinator.com/item?id=36329248
- FireBeyond 3y ago
  > It's amazing to me how many people have conspiracy theories about Yelp reviews, when in reality it's just really hard to determine good vs bad actors in review systems and it's possible to do a bad job as an engineer. When cards are cloned, it's exceptionally easy for a bank to determine which merchant is compromised by looking for common denominators. Take three actors. What are the odds that two of them have reviewed one business, and only that business? Pretty high. All good. Same for all three. Now, what are the odds that two of them have only reviewed two businesses, and both businesses are the same, on both accounts? Still decent, sure. All three actors, same two businesses reviewed, and only them? Getting a little dicier. Two actors, seemingly independent, have each reviewed three businesses only. And they've reviewed the same three businesses, in the same order. And perhaps they each reviewed the first business, then two weeks later, the second business, and then the third. Odds? Getting pretty slim now. A third actor, doing the same? You're stretching the bounds of credulity for sincere reviews. This stuff is basic probability theory. Looking across reviews for sets and unions of reviews and reviewers, and increasing the number of reviews, but still seeing commonality between them points to SOME malfeasance worthy of review.
  - emodendroket 3y ago
    I can see some ways that might go awry (what if I like reviewing every new restaurant that opens?). But more importantly, bad actors will figure out and work around each heuristic like this you make.
    - FireBeyond 3y ago
      Of course they will, but that shouldn't negate the need to identify and shut down interactions which are quite definitively problematic.
      - emodendroket 3y ago
        Another way of saying what I’m trying to say is that at any given time, no matter how much they keep heaping on new heuristics, there will be some missed cases that should be obvious to a human observer.
  - mtlynch 3y ago
    This comment is fairly dismissive of GP's experience and intelligence. If you think you can solve a problem just by spitballing it, and GP's team at yelp couldn't figure it out for years, either you're much smarter than everyone on their team or you're underestimating the problem. I worked on an anti-fraud team in a similar role, and I agree with GP that it's a harder problem than people realize. It's easy to come up with a simple metric that would identify the fraud that you observe, but you don't know what false positives it would catch as collateral damage. Like this example that you find wildly implausible: >Two actors, seemingly independent, have each reviewed three businesses only. And they've reviewed the same three businesses, in the same order. And perhaps they each reviewed the first business, then two weeks later, the second business, and then the third. Odds? Getting pretty slim now. A third actor, doing the same? You're stretching the bounds of credulity for sincere reviews. I think it's highly likely that if you used this metric to find patterns, you'd find fraud, but you'd also find lots of legitimate reviews (e.g., friends who go to the same restaurants together, friends who recommend restaurants to each other).
- emodendroket 3y ago
  Yeah, the only way you could have it work perfectly would be actually manually reviewing each review. Which would make the business model not work.
- winrid 3y ago
  It doesn't matter what you did as an engineer, and in fact automation isnt really a big part of the discussion. If sales people etc have special access to add or remove reviews, which they do, or at a minimum leave negative reviews themselves when you don't pay, it is corrupt. It is well documented at this point that they do this.
  - emodendroket 3y ago
    That’s not what the article describes at all so I’m curious what documentation you are referring to.
    - winrid 3y ago
      There are so many people complaining about the sales people doing this stuff. https://youtu.be/C67Lh4LE5LY https://youtu.be/C67Lh4LE5LY
      - emodendroket 3y ago
        I would believe individual salespeople might do unscrupulous things like enlist their friends to leave bad reviews or make promises they can't keep about altering someone's score but that's a totally different accusation than saying Yelp as a company has a policy of doing this.
        winrid 3y ago
        I doubt they explicitly have such a policy. That'd be suicide.
fmajid 3y ago
There are three kinds of review site: 1. The ones unable to effectively police shills and spam. I'd put Google in that bucket. 2. Those that turn a wilful blind eye to review fraud, as long as they boost sales, like Amazon 3. Those like Yelp or Glassdoor whose entire business model is built around extortion and a protection racket ("nice business you have here, it would be a shame if a bad review happened to it. Unless you buy ads with us, of course").
reisstremblay 3y ago
[dead]
Aerbil313 3y ago
What’s the solution then, without the need to trust centralized companies? I can’t think anything besides system with a WoT (web-of-trust) architecture.
fennecfoxy 3y ago
So many people see the brewing storm of AI as dystopian, but this is one of the tasks that it could be quite good at. Humans are biased, deceptive creatures and whether intentional or not we can misrepresent ourselves, others and every...thing. AI would definitely help to drop a "badge of approval" on something in some cases. Reviews for restaurants may be harder, but just simple fact checking it's going to be so amazing to have a trusted AI that can say "yep, this is most likely true" with a click-through to references/sources/reasons why. Look at trending Twitter tags, I've been looking at #ukrainerussiawar etc a lot and sometimes like 90% of the tweets have _nothing_ to do with the topic, jsut shameless self-promotion spamming tags, or misinformation tweets from either Russia's information warfare team, or quite simply right wing morons parroting propaganda. Would be nice to make all of that go away...
georgeheney 3y ago
[dead]
gnicholas 3y ago
A bit of a side note, but this is super rare: > Meanwhile, Dean took SavantCare to small claims court over the letter and calls she later received, and won a $275 judgment in March 2018 for “negligent infliction of emotional distress.” NIED claims are nearly impossible to win. In law school we were taught to point them out on issue-spotting questions, but that they were basically unwinnable.
- dylan604 3y ago
  so you're claiming that since there was a win, that it was that egregious?
  - gnicholas 3y ago
    Not egregious, certainly for the couple hundred dollars recovered. I was mostly pointing it out (1) because any lawyers here would find it interesting that someone, somewhere, once won on an NIED claim, and (2) so that non-lawyers wouldn't get the sense that NIED claims are winnable, in general. They are often tacked on to other, more legit claims, but they are very rarely won.
    - dylan604 3y ago
      Egregious in the sense that it was so over the top obvious there was no other outcome but to award the win vs some minor squabble between two parties. Not that it was so egregious mountains of cash were expected.
  - anonymouskimmer 3y ago
    The article doesn't say whether Savant Care showed up. If they didn't show up it would have been a default judgement.
    - sidewndr46 3y ago
      Not sure of the jurisdiction issues where the judgement was issued but the last time I looked into it you can't meaningfully collect a judgement from small claims court in my area.
gnicholas 3y ago
> A Yelp post from December 2017 was the first crucial clue. “Melissa R.” wrote, “I was offered money to leave a positive review for Savant Care Clinic. Must be a shady company.” The user attached Craigslist screenshots of the apparent offer, which Dean took her own screenshot of just before Yelp removed the post for violating its terms of service. Why would Yelp forbid people from talking about this? While it is certainly possible to create fake email screenshots, this seems like a relatively small risk compared to the much more common risk that's being called out here. I guess they just don't want people to know how common this type of occurrence is, since it would undermine all trust in their platform.
- trifurcate 3y ago
  The risk (mostly) isn't fake email screenshots, the risk is that I create postings for (or even commit to paying for fake reviews for) my competitor, and tarnish their reputation. It's hard to establish provenance of such an arrangement.
  - gnicholas 3y ago
    I think we're thinking the same thing — that someone would (on their own or for money) write a review claiming that an innocent business had tried to pay for a good review, or take down a bad review. This could happen, but it's just as easy to pay someone to write a review saying the service/food/etc. was lousy. Why spend more time faking an email screenshot? But from Yelp's perspective, a paid-for review claiming bad food is just fine, because it only tarnishes the restaurant's reputation. On the other hand, a claim of fraudulent review manipulation hits both the business and Yelp's reputations.
    - yifanl 3y ago
      It'd be easy for a reader to ignore a review about the quality of food since that's subjective on the reviewer's personal tastes. A review claiming paid-for-fraud isn't so easy to wave off.
- spiznnx 3y ago
ryaniscool 3y ago
First and only Yelp review I ever left was three stars for an Indian restaurant in 2011. It was quickly hidden because "it didn't pass the bot filter" or something. I've never used it since as a reviewer or to find places. It lost all credibility on my first interaction with it.
- gnicholas 3y ago
  I probably would have reacted the same way. But at the same time, Yelp's algorithm probably treats accounts with a track record as more legit than ones that were just created and then wrote a review. I wonder if they ever would have unhid your review if you had kept reviewing, or even just using the app to search for places to go.
- wccrawford 3y ago
  My wife left a bad review for a restaurant and was stalked and harassed by the restaurant owner for years afterwards. Since she couldn't get Yelp to help with that, it makes every review suspect. It doesn't take much of that behavior to convince people to only leave good reviews, and then they're basically useless.
  - gnicholas 3y ago
    How did they know your real name? Because they had your credit card and remembered you from your visit? Whenever I have a bad review to leave, I always wait a couple weeks, to minimize the chance they'll remember me...
  - annoyingnoob 3y ago
    I had a restaurant owner out me by name in response to a Yelp review. I pointed out that her attitude toward customers like me tells the world everything they need to know. That place went out of business, guess it wasn't just me.
  - emodendroket 3y ago
    Yeah I had a similar experience with a plumbing business I chose not to hire after their rude and aggressive salesman came and insulted me in some misguided bid to get me to sign a contract.
majormajor 3y ago
Good for her. Yelp's business model seems to work best if they can pick winners, which is unfortunate. If I search for "dinner" on Yelp for "current location" and look at the map, it leaves out an astonishing number of places that I can see within a mile if I drive down the street. And if your business model is centered around hiding all but the winners - who might pay for placement + reviews both - then you have to try to keep people from talking about it, and the moral interests just get worse and worse. Google Maps isn't any better; OpenTable seems the "least bad" of options I've looked at recently, at least for fancier reservation-having places, in terms of actually showing my options. (It's a broader internet problem in general - "trending" algorithmic reinforcement stuff will push everyone towards the ONE TRUE BEST HOTTEST MOST POPULAR bbq/ramen/whatver place, when probably there's half a dozen other options within 10% of taste and quality, that some people very well could legitimately like better if they heard about them too.)
- floren 3y ago
  > It's a broader internet problem in general - "trending" algorithmic reinforcement stuff will push everyone towards the ONE TRUE BEST HOTTEST MOST POPULAR bbq/ramen/whatver place, when probably there's half a dozen other options within 10% of taste and quality, that some people very well could legitimately like better if they heard about them too. Instagram doesn't help -- gotta get a shot of me eating noodles at the hot new place! There's a new ramen shop in downtown San Mateo that has a 30-person line outside at all times... I can't imagine it's that much better than the half-dozen other ramen places within 3 blocks, but by god it's new, and anyway would people be lining up like this if it wasn't good? I better get in line too...
  - oefrha 3y ago
    > and anyway would people be lining up like this if it wasn't good? That’s why new restaurants hire people to line up outside.
  - sophacles 3y ago
    Back in my bar-tending days we always told the door staff (bouncers) to slow down when checking IDs early in the evening, just to get a line going. It works surprisingly well for drawing in more customers. Totally tangential but follow-ups on that psychology: * some people would walk in and see the still mostly-empty bar and get fed up that they waited in line for "nothing". This was surprisingly few people though. * some people would ask and buy just about any explanation - "oh that guy is new and still learning", "we had someone get in on a fake ID and got a fine the other day so the guys are just being careful" or whatever and the people would be glad they got in early to get a good spot. * sometimes we'd slow down the bar staff a bit too so that there was a line to buy drinks and people were happier that they had to wait in line again - confirming their bias that they must have gotten in at the start of the rush. * some people were wise to the scam and everyone who called us on it thought it was great. They often offered to have their buddies come through the line a few times in exchange for a free drink. * for some reason when you let your buddies, regulars and off-shift staff cut the line and get in quick, the rest of the folks seemed more willing to wait in the line. tl; dr - group psychology of humans is wierd.
whalesalad 3y ago
Serious question: is Yelp still relevant? I feel like outside of select metro areas it’s not useful at all with very dated content.
- heyheyhey 3y ago
  No review site depending on users is perfect but I've found it's the best option for a quick "what's a good restaurant for x cuisine" in America. I've found Google Maps or Tripadvisor to be too hit-or-miss especially when traveling abroad where Yelp isn't as prevalent.
- JumpCrisscross 3y ago
  At least in New York, it is. It’s less prone to the cost noise that infects the other rating services—where a 3 star could be due to bad food or someone being surprised by New York prices. (There is also a scourge of paid top ten articles that makes a simple search useless.)
- angry_moose 3y ago
  The Apple Maps integration seems to be most of what's propping them up these days. It's especially infuriating because you can only see about 2 reviews before you're prompted to download the Yelp app for "full access". I thought Apple was supposed to be working on their own review system or switching to a different service, but I haven't seen anything come of that.
  - schnable 3y ago
    Sometimes they show TripAdvisor reviews instead of Yelp.
    - angry_moose 3y ago
      Huh. Wonder if it's a regional thing? I don't recall ever seeing that and I just checked about 25 places in my city and none of them had TripAdvisor.
deleted 3y ago
[deleted]
exabrial 3y ago
I don’t use the reviews, just the listings. One of my favorite local restaurants purposefully asks for one star reviews on yelp, as they were listed there without their consent.
- gnicholas 3y ago
  Such a great idea. You won't get new customers via Yelp this way, but you will generate some word-of-mouth that could offset that impact. However, this strategy only works if a very small number of businesses use it. Once the novelty wears off, the word-of-mouth benefit vanishes.
mayormcmatt 3y ago
Sort of beside the point, but I wanted to say how much I love the subject's whole vibe (to say nothing of her excellent mission). Her home and attire give me strong nostalgic feelings of late-80s after-church events in the SF Bay Area.
1024core 3y ago
> Dean began documenting her findings, starting with serial offenders from the Facebook groups. In a list of 17 people who reviewed SavantCare on Yelp, she found what she called “a suspicious pattern”: 10 of them had reviewed the same dentist’s office in Los Angeles, nine had raved about the same California wedding DJ company and five claimed to have visited the same Mountain View tattoo parlor. Five of the accounts, she noticed, had even reviewed the same moving company in Florida. SMH... I mean, how difficult is it for Yelp to catch this?!? Hire a high school student with interest in data analysis, and you'll catch these idiots quickly.
- LeifCarrotson 3y ago
  > Hire a high school student with interest in data analysis, and you'll catch these idiots quickly. The calls are coming from inside the house! It's not in Yelp's interests to try to catch this. At least in the short term.
  - marcod 3y ago
    Only because they don't factor in what their users will do when they get wind of it... become ex-users.
  - kromem 3y ago
    Which turned out to be a terrible idea, because now with the importance of data integrity in training AI all these BS reviews everywhere mean Yelp, Amazon, etc are left with effectively worthless data that otherwise could have created very neat tools and offered a secondary revenue stream to just placement/ads.
- timmytokyo 3y ago
  It's amazing the lengths someone will go to in order to avoid doing something when the incentives run in the opposite direction.
- Solvency 3y ago
  "I mean, how difficult is it for Yelp to catch this?!?" The mistake you're making is assuming Yelp: * Wants to "fix" this problem. * Isn't actually complicit in this problem as a means to incentivize small business owners to sign up and pay for Yelp to help "manage their business profile".
dwater 3y ago
It sounds to me like Yelp is upset that other businesses are trying to do the same thing they do. My friend was GM of a new local restaurant here, and at first they were getting lots of 4 and 5 star reviews and almost none below that. But around the time they were contacted by a Yelp sales rep to start paying for promotion and advertising, the 4 and 5 star review stopped coming. My friend watched multiple people post reviews that never appeared. Then negative reviews started appearing from people that he knew never came in because he was working the host station all day. All this activity was reversed once they started paying for Yelp services. It's the kind of behavior that's impossible to prove for anyone outside of Yelp, and they of course claim they don't do it.
- nickstinemates 3y ago
  You'd think there would be some whistleblowers after all of these years of the same allegations. Where are they?
  - lacy_tinpot 3y ago
    secrets can't be kept in any large corporation therefore this is a conspiracy theory of course secrets can be kept in large organizations, but it's funny how often that get repeated.
    - emodendroket 3y ago
      I mean, this isn’t the CIA here. It’s a tech company people are probably filing in and out of in about two years. And the salesmen are all meant to be in on it?
      - brnaftr361 3y ago
        Yeah, if pay incentives are there, and we're talking sales... They're not necessarily people of the highest esteem. And if the hiring party puts a little pressure on the candidate, I could definitely imagine a tight ship. A scrupulous person wouldn't join the Gestapo, y'know?
deleted 3y ago
[deleted]
warrenwilliams 3y ago
[dead]
jroseattle 3y ago
The entire fake-review cottage industry harkens back to web search before Google was a thing. Search results were being gamed as implied relevance, ultimately making Yahoo and Excite and Alta Vista also-rans. Online reviews need their Google-search moment. Something that levels the playing field, establishes a mechanism for credibility, and doesn't require an extortion-based business model.
arbuge 3y ago
While on the subject, are there any review sites anywhere, not just for locations but also of products, that can be trusted? It seems to me that all of them can be and actively are gamed. Googling using the extension site:reddit.com caught on for a reason.
- chasebank 3y ago
  Not that I'm aware of. Reddit/HN has been compromised for years but I guess it's better than the alternatives. Whoever solves trustworthy, honest reviews will be a very, very rich person.
  - stevenwliao 3y ago
    We will never have completely trustworthy user reviews. Reddit/HN quality is probably about as good as it gets. Many small businesses get their viral loop started from glowing reviews from family and friends. That sort of growth hack will never disappear, and the subtle cases will be undetectable. I decline to write such reviews, but others won't.
    - fsckboy 3y ago
      > > Whoever solves trustworthy, honest reviews will be a very, very rich person. > We will never have completely trustworthy user reviews. just reading these to comments helped me figure out how to solve this: since apps have location tracking, for restaurants at least, your reviews should be weighted by how much time you spend close to a restaurant. If you claim to like a restaurant, but you don't eat there much, that means your reviews are not reliable. This unreliability would transfer to reviews of places that are not close to your home/work. bonus points to combine it with confirmed food expenditures on your credit card/applepay.
      - stevenwliao 3y ago
        This doesn't work. It's not that people are making up their restaurant visits. It's that they are going to the restaurant, they had a good time, but they didn't disclose that they are close to the owner, and maybe they overlooked a few things here and there that an impartial reviewer would note.
readthenotes1 3y ago
I was glad to see that the horoscope was only the 7th most popular page on SFGate today. I can only imagine that this means all these recent troubles I've been hearing about San Francisco have been resolved and people do not need the help from their horoscope anymore to know what to do to resolve their problems.
SoftTalker 3y ago
I assumed Yelp reviews can't be trusted from day one. It's obviously a huge target for gaming and scamming, pretty much any review site is. I include Angie, and similar sites in that as well.
TimPC 3y ago
I worked on a start-up that was basically a Yelp competitor with a less-bad business model. Unfortunately, all the incentives in the business are about boosting the performance of your paying customers relative to free businesses on the platform. While we never went as far as faking reviews or helping businesses eliminate bad reviews and we at least were clear to customers about our search lists being curated rather than complete, there were clear pressures towards all the things that lead to Yelp being the shit-show that it is.
notyourwork 3y ago
Yelp is a cesspool of garbage at this point. I ignore it whenever it’s in my search results after they started forcing me to their app.
emodendroket 3y ago
> Before Fake Review Watch and all her research, Dean used review sites often, she said. But now she tells everyone she knows to stick with good old word-of-mouth — especially with recent additional hits to the online system, like the surge of reviews spurred by COVID-19 policies and new reports of AI chatbot-written Amazon reviews. Kind of sick of reading this advice. If I knew someone I trusted to recommend me a contractor why would I be on Yelp looking in the first place? Hello.
- Solvency 3y ago
  This is where NextDoor comes in. I guarantee your entire town is likely already on there talking about contractors they've used. The app is slow and sluggish but it's legitimately where most homeowners and "regular townsie folk" are these days.
  - emodendroket 3y ago
    I am on some Facebook neighborhood groups which I think are similar enough to Nextdoor, but there are plenty of scammers about and also, it’s hard to trust people’s judgment given how many idiotic posts you see floating around.
deleted 3y ago
[deleted]
bilater 3y ago
Reminds me of this rule of thumb when navigating Yelp: https://twitter.com/fwong/status/1569736492247044103 https://twitter.com/fwong/status/1569736492247044103
ryandrake 3y ago
I hate to be dismissive, but this is kind of a "Water Is Wet" article. Does anyone really think online reviews are anything but totally gamed, bought, and paid for? It's kind of a Universal Law Of Spam: Any place on the Internet where users can add content and where content might be profitable to someone, will eventually end up full of spam and paid-for content.
- trabant00 3y ago
  Funny to find your post at the bottom of the thread. I tried to use a few apps for finding restaurants, compared their rating with the experience I had and concluded it's a waste of time and money. There's not even a need to know or understand anything about it. No insider info from Yelp, not experiencing it as a business owner. Just try and see for yourself.
- hwayne 3y ago
  Not everybody is online enough to know this!
- standardUser 3y ago
  Of course some reviews are real, and there's an art to deciphering the wheat from the chaff. The problem is that there's the constant pressure you describe to drown out legitimate reviews or otherwise game the system, so that 'art' becomes more and more tricky and less reliable. I gave up on reading the Yelp tea leaves a long time ago, and other prominent ones like Google Maps and Tripadvisor have been getting worse over time. I can still usually intuit my way through Amazon and and Steam reviews, if there's enough reviews and I take my time.
- Nextgrid 3y ago
  There's huge money to be made pretending spam isn't a real problem. It used to be that spam was universally despised, fought and even prosecuted in some jurisdictions. But it turns out that it's cheaper (and actually profitable) to join the spammers than to fight them, especially if you have an endless supply of marks that will guzzle the spam without second thoughts.
stainablesteel 3y ago
i never use yelp, i can typically tell what a restaraunt is like by the name and pictures on google, and whether they serve craft beer or brand names, i don't look at reviews or how much it costs
gaucheries 3y ago
> Yelp, founded in San Francisco in 2004, is deeply entrenched in American consumer habits is it really? couldn't tell you the last time i loaded yelp.com on purpose.
- NegativeK 3y ago
  People reading HN are not the average American.
- kelnos 3y ago
  Your experience may not be the general American experience. It is a bit ridiculous that I need to point this out.
  - gaucheries 3y ago
    you don’t need to do anything
xyst 3y ago
I am surprised a yelp/google alternative has not been created with a feature to establish provenance of a review. Perhaps something like a receipt which contains a unique link and that person can provide a review through the link. The review will be posted as "verified transaction" on the site and overall score of the restaurant is determined from there. "Anonymous" reviews can still be posted but won't be factored into the overall score and will be deprioritized by the system/algorithm. Of course this does not necessarily stop businesses from pumping out fake transaction tickets and offering users to write fake reviews. But at least a paper trail would be established; and if a historically 1-2 star business with verified transactions and 1-2 reviews per day suddenly sees a spike to 4-5 stars with 10+ reviews per day over the same period. Then it can be flagged as suspicious and team can review the paper trail.
- janalsncm 3y ago
  Additionally, once you start creating a paper trail it starts to look more and more like a concerted criminal fraud operation. Yours is a good idea.
- sensanaty 3y ago
  I actually worked for a startup that did this, albeit it wasn't public-facing, rather we'd incentivize people to leave reviews in exchange for a coupon code or whatever Turns out a lot of restaurant owners would prefer to straight up own the reviews/influence it themselves rather than get legitimate, anonymized ones. There's just not that much of a reason to genuinely improve as a service when there's so much cutthroat competition
  - moneywoes 3y ago
    Did they pivot?
    - sensanaty 3y ago
      Not sure, I left quite a while ago, haven't really kept up with them. I doubt it though, the CEO didn't care too much about infinite growth and just wanted to build a helpful app for the handful of people who would appreciate a service like that
Tycho 3y ago
Reviews are just text associated with a date and a place. Seems like someone should be able to build a simple app or website that the public can use and trust for reviews, that doesn't cost much to run. We don't need huge businesses liek TripAdvisor (which mysteriously doesn't let you sort results any more). There's a few challenges. Fake reviews is one. Also the self-selection bias - people generally only feel compelled to leave a review after a bad experience. Or, being supportive, will be overly generous with the rating. Perhaps if the review site could somehow prompt recent patrons of a set of establishments to not rate but rank the establishments, then some algorithm could convert that into a global score/ranking. You, an otherwise disinterested citizen, receive an email or text asking you to rank restaurants A, B, C which you have recently visited. You are incentivised to respond by a prize draw or something. Would this work? The hard part I suppose is getting the contact details of the potential reviewers.
rfwhyte 3y ago
Online reviews have become one of the dankest cesspits of misinformation and deceit on the entire internet. Amazon's reviews are so thoroughly gamed as to be functionally useless and Amazon knows this and tacitly supports it through their inaction. Yelp and Google Places review are being gamed en-masse by bad actors and scammers and have become so distorted that by relying upon them you're now more likely have a bad experience than a good one. Review websites are stuffed full of advertorials and universally have such super shady advertising / sponsorship policies it's functionally impossible to know whether the review you're reading was paid for or not, and all the big tech sites are so dependent on advertising revenue, they almost never post critical review for fear of upsetting their advertisers. It's a unspoken rule in the industry these days that negative reviews are simply not published. Not to mention damn near every single product review video on YouTube, IG Reels, Tiktok, or wherever else now is either straight up sponsored content where the "Reviewer" is receiving monetary compensation for their so called "Review" or at minimum is receiving the product for free, which creates yet another perverse incentive, as anyone who posts a negative review will simply stop getting free products to review in the future. There are literally dozens of 100+ person companies in China and India who's entire business model is faking online reviews. The entire industry is broken, and the average consumer can't find literally a single source for unbiased, trust-worthy reviews of anything anymore, which I for one believe is the system working exactly as designed and intended. Never forget the rich will sacrifice literally anything that's generally considered to be part of the "Public good" if it means they'll get rich faster, which is exactly what's happening here.
datadeft 3y ago
It is such an easy thing to solve with a blockchain. We do not need a 3rd part like Yelp. Just a chain with 2 actors, restaurants and reviewers.
- 2023throwawayy 3y ago
  And that solves fake reviews... how?
  - datadeft 3y ago
    I am pretty sure that we can use clustering for that. As they said there was a clear pattern the restaurant recognized. If they had all the data it would be easy. The reviewer could also ask for a token from the restaurant before submitting the review or a signature, there are plenty of options.
    - 2023throwawayy 3y ago
      And what exactly does a blockchain provide here that a plain ole' database does not?
      - datadeft 3y ago
        Tamperproof history
hospitalJail 3y ago
Man I forgot about Yelp until I read this. Apparently its because its packaged inside Apple's "Walled Garden" Maps. Seems like they got a weed in there that needs to be picked.
acumenical 3y ago
In the future, reviews will be automated, thus immunizing them from unsavory practices such as paid reviews. Your GPS-enabled wearable compute device will determine which establishment you patronized and for how long, and biometric data will determine whether you enjoyed your time there. These data will be "anonymized" and aggregated. "90% of [brand] Watch wearers like You experienced heightened happiness scores at this restaurant!" Trust in reviews will have been regained and the world will be a better place.
- morkalork 3y ago
  Hello, [restaurant name], this is Alice from [brand] calling. Would you be interested in our premium marketing package? Features include enhanced filtering of "anomalous" (negative) happiness-score(tm) readings. The subscription costs only 1,337 credits/month.
  - acumenical 3y ago
    Good point if the app was independent like Yelp. I envisioned this as a first party product from a more powerful company, which makes the Watch as well, that could then directly integrate the consumer/review data into their Maps app on their Phones and other devices. Alice from [brand] would then have to find a way to get customers to take off their watches or somehow artificially inflate their happiness scores. Or, funnel naturally happier people to their establishment, similar to lead sourcing.
kingofkyiv 3y ago
[dead]
cromulent 3y ago
This article about TripAdvisor immediately comes to mind. https://www.vice.com/en/article/434gqw/i-made-my-shed-the-top-rated-restaurant-on-tripadvisor https://www.vice.com/en/article/434gqw/i-made-my-shed-the-to...
kristopolous 3y ago
It's a protection racket. Total scam. If you have 20 minutes I did a show about it 2 years ago https://youtube.com/watch?v=e9rmh8O_ksM https://youtube.com/watch?v=e9rmh8O_ksM. I guess you can 2.0 speed that and get it done in 10.
- theshrike79 3y ago
  https://www.youtube.com/watch?v=dechvhb0Meo https://www.youtube.com/watch?v=dechvhb0Meo
  - kristopolous 3y ago
    The original model was as a long-form instagram live stream which I understand nobody did but I was being experimental. Basically a single shot, single take show on alternative platforms. The portrait format was to accommodate that media expectation. I made dozens of them ... they failed to gain traction. A few are over 100 views a couple years later (like this: https://www.youtube.com/watch?v=3byGVjPq468 https://www.youtube.com/watch?v=3byGVjPq468) but I have still failed to crack the world of social media. It's not zero ... that's nice I guess. I'm planning to do a reboot sometime soon. I took a couple weeklong trips around the country to some research libraries in the past year or so, but I'd like to execute differently to try to be more effective. This was my third show run attempt. I think I may need a new foil for my host. The guy I did the shows with is a longtime TV comedy writer (https://www.imdb.com/name/nm1104065/ https://www.imdb.com/name/nm1104065/) but he's not exactly a second city graduate.
balderdash 3y ago
I think I’m more surprised that anyone places any faith in a yelp review
louzell 3y ago
Go Kay Dean! Yelp should hire her and give her a team. It's a win-win. She is clearly passionate about the problem, and effective. Yelp gains added trust of their bread and butter, the review system.
hexo 3y ago
Who does really use Yelp? I've seen no one. Is it just US-only thing, or?
bborud 3y ago
I deleted the Yelp app after a few too many bad restaurant experiences AND the fact that they removed reviews I wrote that were factual and helpful.
Graziano_M 3y ago
Seeing the 'Moving company from Florida' triggered me. I was scammed by a moving company from Florida in 2020, and I did a deep dive to look into them and I found huge networks of moving companies, scammers, & reviewers, all in cahoots. I actually managed to get one company's fake reviews removed from google maps, bringing them from a 4.8+ to a 1.x overnight. Within a few months they were back to > 4, but the owners had sold the company and started a new one. It's insane how deep this goes. At best it's done by companies that want to cheat the system by getting an unfairly good review, but at its worst it's used to trick users into a false sense of confidence so that they can be scammed.
- kansface 3y ago
  Yeah, I feel like Yelp et al chiefly facilitate fraud at this point.
  - datavirtue 3y ago
    Any silicon valley middle-man venture is a fraud/scam engine.
- m463 3y ago
  Same with tow trucks. Trying to find a tow truck on your own is a HUGE mess. Finding a tow company in the area invariably leads to some sort of "virtual tow truck call center" that takes a bunch of money and subcontracts out to actual tow truck drivers. My solution was to pay for the roadside assistance via my car insurance and call them to sort things out. Now emergency plumbers - same mess, but I haven't figured out a solution for that.
  - bombcar 3y ago
    I just give up and use commercial services. They’re usually willing to do residential or recommend someone who can. For tow trucks just search for ones that can do a semi. They’ll know who to call.
  - jstarfish 3y ago
fernly 3y ago
Posted article couldn't trouble to link Ms. Dean's actual channel. Here it is: https://www.youtube.com/@fakereviewwatch https://www.youtube.com/@fakereviewwatch It's pretty good stuff. She builds a devastating case very methodically, showing how the identical glowing review text is posted to yelp, to FB, and to Google, by different users -- and then traces other reviews posted by those same user ids showing how they are plagiarizing Tripadvisor posts.
rhyme-boss 3y ago
How do you aggregate people's preferences into scores and not have it get gamed?
g42gregory 3y ago
Who thinks that these reviews can be trusted? When I read Yelp reviews, I look for the absence of negative, not presence of positive.
- datavirtue 3y ago
  Perhaps you could just scrape yelp and use machine learning to show the patterns?
- bombcar 3y ago
  I just read a few of the middle/bad reviews and see if they seem like reasonable complaints.
- NSMutableSet 3y ago
  How I use Yelp reviews: 5-star, less than 100 reviews: probably paid for reviews. I've never been to a "bad" place in this category, but I usually avoid it unless it's the most appealing option, so my sample size is small 4+ star, 1000+ reviews: definitely not bad. i would rate every place ive been to in this category as good or excellent, although some are overpriced. all of my favorite places in the bay area fall between 4.0 and 4.5 3.5 star, 100+ reviews: possibly good, probably not bad. ill check reviews, if people are primarily complaining about service and not food, it's probably good. i never pick 3.5 places if it's somewhere i've never been before, but i won't veto one if someone else suggests it 3.5 star, less than 100 reviews: i avoid these completely. i've still never been to one less than 3.5 stars, any number reviews: i also avoid these completely
klabb3 3y ago
If you think it’s bad now (spoiler: it is worse than you think), this is merely the calm before the storm. We haven’t yet seen the massive saturation of LLM generated content on everything from reviews to job applications to anything else you can possibly think of. The friction of sentiment-hacking operations will be dramatically reduced and we don’t have any effective mitigations. To make things worse, the only ones who can integrate the solutions (the platform owners) are not incentivized to take it very seriously. Fun times ahead!
emeril 3y ago
I don't work at Yelp though I'm pretty sure that the reviews which stay on the website for any length of time are those posted by users who actively use the website to search for businesses (i.e., tracked via their cookie) i.e., while there certainly can be paid/manipulated reviews, the barrier to doing so is higher here than elsewhere (e.g., google reviews, tripadvisor, etc.) and there's enough people who are happy to use other less controlled sites, I haven't witnessed any truly widespread fraudulent reviews like I have elsewhere) Generally speaking, I'd say Yelp reviews are, generally speaking, comparatively the most reliable I can't speak to how they sort/present search results - that does seem potentially shady
11thEarlOfMar 3y ago
What astonishes me about both Yelp! and Trip Advisor is their complete inability to grow their share price, even with near-monopolies in Restaurants (Yelp!) and Hotels (TripAdvisor). Yelp! catalogs 205 Million reviews last year. 30 Million for Trip Advisor. But even with such dominance, in the last 10 years, Yelp! is up about 38% and Trip Advisor is down 42%. What is it about their businesses that makes profitability growth so elusive?
Seattle3503 3y ago
> She also reached out to Sasha Ganji, the lawyer whose signature was appended to the letter she’d received from the law firm appearing to represent SavantCare. Ganji never touched the document, he told Dean; apparently someone had used his name and signature without his knowledge. That is a serious thing to do. This is criminal fraud right?
kelnos 3y ago
The article links to one of her videos where she talks about the owner of a veterinary practice who buys reviews. I brought one of our cats there twice last year for after-hours urgent care, and my experience was -- fortunately -- positive both times. Friends bring their dog there for primary care regularly. I'm just really wondering why the owner feels the need to do this. Is it a situation where you just have to do it, else you'll get buried under your competitors? Should I expect that our primary vet does this too, and that the other vet practices in our area also do it? Shady either way, but could it be a damned if you do, damned if you don't type thing? Certainly there are business sectors where there's so much competition that some of them will resort to shady tactics like this to gain customers. But if anything, there's a shortage of vet practices in San Francisco (and the entire bay area, really). Gaining customers here feels like it's more about having available appointments in a reasonable time frame than anything else.
- pcurve 3y ago
  "I'm just really wondering why the owner feels the need to do this." Everyone knows people who had negative reviewers are more likely to leave reviews, but what people don't often understand is, percentage of people who actually leave ANY review is still exceedingly small. I've heard as long as 1 in a few hundred or even a thousand. So getting that first few positive ones is critical, to kick off the virtuous cycle of customer acquisition. You'd better deliver though, once customers start coming in.
- toxicFork 3y ago
  It could just be the recommended marketing practice by the local small business school. Right after "Rent a week on the ad spot by the bus station" could be "buy a few fake reviews on yelp".
janandonly 3y ago
Would it be possible to build a better YELP, maybe based on open standards like Nostr? Is it possible to vote anonymously without being able to fake those votes/reviews?
jadamczyk 3y ago
When I was on a trip in the US for vacation, I found out that most of the review services can’t be trusted. Despite good or bad reviews it was a complete lottery. Meanwhile in Poland when a restaurant has a good review online you can actually expect reasonable service and food quality.

531 points by gnicholas 3y ago

Edit: I would have loved to add to the conversation, but people do love their pitchforks and I should be working instead of commenting on HN. :-) Admitting you worked for Yelp on HN: not even once

BonoboIO 3y ago
What can you say about these accusations brought up in this thread? https://news.ycombinator.com/item?id=36329248 https://news.ycombinator.com/item?id=36329248
FireBeyond 3y ago
> It's amazing to me how many people have conspiracy theories about Yelp reviews, when in reality it's just really hard to determine good vs bad actors in review systems and it's possible to do a bad job as an engineer. When cards are cloned, it's exceptionally easy for a bank to determine which merchant is compromised by looking for common denominators. Take three actors. What are the odds that two of them have reviewed one business, and only that business? Pretty high. All good. Same for all three. Now, what are the odds that two of them have only reviewed two businesses, and both businesses are the same, on both accounts? Still decent, sure. All three actors, same two businesses reviewed, and only them? Getting a little dicier. Two actors, seemingly independent, have each reviewed three businesses only. And they've reviewed the same three businesses, in the same order. And perhaps they each reviewed the first business, then two weeks later, the second business, and then the third. Odds? Getting pretty slim now. A third actor, doing the same? You're stretching the bounds of credulity for sincere reviews. This stuff is basic probability theory. Looking across reviews for sets and unions of reviews and reviewers, and increasing the number of reviews, but still seeing commonality between them points to SOME malfeasance worthy of review.
- emodendroket 3y ago
  I can see some ways that might go awry (what if I like reviewing every new restaurant that opens?). But more importantly, bad actors will figure out and work around each heuristic like this you make.
  - FireBeyond 3y ago
    Of course they will, but that shouldn't negate the need to identify and shut down interactions which are quite definitively problematic.
    - emodendroket 3y ago
      Another way of saying what I’m trying to say is that at any given time, no matter how much they keep heaping on new heuristics, there will be some missed cases that should be obvious to a human observer.
- mtlynch 3y ago
  This comment is fairly dismissive of GP's experience and intelligence. If you think you can solve a problem just by spitballing it, and GP's team at yelp couldn't figure it out for years, either you're much smarter than everyone on their team or you're underestimating the problem. I worked on an anti-fraud team in a similar role, and I agree with GP that it's a harder problem than people realize. It's easy to come up with a simple metric that would identify the fraud that you observe, but you don't know what false positives it would catch as collateral damage. Like this example that you find wildly implausible: >Two actors, seemingly independent, have each reviewed three businesses only. And they've reviewed the same three businesses, in the same order. And perhaps they each reviewed the first business, then two weeks later, the second business, and then the third. Odds? Getting pretty slim now. A third actor, doing the same? You're stretching the bounds of credulity for sincere reviews. I think it's highly likely that if you used this metric to find patterns, you'd find fraud, but you'd also find lots of legitimate reviews (e.g., friends who go to the same restaurants together, friends who recommend restaurants to each other).
emodendroket 3y ago
Yeah, the only way you could have it work perfectly would be actually manually reviewing each review. Which would make the business model not work.
winrid 3y ago
It doesn't matter what you did as an engineer, and in fact automation isnt really a big part of the discussion. If sales people etc have special access to add or remove reviews, which they do, or at a minimum leave negative reviews themselves when you don't pay, it is corrupt. It is well documented at this point that they do this.
- emodendroket 3y ago
  That’s not what the article describes at all so I’m curious what documentation you are referring to.
  - winrid 3y ago
    There are so many people complaining about the sales people doing this stuff. https://youtu.be/C67Lh4LE5LY https://youtu.be/C67Lh4LE5LY
    - emodendroket 3y ago
      I would believe individual salespeople might do unscrupulous things like enlist their friends to leave bad reviews or make promises they can't keep about altering someone's score but that's a totally different accusation than saying Yelp as a company has a policy of doing this.
      - winrid 3y ago
        I doubt they explicitly have such a policy. That'd be suicide.

fmajid 3y ago

There are three kinds of review site: 1. The ones unable to effectively police shills and spam. I'd put Google in that bucket. 2. Those that turn a wilful blind eye to review fraud, as long as they boost sales, like Amazon 3. Those like Yelp or Glassdoor whose entire business model is built around extortion and a protection racket ("nice business you have here, it would be a shame if a bad review happened to it. Unless you buy ads with us, of course").

reisstremblay 3y ago

[dead]

Aerbil313 3y ago

What’s the solution then, without the need to trust centralized companies? I can’t think anything besides system with a WoT (web-of-trust) architecture.

fennecfoxy 3y ago

So many people see the brewing storm of AI as dystopian, but this is one of the tasks that it could be quite good at. Humans are biased, deceptive creatures and whether intentional or not we can misrepresent ourselves, others and every...thing. AI would definitely help to drop a "badge of approval" on something in some cases. Reviews for restaurants may be harder, but just simple fact checking it's going to be so amazing to have a trusted AI that can say "yep, this is most likely true" with a click-through to references/sources/reasons why. Look at trending Twitter tags, I've been looking at #ukrainerussiawar etc a lot and sometimes like 90% of the tweets have _nothing_ to do with the topic, jsut shameless self-promotion spamming tags, or misinformation tweets from either Russia's information warfare team, or quite simply right wing morons parroting propaganda. Would be nice to make all of that go away...

georgeheney 3y ago

[dead]

gnicholas 3y ago

A bit of a side note, but this is super rare: > Meanwhile, Dean took SavantCare to small claims court over the letter and calls she later received, and won a $275 judgment in March 2018 for “negligent infliction of emotional distress.” NIED claims are nearly impossible to win. In law school we were taught to point them out on issue-spotting questions, but that they were basically unwinnable.

dylan604 3y ago
so you're claiming that since there was a win, that it was that egregious?
- gnicholas 3y ago
  Not egregious, certainly for the couple hundred dollars recovered. I was mostly pointing it out (1) because any lawyers here would find it interesting that someone, somewhere, once won on an NIED claim, and (2) so that non-lawyers wouldn't get the sense that NIED claims are winnable, in general. They are often tacked on to other, more legit claims, but they are very rarely won.
  - dylan604 3y ago
    Egregious in the sense that it was so over the top obvious there was no other outcome but to award the win vs some minor squabble between two parties. Not that it was so egregious mountains of cash were expected.
- anonymouskimmer 3y ago
  The article doesn't say whether Savant Care showed up. If they didn't show up it would have been a default judgement.
  - sidewndr46 3y ago
    Not sure of the jurisdiction issues where the judgement was issued but the last time I looked into it you can't meaningfully collect a judgement from small claims court in my area.

gnicholas 3y ago

> A Yelp post from December 2017 was the first crucial clue. “Melissa R.” wrote, “I was offered money to leave a positive review for Savant Care Clinic. Must be a shady company.” The user attached Craigslist screenshots of the apparent offer, which Dean took her own screenshot of just before Yelp removed the post for violating its terms of service. Why would Yelp forbid people from talking about this? While it is certainly possible to create fake email screenshots, this seems like a relatively small risk compared to the much more common risk that's being called out here. I guess they just don't want people to know how common this type of occurrence is, since it would undermine all trust in their platform.

trifurcate 3y ago
The risk (mostly) isn't fake email screenshots, the risk is that I create postings for (or even commit to paying for fake reviews for) my competitor, and tarnish their reputation. It's hard to establish provenance of such an arrangement.
- gnicholas 3y ago
  I think we're thinking the same thing — that someone would (on their own or for money) write a review claiming that an innocent business had tried to pay for a good review, or take down a bad review. This could happen, but it's just as easy to pay someone to write a review saying the service/food/etc. was lousy. Why spend more time faking an email screenshot? But from Yelp's perspective, a paid-for review claiming bad food is just fine, because it only tarnishes the restaurant's reputation. On the other hand, a claim of fraudulent review manipulation hits both the business and Yelp's reputations.
  - yifanl 3y ago
    It'd be easy for a reader to ignore a review about the quality of food since that's subjective on the reviewer's personal tastes. A review claiming paid-for-fraud isn't so easy to wave off.
spiznnx 3y ago

ryaniscool 3y ago

First and only Yelp review I ever left was three stars for an Indian restaurant in 2011. It was quickly hidden because "it didn't pass the bot filter" or something. I've never used it since as a reviewer or to find places. It lost all credibility on my first interaction with it.

gnicholas 3y ago
I probably would have reacted the same way. But at the same time, Yelp's algorithm probably treats accounts with a track record as more legit than ones that were just created and then wrote a review. I wonder if they ever would have unhid your review if you had kept reviewing, or even just using the app to search for places to go.
wccrawford 3y ago
My wife left a bad review for a restaurant and was stalked and harassed by the restaurant owner for years afterwards. Since she couldn't get Yelp to help with that, it makes every review suspect. It doesn't take much of that behavior to convince people to only leave good reviews, and then they're basically useless.
- gnicholas 3y ago
  How did they know your real name? Because they had your credit card and remembered you from your visit? Whenever I have a bad review to leave, I always wait a couple weeks, to minimize the chance they'll remember me...
- annoyingnoob 3y ago
  I had a restaurant owner out me by name in response to a Yelp review. I pointed out that her attitude toward customers like me tells the world everything they need to know. That place went out of business, guess it wasn't just me.
- emodendroket 3y ago
  Yeah I had a similar experience with a plumbing business I chose not to hire after their rude and aggressive salesman came and insulted me in some misguided bid to get me to sign a contract.

majormajor 3y ago

Good for her. Yelp's business model seems to work best if they can pick winners, which is unfortunate. If I search for "dinner" on Yelp for "current location" and look at the map, it leaves out an astonishing number of places that I can see within a mile if I drive down the street. And if your business model is centered around hiding all but the winners - who might pay for placement + reviews both - then you have to try to keep people from talking about it, and the moral interests just get worse and worse. Google Maps isn't any better; OpenTable seems the "least bad" of options I've looked at recently, at least for fancier reservation-having places, in terms of actually showing my options. (It's a broader internet problem in general - "trending" algorithmic reinforcement stuff will push everyone towards the ONE TRUE BEST HOTTEST MOST POPULAR bbq/ramen/whatver place, when probably there's half a dozen other options within 10% of taste and quality, that some people very well could legitimately like better if they heard about them too.)

floren 3y ago
> It's a broader internet problem in general - "trending" algorithmic reinforcement stuff will push everyone towards the ONE TRUE BEST HOTTEST MOST POPULAR bbq/ramen/whatver place, when probably there's half a dozen other options within 10% of taste and quality, that some people very well could legitimately like better if they heard about them too. Instagram doesn't help -- gotta get a shot of me eating noodles at the hot new place! There's a new ramen shop in downtown San Mateo that has a 30-person line outside at all times... I can't imagine it's that much better than the half-dozen other ramen places within 3 blocks, but by god it's new, and anyway would people be lining up like this if it wasn't good? I better get in line too...
- oefrha 3y ago
  > and anyway would people be lining up like this if it wasn't good? That’s why new restaurants hire people to line up outside.
- sophacles 3y ago
  Back in my bar-tending days we always told the door staff (bouncers) to slow down when checking IDs early in the evening, just to get a line going. It works surprisingly well for drawing in more customers. Totally tangential but follow-ups on that psychology: * some people would walk in and see the still mostly-empty bar and get fed up that they waited in line for "nothing". This was surprisingly few people though. * some people would ask and buy just about any explanation - "oh that guy is new and still learning", "we had someone get in on a fake ID and got a fine the other day so the guys are just being careful" or whatever and the people would be glad they got in early to get a good spot. * sometimes we'd slow down the bar staff a bit too so that there was a line to buy drinks and people were happier that they had to wait in line again - confirming their bias that they must have gotten in at the start of the rush. * some people were wise to the scam and everyone who called us on it thought it was great. They often offered to have their buddies come through the line a few times in exchange for a free drink. * for some reason when you let your buddies, regulars and off-shift staff cut the line and get in quick, the rest of the folks seemed more willing to wait in the line. tl; dr - group psychology of humans is wierd.

whalesalad 3y ago

Serious question: is Yelp still relevant? I feel like outside of select metro areas it’s not useful at all with very dated content.

heyheyhey 3y ago
No review site depending on users is perfect but I've found it's the best option for a quick "what's a good restaurant for x cuisine" in America. I've found Google Maps or Tripadvisor to be too hit-or-miss especially when traveling abroad where Yelp isn't as prevalent.
JumpCrisscross 3y ago
At least in New York, it is. It’s less prone to the cost noise that infects the other rating services—where a 3 star could be due to bad food or someone being surprised by New York prices. (There is also a scourge of paid top ten articles that makes a simple search useless.)
angry_moose 3y ago
The Apple Maps integration seems to be most of what's propping them up these days. It's especially infuriating because you can only see about 2 reviews before you're prompted to download the Yelp app for "full access". I thought Apple was supposed to be working on their own review system or switching to a different service, but I haven't seen anything come of that.
- schnable 3y ago
  Sometimes they show TripAdvisor reviews instead of Yelp.
  - angry_moose 3y ago
    Huh. Wonder if it's a regional thing? I don't recall ever seeing that and I just checked about 25 places in my city and none of them had TripAdvisor.

deleted 3y ago

[deleted]

exabrial 3y ago

I don’t use the reviews, just the listings. One of my favorite local restaurants purposefully asks for one star reviews on yelp, as they were listed there without their consent.

gnicholas 3y ago
Such a great idea. You won't get new customers via Yelp this way, but you will generate some word-of-mouth that could offset that impact. However, this strategy only works if a very small number of businesses use it. Once the novelty wears off, the word-of-mouth benefit vanishes.

mayormcmatt 3y ago

Sort of beside the point, but I wanted to say how much I love the subject's whole vibe (to say nothing of her excellent mission). Her home and attire give me strong nostalgic feelings of late-80s after-church events in the SF Bay Area.

1024core 3y ago

> Dean began documenting her findings, starting with serial offenders from the Facebook groups. In a list of 17 people who reviewed SavantCare on Yelp, she found what she called “a suspicious pattern”: 10 of them had reviewed the same dentist’s office in Los Angeles, nine had raved about the same California wedding DJ company and five claimed to have visited the same Mountain View tattoo parlor. Five of the accounts, she noticed, had even reviewed the same moving company in Florida. SMH... I mean, how difficult is it for Yelp to catch this?!? Hire a high school student with interest in data analysis, and you'll catch these idiots quickly.

LeifCarrotson 3y ago
> Hire a high school student with interest in data analysis, and you'll catch these idiots quickly. The calls are coming from inside the house! It's not in Yelp's interests to try to catch this. At least in the short term.
- marcod 3y ago
  Only because they don't factor in what their users will do when they get wind of it... become ex-users.
- kromem 3y ago
  Which turned out to be a terrible idea, because now with the importance of data integrity in training AI all these BS reviews everywhere mean Yelp, Amazon, etc are left with effectively worthless data that otherwise could have created very neat tools and offered a secondary revenue stream to just placement/ads.
timmytokyo 3y ago
It's amazing the lengths someone will go to in order to avoid doing something when the incentives run in the opposite direction.
Solvency 3y ago
"I mean, how difficult is it for Yelp to catch this?!?" The mistake you're making is assuming Yelp: * Wants to "fix" this problem. * Isn't actually complicit in this problem as a means to incentivize small business owners to sign up and pay for Yelp to help "manage their business profile".

dwater 3y ago

It sounds to me like Yelp is upset that other businesses are trying to do the same thing they do. My friend was GM of a new local restaurant here, and at first they were getting lots of 4 and 5 star reviews and almost none below that. But around the time they were contacted by a Yelp sales rep to start paying for promotion and advertising, the 4 and 5 star review stopped coming. My friend watched multiple people post reviews that never appeared. Then negative reviews started appearing from people that he knew never came in because he was working the host station all day. All this activity was reversed once they started paying for Yelp services. It's the kind of behavior that's impossible to prove for anyone outside of Yelp, and they of course claim they don't do it.

nickstinemates 3y ago
You'd think there would be some whistleblowers after all of these years of the same allegations. Where are they?
- lacy_tinpot 3y ago
  secrets can't be kept in any large corporation therefore this is a conspiracy theory of course secrets can be kept in large organizations, but it's funny how often that get repeated.
  - emodendroket 3y ago
    I mean, this isn’t the CIA here. It’s a tech company people are probably filing in and out of in about two years. And the salesmen are all meant to be in on it?
    - brnaftr361 3y ago
      Yeah, if pay incentives are there, and we're talking sales... They're not necessarily people of the highest esteem. And if the hiring party puts a little pressure on the candidate, I could definitely imagine a tight ship. A scrupulous person wouldn't join the Gestapo, y'know?

deleted 3y ago

[deleted]

warrenwilliams 3y ago

[dead]

jroseattle 3y ago

The entire fake-review cottage industry harkens back to web search before Google was a thing. Search results were being gamed as implied relevance, ultimately making Yahoo and Excite and Alta Vista also-rans. Online reviews need their Google-search moment. Something that levels the playing field, establishes a mechanism for credibility, and doesn't require an extortion-based business model.

arbuge 3y ago

While on the subject, are there any review sites anywhere, not just for locations but also of products, that can be trusted? It seems to me that all of them can be and actively are gamed. Googling using the extension site:reddit.com caught on for a reason.

chasebank 3y ago
Not that I'm aware of. Reddit/HN has been compromised for years but I guess it's better than the alternatives. Whoever solves trustworthy, honest reviews will be a very, very rich person.
- stevenwliao 3y ago
  We will never have completely trustworthy user reviews. Reddit/HN quality is probably about as good as it gets. Many small businesses get their viral loop started from glowing reviews from family and friends. That sort of growth hack will never disappear, and the subtle cases will be undetectable. I decline to write such reviews, but others won't.
  - fsckboy 3y ago
    > > Whoever solves trustworthy, honest reviews will be a very, very rich person. > We will never have completely trustworthy user reviews. just reading these to comments helped me figure out how to solve this: since apps have location tracking, for restaurants at least, your reviews should be weighted by how much time you spend close to a restaurant. If you claim to like a restaurant, but you don't eat there much, that means your reviews are not reliable. This unreliability would transfer to reviews of places that are not close to your home/work. bonus points to combine it with confirmed food expenditures on your credit card/applepay.
    - stevenwliao 3y ago
      This doesn't work. It's not that people are making up their restaurant visits. It's that they are going to the restaurant, they had a good time, but they didn't disclose that they are close to the owner, and maybe they overlooked a few things here and there that an impartial reviewer would note.

readthenotes1 3y ago

I was glad to see that the horoscope was only the 7th most popular page on SFGate today. I can only imagine that this means all these recent troubles I've been hearing about San Francisco have been resolved and people do not need the help from their horoscope anymore to know what to do to resolve their problems.

SoftTalker 3y ago

I assumed Yelp reviews can't be trusted from day one. It's obviously a huge target for gaming and scamming, pretty much any review site is. I include Angie, and similar sites in that as well.

TimPC 3y ago

I worked on a start-up that was basically a Yelp competitor with a less-bad business model. Unfortunately, all the incentives in the business are about boosting the performance of your paying customers relative to free businesses on the platform. While we never went as far as faking reviews or helping businesses eliminate bad reviews and we at least were clear to customers about our search lists being curated rather than complete, there were clear pressures towards all the things that lead to Yelp being the shit-show that it is.

notyourwork 3y ago

Yelp is a cesspool of garbage at this point. I ignore it whenever it’s in my search results after they started forcing me to their app.

emodendroket 3y ago

> Before Fake Review Watch and all her research, Dean used review sites often, she said. But now she tells everyone she knows to stick with good old word-of-mouth — especially with recent additional hits to the online system, like the surge of reviews spurred by COVID-19 policies and new reports of AI chatbot-written Amazon reviews. Kind of sick of reading this advice. If I knew someone I trusted to recommend me a contractor why would I be on Yelp looking in the first place? Hello.

Solvency 3y ago
This is where NextDoor comes in. I guarantee your entire town is likely already on there talking about contractors they've used. The app is slow and sluggish but it's legitimately where most homeowners and "regular townsie folk" are these days.
- emodendroket 3y ago
  I am on some Facebook neighborhood groups which I think are similar enough to Nextdoor, but there are plenty of scammers about and also, it’s hard to trust people’s judgment given how many idiotic posts you see floating around.

deleted 3y ago

[deleted]

bilater 3y ago

Reminds me of this rule of thumb when navigating Yelp: https://twitter.com/fwong/status/1569736492247044103 https://twitter.com/fwong/status/1569736492247044103

ryandrake 3y ago

I hate to be dismissive, but this is kind of a "Water Is Wet" article. Does anyone really think online reviews are anything but totally gamed, bought, and paid for? It's kind of a Universal Law Of Spam: Any place on the Internet where users can add content and where content might be profitable to someone, will eventually end up full of spam and paid-for content.

trabant00 3y ago
Funny to find your post at the bottom of the thread. I tried to use a few apps for finding restaurants, compared their rating with the experience I had and concluded it's a waste of time and money. There's not even a need to know or understand anything about it. No insider info from Yelp, not experiencing it as a business owner. Just try and see for yourself.
hwayne 3y ago
Not everybody is online enough to know this!
standardUser 3y ago
Of course some reviews are real, and there's an art to deciphering the wheat from the chaff. The problem is that there's the constant pressure you describe to drown out legitimate reviews or otherwise game the system, so that 'art' becomes more and more tricky and less reliable. I gave up on reading the Yelp tea leaves a long time ago, and other prominent ones like Google Maps and Tripadvisor have been getting worse over time. I can still usually intuit my way through Amazon and and Steam reviews, if there's enough reviews and I take my time.
Nextgrid 3y ago
There's huge money to be made pretending spam isn't a real problem. It used to be that spam was universally despised, fought and even prosecuted in some jurisdictions. But it turns out that it's cheaper (and actually profitable) to join the spammers than to fight them, especially if you have an endless supply of marks that will guzzle the spam without second thoughts.

stainablesteel 3y ago

i never use yelp, i can typically tell what a restaraunt is like by the name and pictures on google, and whether they serve craft beer or brand names, i don't look at reviews or how much it costs

gaucheries 3y ago

> Yelp, founded in San Francisco in 2004, is deeply entrenched in American consumer habits is it really? couldn't tell you the last time i loaded yelp.com on purpose.

NegativeK 3y ago
People reading HN are not the average American.
kelnos 3y ago
Your experience may not be the general American experience. It is a bit ridiculous that I need to point this out.
- gaucheries 3y ago
  you don’t need to do anything

xyst 3y ago

I am surprised a yelp/google alternative has not been created with a feature to establish provenance of a review. Perhaps something like a receipt which contains a unique link and that person can provide a review through the link. The review will be posted as "verified transaction" on the site and overall score of the restaurant is determined from there. "Anonymous" reviews can still be posted but won't be factored into the overall score and will be deprioritized by the system/algorithm. Of course this does not necessarily stop businesses from pumping out fake transaction tickets and offering users to write fake reviews. But at least a paper trail would be established; and if a historically 1-2 star business with verified transactions and 1-2 reviews per day suddenly sees a spike to 4-5 stars with 10+ reviews per day over the same period. Then it can be flagged as suspicious and team can review the paper trail.

janalsncm 3y ago
Additionally, once you start creating a paper trail it starts to look more and more like a concerted criminal fraud operation. Yours is a good idea.
sensanaty 3y ago
I actually worked for a startup that did this, albeit it wasn't public-facing, rather we'd incentivize people to leave reviews in exchange for a coupon code or whatever Turns out a lot of restaurant owners would prefer to straight up own the reviews/influence it themselves rather than get legitimate, anonymized ones. There's just not that much of a reason to genuinely improve as a service when there's so much cutthroat competition
- moneywoes 3y ago
  Did they pivot?
  - sensanaty 3y ago
    Not sure, I left quite a while ago, haven't really kept up with them. I doubt it though, the CEO didn't care too much about infinite growth and just wanted to build a helpful app for the handful of people who would appreciate a service like that

Tycho 3y ago

Reviews are just text associated with a date and a place. Seems like someone should be able to build a simple app or website that the public can use and trust for reviews, that doesn't cost much to run. We don't need huge businesses liek TripAdvisor (which mysteriously doesn't let you sort results any more). There's a few challenges. Fake reviews is one. Also the self-selection bias - people generally only feel compelled to leave a review after a bad experience. Or, being supportive, will be overly generous with the rating. Perhaps if the review site could somehow prompt recent patrons of a set of establishments to not rate but rank the establishments, then some algorithm could convert that into a global score/ranking. You, an otherwise disinterested citizen, receive an email or text asking you to rank restaurants A, B, C which you have recently visited. You are incentivised to respond by a prize draw or something. Would this work? The hard part I suppose is getting the contact details of the potential reviewers.

rfwhyte 3y ago

Online reviews have become one of the dankest cesspits of misinformation and deceit on the entire internet. Amazon's reviews are so thoroughly gamed as to be functionally useless and Amazon knows this and tacitly supports it through their inaction. Yelp and Google Places review are being gamed en-masse by bad actors and scammers and have become so distorted that by relying upon them you're now more likely have a bad experience than a good one. Review websites are stuffed full of advertorials and universally have such super shady advertising / sponsorship policies it's functionally impossible to know whether the review you're reading was paid for or not, and all the big tech sites are so dependent on advertising revenue, they almost never post critical review for fear of upsetting their advertisers. It's a unspoken rule in the industry these days that negative reviews are simply not published. Not to mention damn near every single product review video on YouTube, IG Reels, Tiktok, or wherever else now is either straight up sponsored content where the "Reviewer" is receiving monetary compensation for their so called "Review" or at minimum is receiving the product for free, which creates yet another perverse incentive, as anyone who posts a negative review will simply stop getting free products to review in the future. There are literally dozens of 100+ person companies in China and India who's entire business model is faking online reviews. The entire industry is broken, and the average consumer can't find literally a single source for unbiased, trust-worthy reviews of anything anymore, which I for one believe is the system working exactly as designed and intended. Never forget the rich will sacrifice literally anything that's generally considered to be part of the "Public good" if it means they'll get rich faster, which is exactly what's happening here.

datadeft 3y ago

It is such an easy thing to solve with a blockchain. We do not need a 3rd part like Yelp. Just a chain with 2 actors, restaurants and reviewers.

2023throwawayy 3y ago
And that solves fake reviews... how?
- datadeft 3y ago
  I am pretty sure that we can use clustering for that. As they said there was a clear pattern the restaurant recognized. If they had all the data it would be easy. The reviewer could also ask for a token from the restaurant before submitting the review or a signature, there are plenty of options.
  - 2023throwawayy 3y ago
    And what exactly does a blockchain provide here that a plain ole' database does not?
    - datadeft 3y ago
      Tamperproof history

hospitalJail 3y ago

Man I forgot about Yelp until I read this. Apparently its because its packaged inside Apple's "Walled Garden" Maps. Seems like they got a weed in there that needs to be picked.

acumenical 3y ago

In the future, reviews will be automated, thus immunizing them from unsavory practices such as paid reviews. Your GPS-enabled wearable compute device will determine which establishment you patronized and for how long, and biometric data will determine whether you enjoyed your time there. These data will be "anonymized" and aggregated. "90% of [brand] Watch wearers like You experienced heightened happiness scores at this restaurant!" Trust in reviews will have been regained and the world will be a better place.

morkalork 3y ago
Hello, [restaurant name], this is Alice from [brand] calling. Would you be interested in our premium marketing package? Features include enhanced filtering of "anomalous" (negative) happiness-score(tm) readings. The subscription costs only 1,337 credits/month.
- acumenical 3y ago
  Good point if the app was independent like Yelp. I envisioned this as a first party product from a more powerful company, which makes the Watch as well, that could then directly integrate the consumer/review data into their Maps app on their Phones and other devices. Alice from [brand] would then have to find a way to get customers to take off their watches or somehow artificially inflate their happiness scores. Or, funnel naturally happier people to their establishment, similar to lead sourcing.

kingofkyiv 3y ago

[dead]

cromulent 3y ago

This article about TripAdvisor immediately comes to mind. https://www.vice.com/en/article/434gqw/i-made-my-shed-the-top-rated-restaurant-on-tripadvisor https://www.vice.com/en/article/434gqw/i-made-my-shed-the-to...

kristopolous 3y ago

It's a protection racket. Total scam. If you have 20 minutes I did a show about it 2 years ago https://youtube.com/watch?v=e9rmh8O_ksM https://youtube.com/watch?v=e9rmh8O_ksM. I guess you can 2.0 speed that and get it done in 10.

theshrike79 3y ago
https://www.youtube.com/watch?v=dechvhb0Meo https://www.youtube.com/watch?v=dechvhb0Meo
- kristopolous 3y ago
  The original model was as a long-form instagram live stream which I understand nobody did but I was being experimental. Basically a single shot, single take show on alternative platforms. The portrait format was to accommodate that media expectation. I made dozens of them ... they failed to gain traction. A few are over 100 views a couple years later (like this: https://www.youtube.com/watch?v=3byGVjPq468 https://www.youtube.com/watch?v=3byGVjPq468) but I have still failed to crack the world of social media. It's not zero ... that's nice I guess. I'm planning to do a reboot sometime soon. I took a couple weeklong trips around the country to some research libraries in the past year or so, but I'd like to execute differently to try to be more effective. This was my third show run attempt. I think I may need a new foil for my host. The guy I did the shows with is a longtime TV comedy writer (https://www.imdb.com/name/nm1104065/ https://www.imdb.com/name/nm1104065/) but he's not exactly a second city graduate.

balderdash 3y ago

I think I’m more surprised that anyone places any faith in a yelp review

louzell 3y ago

Go Kay Dean! Yelp should hire her and give her a team. It's a win-win. She is clearly passionate about the problem, and effective. Yelp gains added trust of their bread and butter, the review system.

hexo 3y ago

Who does really use Yelp? I've seen no one. Is it just US-only thing, or?

bborud 3y ago

I deleted the Yelp app after a few too many bad restaurant experiences AND the fact that they removed reviews I wrote that were factual and helpful.

Graziano_M 3y ago

Seeing the 'Moving company from Florida' triggered me. I was scammed by a moving company from Florida in 2020, and I did a deep dive to look into them and I found huge networks of moving companies, scammers, & reviewers, all in cahoots. I actually managed to get one company's fake reviews removed from google maps, bringing them from a 4.8+ to a 1.x overnight. Within a few months they were back to > 4, but the owners had sold the company and started a new one. It's insane how deep this goes. At best it's done by companies that want to cheat the system by getting an unfairly good review, but at its worst it's used to trick users into a false sense of confidence so that they can be scammed.

kansface 3y ago
Yeah, I feel like Yelp et al chiefly facilitate fraud at this point.
- datavirtue 3y ago
  Any silicon valley middle-man venture is a fraud/scam engine.
m463 3y ago
Same with tow trucks. Trying to find a tow truck on your own is a HUGE mess. Finding a tow company in the area invariably leads to some sort of "virtual tow truck call center" that takes a bunch of money and subcontracts out to actual tow truck drivers. My solution was to pay for the roadside assistance via my car insurance and call them to sort things out. Now emergency plumbers - same mess, but I haven't figured out a solution for that.
- bombcar 3y ago
  I just give up and use commercial services. They’re usually willing to do residential or recommend someone who can. For tow trucks just search for ones that can do a semi. They’ll know who to call.
- jstarfish 3y ago

fernly 3y ago

Posted article couldn't trouble to link Ms. Dean's actual channel. Here it is: https://www.youtube.com/@fakereviewwatch https://www.youtube.com/@fakereviewwatch It's pretty good stuff. She builds a devastating case very methodically, showing how the identical glowing review text is posted to yelp, to FB, and to Google, by different users -- and then traces other reviews posted by those same user ids showing how they are plagiarizing Tripadvisor posts.

rhyme-boss 3y ago

How do you aggregate people's preferences into scores and not have it get gamed?

g42gregory 3y ago

Who thinks that these reviews can be trusted? When I read Yelp reviews, I look for the absence of negative, not presence of positive.

datavirtue 3y ago
Perhaps you could just scrape yelp and use machine learning to show the patterns?
bombcar 3y ago
I just read a few of the middle/bad reviews and see if they seem like reasonable complaints.
NSMutableSet 3y ago
How I use Yelp reviews: 5-star, less than 100 reviews: probably paid for reviews. I've never been to a "bad" place in this category, but I usually avoid it unless it's the most appealing option, so my sample size is small 4+ star, 1000+ reviews: definitely not bad. i would rate every place ive been to in this category as good or excellent, although some are overpriced. all of my favorite places in the bay area fall between 4.0 and 4.5 3.5 star, 100+ reviews: possibly good, probably not bad. ill check reviews, if people are primarily complaining about service and not food, it's probably good. i never pick 3.5 places if it's somewhere i've never been before, but i won't veto one if someone else suggests it 3.5 star, less than 100 reviews: i avoid these completely. i've still never been to one less than 3.5 stars, any number reviews: i also avoid these completely

klabb3 3y ago

If you think it’s bad now (spoiler: it is worse than you think), this is merely the calm before the storm. We haven’t yet seen the massive saturation of LLM generated content on everything from reviews to job applications to anything else you can possibly think of. The friction of sentiment-hacking operations will be dramatically reduced and we don’t have any effective mitigations. To make things worse, the only ones who can integrate the solutions (the platform owners) are not incentivized to take it very seriously. Fun times ahead!

emeril 3y ago

I don't work at Yelp though I'm pretty sure that the reviews which stay on the website for any length of time are those posted by users who actively use the website to search for businesses (i.e., tracked via their cookie) i.e., while there certainly can be paid/manipulated reviews, the barrier to doing so is higher here than elsewhere (e.g., google reviews, tripadvisor, etc.) and there's enough people who are happy to use other less controlled sites, I haven't witnessed any truly widespread fraudulent reviews like I have elsewhere) Generally speaking, I'd say Yelp reviews are, generally speaking, comparatively the most reliable I can't speak to how they sort/present search results - that does seem potentially shady

11thEarlOfMar 3y ago

What astonishes me about both Yelp! and Trip Advisor is their complete inability to grow their share price, even with near-monopolies in Restaurants (Yelp!) and Hotels (TripAdvisor). Yelp! catalogs 205 Million reviews last year. 30 Million for Trip Advisor. But even with such dominance, in the last 10 years, Yelp! is up about 38% and Trip Advisor is down 42%. What is it about their businesses that makes profitability growth so elusive?

Seattle3503 3y ago

> She also reached out to Sasha Ganji, the lawyer whose signature was appended to the letter she’d received from the law firm appearing to represent SavantCare. Ganji never touched the document, he told Dean; apparently someone had used his name and signature without his knowledge. That is a serious thing to do. This is criminal fraud right?

kelnos 3y ago

The article links to one of her videos where she talks about the owner of a veterinary practice who buys reviews. I brought one of our cats there twice last year for after-hours urgent care, and my experience was -- fortunately -- positive both times. Friends bring their dog there for primary care regularly. I'm just really wondering why the owner feels the need to do this. Is it a situation where you just have to do it, else you'll get buried under your competitors? Should I expect that our primary vet does this too, and that the other vet practices in our area also do it? Shady either way, but could it be a damned if you do, damned if you don't type thing? Certainly there are business sectors where there's so much competition that some of them will resort to shady tactics like this to gain customers. But if anything, there's a shortage of vet practices in San Francisco (and the entire bay area, really). Gaining customers here feels like it's more about having available appointments in a reasonable time frame than anything else.

pcurve 3y ago
"I'm just really wondering why the owner feels the need to do this." Everyone knows people who had negative reviewers are more likely to leave reviews, but what people don't often understand is, percentage of people who actually leave ANY review is still exceedingly small. I've heard as long as 1 in a few hundred or even a thousand. So getting that first few positive ones is critical, to kick off the virtuous cycle of customer acquisition. You'd better deliver though, once customers start coming in.
toxicFork 3y ago
It could just be the recommended marketing practice by the local small business school. Right after "Rent a week on the ad spot by the bus station" could be "buy a few fake reviews on yelp".

janandonly 3y ago

Would it be possible to build a better YELP, maybe based on open standards like Nostr? Is it possible to vote anonymously without being able to fake those votes/reviews?

jadamczyk 3y ago

When I was on a trip in the US for vacation, I found out that most of the review services can’t be trusted. Despite good or bad reviews it was a complete lottery. Meanwhile in Poland when a restaurant has a good review online you can actually expect reasonable service and food quality.