5 ms·

Yelp is fundamentally flawed. Firstly, they have a clear conflict of interest, which has been discussed many times, when it comes to selling advertising. Buyin

by cletus 15y ago

Yelp is fundamentally flawed. Firstly, they have a clear conflict of interest, which has been discussed many times, when it comes to selling advertising. Buying advertising (anecdotally) seems to make bad reviews magically disappear. Secondly, and this has always been the problem with "local", is you need a certain critical mass for it to be usable. You can argue that Yelp has reached this point in many cases (although see the next two points) but there are many businesses with <5 reviews. Third, there is too much friction in asking people to review (and even rate things). Most people simply don't and probably never will. This exacerbates the "critical mass" problem but also introduces a selection bias. The people who comment and rate aren't necessarily representative of general opinions or you (the personalization problem). I've gone to eat at some places in NYC that are 3.5+ stars that have varied from average to terrible. In some cases I've gone with someone who shared this positive review but--and I realize the counterargument to this is that it's subjective--they're just wrong. Fourth, there is a clear fraud problem with reviews and ratings. People are clearly paid to give positive reviews (eg you see someone rate a given car dealership in the Bay area on one day and then another in Maine the next day and so on). Of any of the companies in "local", IMHO Google is in the best position to deal with this particular problem (disclaimer: I work for Google). Lastly, as such reviews become increasingly important, there is the issue of extortion. If this hasn't happened already it will. Criminals already target websites with DDoS attacks that go away if the site in question pays what amounts to "protection money". There's nothing really to stop such criminal enterprises shaking down businesses with the threat of a bad slew of reviews. It's worth making extra mention of personalization. Many (Google included) seem to view "social search" and "social recommendations" as some kind of panacea to some or even all of these problems. I disagree. I know a couple of people who, say, like Adam Sandler movies. I do not. Not even remotely. Their movie recommendations are so diametrically opposed to mine that I can pretty much take the opposite of what they recommend. The way forward with this will be something like the Netflix model (IMHO) where these great data mining systems will attempt to find people who are like me and have similar tastes whose recommendations will likely coincide with mine.

ed209 15y ago
Friction on posting a review is a huge problem. On the flip side, you ask any of your friends "where's a good place to eat tonight?" no doubt they'll have a few reviews in them and promptly let you know some places you should try. I think this industry needs more of a q&a (quora type) approach where I can ask the same question that I would ask my friends. That way I'm only recommending places (take that as a 4+ star review) and not criticising businesses. This benefits both customers by show places that have been recommended multiple times as an indicator of quality and business that don't suffer as a result of a bad night or rouge staff member. This approach makes it easier to actually find what you too because by virtue of the way that place is suggested ("where is a good spot for breakfast?") we already get a ton of meta data.
nathanwdavis 15y ago
This does bode well for sites like AngiesList.com which charge a small membership fee (to end users), don't allow anonymous reviews, and allow businesses to respond publicly to individual review's. (I work for Angie's List)
- Drbble 15y ago
  My understanding was that vendors just pay the entry fee to game Angie List reviews.
pbreit 15y ago
There's a lot of FUD in there (maybe a conflict of interest?). The media industry operates fine on an ad-based model. Yes, reports of disappearing reviews have only been anecdotal and never close to proven (and obviously unnecessary). People love to review things and there are tons of ways to incent such. It should be possible to manage fraudulent reviews as many other companies do (Google included)(Max and Jeremy also have a lot of experience there). The extortion thing is simply ridiculous. Yelp was originally much more "social grafty" and has stepped back significantly from that. I have little doubt it will continue to find ways to maximize review relevance aside from social graph.
- prof_hobart 15y ago
  Not sure what you mean by the media industry working fine on an ad-based model. If you mean it makes money then possibly it does, but if you mean that it typically manages to remain a trustworthy independent source of news/reviews etc, then I'd strongly suggest that it fails quite badly. Advertisers threatening to pull their accounts because the publication/channel has done something they don't like is the most visible manifestation of this, but the modern newspaper industry is basically run at the behest of advertisers and the typical output of these papers (just look at the amount of PR-driven stories that make it into the average paper on a daily basis) largely reflects this.
dmoy 15y ago
Collaborative filtering would be cool, yes. However the situation for netflix is a bit different than restaurants. It's hard to get really good overlap with a large number of people for a lunch that exists only in one specific location, compared to a film which can be viewed anywhere in the world, at any time, by anyone (and then encouraged to review by their direct point of sale (Netflix)). Also the cost (mostly time) in generating a lot of reviews by a single person to establish any kind of baseline is much higher. And, on top of that, it doesn't help the random anonymous user with 0 reviews submitted as much. Someone should get on that though, because if you could solve the problem of how to set up proper recommendation systems for food, that'd be awesome. Who knows, maybe Yelp as experimented with this? I don't know what kind of logged-in userbase they have though. Maybe if you had something integrated with the restaurants, where people could be logged in via their smartphones, and give a review to some NFC device at the restaurant. Lower the barrier of reviewing, encourage reviews right at the point of sale (similar to reviewing on netflix.com). Someone must have already thought of this before though.
jonnathanson 15y ago
I believe there's room in the local space for a service that combines check-ins with reviews. In other words, only real people, who've physically been to a given restaurant, are able to review it. While you'd still suffer from selection-bias problems with this model, at least the self-selectors would be people with a propensity to try a lot of restaurants and review them -- not population-representative, perhaps, but at least dedicated to the task. Of course, there are plenty of ways to game such a system. (You don't actually have to be at a restaurant to check into it, but you do need to be within a certain geographic radius -- so this would cut down on bogus or fraudulent reviews from people nowhere near the location). But those methods are harder to pull off, and they necessarily apply to a much smaller subset of potential gamers -- thereby making gaming a smaller-scale proposition. Collaborative filtering would lend itself well to this model, too. Toss in some sort of karma system, as well, and you've got a check against system gaming and low-quality reviews.
- JoachimSchipper 15y ago
  Believing the device's idea of where it is isn't going to help against any kind of serious criminal (get root, hook API, profit; and what about people without smartphones, anyway?). Triangulating a phone via cell phone towers might work, but that opens a whole new can of worms: you need the carriers to cooperate; it's a privacy nightmare; conclusively linking a Yelp username to a physical SIM card is not easy; and so forth.
  - jonnathanson 15y ago
    True, but at the very least, we're reducing the set of all possible fraudsters and system-gamers. How many of the fraudulent reviews on Yelp do we really believe are coming from fraudsters with this level of sophistication? My guess is that it's a pretty small percentage. Seems far more likely that most bogus reviews are coming from competitive small business owners, paid-off users, and other relatively average folk with a minimal level of technical understanding -- or even sufficient malice to drive them to become more sophisticated. (It doesn't help that Yelp may, or may not, be assigning greater weight to negative reviews unless/until a small business owner buys into their advertising).
rabbitonrails 15y ago
Queue the class action lawsuit.
agillentine 15y ago
"The way forward with this will be something like the Netflix model (IMHO) where these great data mining systems will attempt to find people who are like me and have similar tastes whose recommendations will likely coincide with mine." My company Whit.li has built an api to do just that. We are launching this weekend at SXSW check it out at whit.li.

by cletus 15y ago

ed209 15y ago

Friction on posting a review is a huge problem. On the flip side, you ask any of your friends "where's a good place to eat tonight?" no doubt they'll have a few reviews in them and promptly let you know some places you should try. I think this industry needs more of a q&a (quora type) approach where I can ask the same question that I would ask my friends. That way I'm only recommending places (take that as a 4+ star review) and not criticising businesses. This benefits both customers by show places that have been recommended multiple times as an indicator of quality and business that don't suffer as a result of a bad night or rouge staff member. This approach makes it easier to actually find what you too because by virtue of the way that place is suggested ("where is a good spot for breakfast?") we already get a ton of meta data.

nathanwdavis 15y ago

This does bode well for sites like AngiesList.com which charge a small membership fee (to end users), don't allow anonymous reviews, and allow businesses to respond publicly to individual review's. (I work for Angie's List)

Drbble 15y ago
My understanding was that vendors just pay the entry fee to game Angie List reviews.

pbreit 15y ago

There's a lot of FUD in there (maybe a conflict of interest?). The media industry operates fine on an ad-based model. Yes, reports of disappearing reviews have only been anecdotal and never close to proven (and obviously unnecessary). People love to review things and there are tons of ways to incent such. It should be possible to manage fraudulent reviews as many other companies do (Google included)(Max and Jeremy also have a lot of experience there). The extortion thing is simply ridiculous. Yelp was originally much more "social grafty" and has stepped back significantly from that. I have little doubt it will continue to find ways to maximize review relevance aside from social graph.

prof_hobart 15y ago
Not sure what you mean by the media industry working fine on an ad-based model. If you mean it makes money then possibly it does, but if you mean that it typically manages to remain a trustworthy independent source of news/reviews etc, then I'd strongly suggest that it fails quite badly. Advertisers threatening to pull their accounts because the publication/channel has done something they don't like is the most visible manifestation of this, but the modern newspaper industry is basically run at the behest of advertisers and the typical output of these papers (just look at the amount of PR-driven stories that make it into the average paper on a daily basis) largely reflects this.

dmoy 15y ago

Collaborative filtering would be cool, yes. However the situation for netflix is a bit different than restaurants. It's hard to get really good overlap with a large number of people for a lunch that exists only in one specific location, compared to a film which can be viewed anywhere in the world, at any time, by anyone (and then encouraged to review by their direct point of sale (Netflix)). Also the cost (mostly time) in generating a lot of reviews by a single person to establish any kind of baseline is much higher. And, on top of that, it doesn't help the random anonymous user with 0 reviews submitted as much. Someone should get on that though, because if you could solve the problem of how to set up proper recommendation systems for food, that'd be awesome. Who knows, maybe Yelp as experimented with this? I don't know what kind of logged-in userbase they have though. Maybe if you had something integrated with the restaurants, where people could be logged in via their smartphones, and give a review to some NFC device at the restaurant. Lower the barrier of reviewing, encourage reviews right at the point of sale (similar to reviewing on netflix.com). Someone must have already thought of this before though.

jonnathanson 15y ago

I believe there's room in the local space for a service that combines check-ins with reviews. In other words, only real people, who've physically been to a given restaurant, are able to review it. While you'd still suffer from selection-bias problems with this model, at least the self-selectors would be people with a propensity to try a lot of restaurants and review them -- not population-representative, perhaps, but at least dedicated to the task. Of course, there are plenty of ways to game such a system. (You don't actually have to be at a restaurant to check into it, but you do need to be within a certain geographic radius -- so this would cut down on bogus or fraudulent reviews from people nowhere near the location). But those methods are harder to pull off, and they necessarily apply to a much smaller subset of potential gamers -- thereby making gaming a smaller-scale proposition. Collaborative filtering would lend itself well to this model, too. Toss in some sort of karma system, as well, and you've got a check against system gaming and low-quality reviews.

JoachimSchipper 15y ago
Believing the device's idea of where it is isn't going to help against any kind of serious criminal (get root, hook API, profit; and what about people without smartphones, anyway?). Triangulating a phone via cell phone towers might work, but that opens a whole new can of worms: you need the carriers to cooperate; it's a privacy nightmare; conclusively linking a Yelp username to a physical SIM card is not easy; and so forth.
- jonnathanson 15y ago
  True, but at the very least, we're reducing the set of all possible fraudsters and system-gamers. How many of the fraudulent reviews on Yelp do we really believe are coming from fraudsters with this level of sophistication? My guess is that it's a pretty small percentage. Seems far more likely that most bogus reviews are coming from competitive small business owners, paid-off users, and other relatively average folk with a minimal level of technical understanding -- or even sufficient malice to drive them to become more sophisticated. (It doesn't help that Yelp may, or may not, be assigning greater weight to negative reviews unless/until a small business owner buys into their advertising).

rabbitonrails 15y ago

Queue the class action lawsuit.

agillentine 15y ago

"The way forward with this will be something like the Netflix model (IMHO) where these great data mining systems will attempt to find people who are like me and have similar tastes whose recommendations will likely coincide with mine." My company Whit.li has built an api to do just that. We are launching this weekend at SXSW check it out at whit.li.