3 ms·

Spam filtering is hard? You are a programmer, imagine trying to find a way to objectively and fairly filter reviews. Unlike with emails you don't know whose lab

by timtadh 14y ago

Spam filtering is hard? You are a programmer, imagine trying to find a way to objectively and fairly filter reviews. Unlike with emails you don't know whose labels to trust. There are good business owners who will only report spam reviews as spam. Then there are bad business owners who will report negative ham as spam to try and boost their reviews. Finally, there are bad business owners who will use shell accounts to submit positive spam and then mark it as ham. In general the labels from business owners are not trust worthy which gets them into these situations. Thus they try and create other ways of identifying spam: like user engagement on the site. However, engagement doesn't always work as a metric as spammers can cultivate spam accounts by posting lots of "legitimate" reviews on unrelated businesses and then sell reviews from the accounts to attack (boost) specific businesses. Finally, users who have a tremendous experience at a business and go on Yelp to give them 5 star reviews may have no history with yelp except as a passive browser. The engagement metric will weight poorly for their review. So what is to be done? I don't have a good answer for that. This seems more like a human problem than a technical one. The best course seems to be: act ethically, serve your customers with compassion, write responses offering assistance to negative reviewers, and if you feel Yelp (and similar review system are a bad idea) put your advertising dollars elsewhere.

yoda_sl 14y ago
They are various ways to solve the problem but again it is not in Yelp's interest. Yelp's business is about selling Ad and they don't really care about the underlying issues. One simple thing that could be added to their system was a way to verify that a reviewer did really have a business relation for posting a reviews. They are various ways for solving that. Even a simpler approach: why hide those reviews. Keep them here and introduce a way for other viewer to rate the reviews aka 'X people found that review helpful'. That way all reviews are visible and the random person reading the Yelp page can decide for themselves. Unfortunately Yelp and their filtering system have a different approach and instead usually end up filtering real customer reviews and leave the more 'spammy' visible. Then after a few weeks like that, you will receive a phone call from Yelp sales folks telling you that if you purchase 'ads' then your potential 5 stars review (if any are left visible) will be pushed to the top. Sorry but for me I consider that a scam!
- timtadh 14y ago
  There is no relationship between the sales calls and the filtering algorithm. That is what the dismissed class action lawsuit was about. All "evidence" people have is purely anecdotal. Verifying a "business" relationship may work but it still won't solve all of the problems. You can still pay people to show up at your restaurant, pay with their credit card and write a five star review. You can still pay people to show up at a competitors restaurant, pay with their credit card and write a one star review. It simply increases the cost of the spam (which is a good thing). They should probably create a way to do this but it won't solve the fundamental problem: the reviewer pool is an unreliable source of information. "X people found that review helpful" A helpful review is not the same as an accurate review based on experience. I would find a well written review helpful. However, the review could still be inaccurate or fake just well written. This is simply another data point for spam detection and one with many confounders at that. One has to simply google around about the state of Amazon reviews to see the flaws in this approach. Once again. I stand by my assertion, detecting spam reviews is extremely difficult because it is a human problem.
chris_wot 14y ago
Spam filtering is hard? You are a programmer, imagine trying to find a way to objectively and fairly filter reviews and So what is to be done? From a consumer's point of view, don't use Yelp. Easy.

by timtadh 14y ago

yoda_sl 14y ago

They are various ways to solve the problem but again it is not in Yelp's interest. Yelp's business is about selling Ad and they don't really care about the underlying issues. One simple thing that could be added to their system was a way to verify that a reviewer did really have a business relation for posting a reviews. They are various ways for solving that. Even a simpler approach: why hide those reviews. Keep them here and introduce a way for other viewer to rate the reviews aka 'X people found that review helpful'. That way all reviews are visible and the random person reading the Yelp page can decide for themselves. Unfortunately Yelp and their filtering system have a different approach and instead usually end up filtering real customer reviews and leave the more 'spammy' visible. Then after a few weeks like that, you will receive a phone call from Yelp sales folks telling you that if you purchase 'ads' then your potential 5 stars review (if any are left visible) will be pushed to the top. Sorry but for me I consider that a scam!

timtadh 14y ago
There is no relationship between the sales calls and the filtering algorithm. That is what the dismissed class action lawsuit was about. All "evidence" people have is purely anecdotal. Verifying a "business" relationship may work but it still won't solve all of the problems. You can still pay people to show up at your restaurant, pay with their credit card and write a five star review. You can still pay people to show up at a competitors restaurant, pay with their credit card and write a one star review. It simply increases the cost of the spam (which is a good thing). They should probably create a way to do this but it won't solve the fundamental problem: the reviewer pool is an unreliable source of information. "X people found that review helpful" A helpful review is not the same as an accurate review based on experience. I would find a well written review helpful. However, the review could still be inaccurate or fake just well written. This is simply another data point for spam detection and one with many confounders at that. One has to simply google around about the state of Amazon reviews to see the flaws in this approach. Once again. I stand by my assertion, detecting spam reviews is extremely difficult because it is a human problem.

chris_wot 14y ago

Spam filtering is hard? You are a programmer, imagine trying to find a way to objectively and fairly filter reviews and So what is to be done? From a consumer's point of view, don't use Yelp. Easy.