3 ms·

A Recommendation for Google's Webspam Team

43 points by duck 16y ago

ladon86 16y ago
Great post. I don't know a lot about how well this would work in practice (perhaps Matt Cutts could comment), but anchor text link gaming is a huge problem and this sounds like a decent solution. Many of Google's results have started to 'feel' really bad lately - I'm doing a lot more wading through crap and a lot less finding what I want quickly.
WillyF 16y ago
I like Rand's proposed solution, and I'm confident that the folks at Google can come up with something even better and more effective. I'm finding that natural link building is getting harder and harder. Much of this is because social media sites (that often use nofollow) are now how people talk about stuff they like. And when they do have a Tumblr or Wordpress.com blog, the links aren't very valuable. Most of the valuable links that I've been able to build lately have come from my asking people to link to my sites. In a way, we're seeing more and more PageRank inequality. The people who control the sites with the most link equity know what they have, and they're not willing to share. I'm lucky that my niche isn't completely commercial, so there are still plenty of easy, valuable links to build, but I can't imagine how tough it would be to SEO a site in a competitive, commercial niche.
tnorthcutt 16y ago
What is the purpose of only notifying 65% of the sites found (and publicizing it that way)? Is that to create some uncertainty, so that even sites that weren't discovered by this method will give some thought to changing, for fear that they are part of the 35% not notified?
- bryanh 16y ago
  I thought it was for exactly this reason. He doesn't elaborate though.
praptak 16y ago
The proposed solution requires a lot of manual tweaking: "Have manual spam raters spot check through a significant sample size of the pages", "If the false positives follow some easily identifiable pattern, write code to exclude them and their ilk from the filtration system." Google seems to avoid this kind of thing as a principle.
- WillyF 16y ago
  That's the image that they like to create, and I think for the most part they like to have the algorithm do as much of the work as it possibly can; however, this post by Aaron Wall that includes Google's General Guidelines for Remote Quality Graders might make you think differently: http://www.seobook.com/full-text-googles-general-guidelines-remote-quality-raters-april-2007 http://www.seobook.com/full-text-googles-general-guidelines-... (Looks like the document has been taken offline since.) In fact, I think that using manual raters to tweak the algorithm makes a lot of sense.
- lmkg 16y ago
  Not really. This step basically says either "Look at the data and use that to guide how you write your algorithms" or "manually curate a statistically significant set of training data." You're not having the humans look at the data and include/exclude links or sites, you're having them look at the data to figure out heuristics for applying to the eight hundred billion pages they didn't look at. This step doesn't need to scale, so manual labor is not an incorrect solution. Given the context, and the ever-evolving nature of SEO, having human review is necessary part of the process, and this is the best way to add that human element in a scalable fashion.
jefflinwood 16y ago
Rand's blog post demonstrates that there's now a pretty big disconnect between anchor text keywords and actual link quality - he makes an excellent point that natural links aren't likely to put the keywords directly into the anchor. It's such an interesting problem - on one hand, you have autoblogs populating the internet with Made-For-Adsense keyword-stuffed content that would soak up long tail search results, so Google can't just ignore inbound links as a measure of relevance, and on the other, you have keyword-stuffed anchor text links that aren't particularly relevant to the content of the page they're on. What other page metrics would one use to measure relevancy today that wouldn't be as susceptible to SEO gaming?
pornel 16y ago
Maybe they're not tackling this problem, because they realize it's futile? If exact keywords stop working, spammers can easily start adding extra words or variations to anchor text (with something like Markov chains one can create lots of believable variations).
DanielBMarkham 16y ago
The only thing I would add would be to make sure the links are cross-domain. I can imagine several different websites operating in a target niche, say home mortgages, that might have menus and pages and such that reflect common search terms in that area. In fact, if you think about it, if you're writing for a niche you should anticipate what people want to know about that niche, and search terms give you that, so there's quite a bit of commonality. But as long as the links were cross-domain, sure, makes sense to me. I just wonder if there aren't a lot of more edge cases like this. At the end of the article he kind of waves his hands around and says something like "and if this doesn't work, then you can use artificial intelligence and machine learning" That's a sign that perhaps there are many more holes in this line of attack that he just doesn't care to deal with. Don't know.

43 points by duck 16y ago

Great post. I don't know a lot about how well this would work in practice (perhaps Matt Cutts could comment), but anchor text link gaming is a huge problem and this sounds like a decent solution. Many of Google's results have started to 'feel' really bad lately - I'm doing a lot more wading through crap and a lot less finding what I want quickly.

WillyF 16y ago

I like Rand's proposed solution, and I'm confident that the folks at Google can come up with something even better and more effective. I'm finding that natural link building is getting harder and harder. Much of this is because social media sites (that often use nofollow) are now how people talk about stuff they like. And when they do have a Tumblr or Wordpress.com blog, the links aren't very valuable. Most of the valuable links that I've been able to build lately have come from my asking people to link to my sites. In a way, we're seeing more and more PageRank inequality. The people who control the sites with the most link equity know what they have, and they're not willing to share. I'm lucky that my niche isn't completely commercial, so there are still plenty of easy, valuable links to build, but I can't imagine how tough it would be to SEO a site in a competitive, commercial niche.

tnorthcutt 16y ago

What is the purpose of only notifying 65% of the sites found (and publicizing it that way)? Is that to create some uncertainty, so that even sites that weren't discovered by this method will give some thought to changing, for fear that they are part of the 35% not notified?

bryanh 16y ago
I thought it was for exactly this reason. He doesn't elaborate though.

praptak 16y ago

The proposed solution requires a lot of manual tweaking: "Have manual spam raters spot check through a significant sample size of the pages", "If the false positives follow some easily identifiable pattern, write code to exclude them and their ilk from the filtration system." Google seems to avoid this kind of thing as a principle.

WillyF 16y ago
That's the image that they like to create, and I think for the most part they like to have the algorithm do as much of the work as it possibly can; however, this post by Aaron Wall that includes Google's General Guidelines for Remote Quality Graders might make you think differently: http://www.seobook.com/full-text-googles-general-guidelines-remote-quality-raters-april-2007 http://www.seobook.com/full-text-googles-general-guidelines-... (Looks like the document has been taken offline since.) In fact, I think that using manual raters to tweak the algorithm makes a lot of sense.
lmkg 16y ago
Not really. This step basically says either "Look at the data and use that to guide how you write your algorithms" or "manually curate a statistically significant set of training data." You're not having the humans look at the data and include/exclude links or sites, you're having them look at the data to figure out heuristics for applying to the eight hundred billion pages they didn't look at. This step doesn't need to scale, so manual labor is not an incorrect solution. Given the context, and the ever-evolving nature of SEO, having human review is necessary part of the process, and this is the best way to add that human element in a scalable fashion.

jefflinwood 16y ago

Rand's blog post demonstrates that there's now a pretty big disconnect between anchor text keywords and actual link quality - he makes an excellent point that natural links aren't likely to put the keywords directly into the anchor. It's such an interesting problem - on one hand, you have autoblogs populating the internet with Made-For-Adsense keyword-stuffed content that would soak up long tail search results, so Google can't just ignore inbound links as a measure of relevance, and on the other, you have keyword-stuffed anchor text links that aren't particularly relevant to the content of the page they're on. What other page metrics would one use to measure relevancy today that wouldn't be as susceptible to SEO gaming?

pornel 16y ago

Maybe they're not tackling this problem, because they realize it's futile? If exact keywords stop working, spammers can easily start adding extra words or variations to anchor text (with something like Markov chains one can create lots of believable variations).

DanielBMarkham 16y ago

The only thing I would add would be to make sure the links are cross-domain. I can imagine several different websites operating in a target niche, say home mortgages, that might have menus and pages and such that reflect common search terms in that area. In fact, if you think about it, if you're writing for a niche you should anticipate what people want to know about that niche, and search terms give you that, so there's quite a bit of commonality. But as long as the links were cross-domain, sure, makes sense to me. I just wonder if there aren't a lot of more edge cases like this. At the end of the article he kind of waves his hands around and says something like "and if this doesn't work, then you can use artificial intelligence and machine learning" That's a sign that perhaps there are many more holes in this line of attack that he just doesn't care to deal with. Don't know.