6 ms·

Why did Google blacklist all of your Tldscan sites? Was it just because your sites' content was updated automatically? Or was it because you did something wrong

Why did Google blacklist all of your Tldscan sites? Was it just because your sites' content was updated automatically? Or was it because you did something wrong for SEO?

epi0Bauqu 16y ago
Here's my (completely unsubstantiated) theory. It happend literally the day after crossing $500 & 50K views in adsense. I'm guessing one of those was a trigger for manual review by some contractor, perhaps overseas. They looked at my sites for 3 sec, found them to be cookie-cutter and decided to blacklist the account. I get the impression they shoot first, ask questions later. I didn't feel like dealing with it all or starting over so I just moved on to other things.
- jacquesm 16y ago
  I wonder how many otherwise viable little businesses fall victim to this heavy handed approach. Is this one of the reasons DDG got started?
  - epi0Bauqu 16y ago
    Well, it did push me onto other things, though I wasn't (at least consciously) looking to get back at Google.
- stakent 16y ago
  In my case some clicks worth over $2 (if I remember correctly) triggered manual review. They sent me canned email about tos etc. The funny part is that the domain in question was already expired when email arrived because I've decided to stop this venture.
- asmithmd1 16y ago
  I talked to someone from Google at I/O who should know and he claimed they don't play "Whack a mole" with websites. They will tweak their ranking algorithm to punish the behavior they see in a web site they don't want to be ranked.
  - bobds 16y ago
    Lies, damned lies.
  - epi0Bauqu 16y ago
    Interesting, but in this case it wasn't that it was ranked lower, but that one day I had 25 domains indexed fine and the next day they were no where to be found, i.e. not in the index at all. And I had plenty of other domains (not in that adsense account) that were still ranked fine.
  - bigiain 16y ago
    Matt Cutts often tells us this - but he talks specifically from a search web spam perspective, I suspect the Adsense team have different rules and probably can (and do) "play Whack a mole" when appropriate.
  - Matt_Cutts 16y ago
    That was probably me. We have two sides to the webspam team at Google: engineering and manual. We definitely prefer to write algorithms so that we avoid dealing with individual websites--the idea is that you strive to fix the root cause of an issue, not to tackle specific sites. However, if we see a website that violates our guidelines and that gets past the algorithms, we are willing to take manual action. Where possible, we use the output of the manual team not only to reduce spam itself, but to train the next iteration of algorithms. For example, one of the big issues in blackhat spam this past year was illegally hacked sites. Our algorithms weren't doing the best job on hacked sites, so the manual team kept an eye out for hacked sites to remove them (and often to alert the website owners that they'd been hacked). The data generated by the manual team helped us build and deploy multiple new algorithms to detect hacked sites, leading to a 90% reduction in the number of hacked sites showing up in Google's search results in the past few months. That decrease in hacked spam in turn frees up the manual team to tackle the next bleeding-edge technique the spammers use. I suspect every major search engine uses similar approaches: try to stop the majority of spam with algorithms, but be willing to take action in the mean time while engineers work to improve the algorithms.
    - 16y ago
- Matt_Cutts 16y ago
  I don't think that was the issue. The fact is that if you've got dozens of websites, each of which has lists of domains/IPs like http://www.mattcutts.com/images/verypopularwebsites-com.png http://www.mattcutts.com/images/verypopularwebsites-com.png , that is the sort of thing that users complain about and don't want showing up when they do a search. Especially if sites have autogenerated boilerplate content for each one of those links. I mean, if you're auto-generating a page that has this text: "Elcorillord.com Common misspellings and typos: Elcroillord.com, Elcorilolrd.com, Elcori.lord.com, Elcofillord.com, www.elcorillord.com, lEcorillord.com, Elcotillord.com, Elcorillor.com, Elcoeillord.com, Elcori,lord.com, Wlcorillord.com, Elcorjllord.com, Elcorillord.coom, Elocrillord.com, Elcor8llord.com, Elvorillord.com, Elcorillprd.com, Elcorillord.cim, Elcorillorf.com, Elcorilloed.com, Elorillord.com, Elckrillord.com, Elcoriplord.com, Elcorillord.ckm, Elcorillord.cm, Elcorillord.ccom, Epcorillord.com, Elcoril;ord.com, Elcoirllord.com, Elcoriillord.com, Elforillord.com, 3lcorillord.com, Elcorollord.com, Elcorillordd.com, Elcorill0rd.com, Elcorillord/com, Elcoriolord.com, Ekcorillord.com, Elcorillord.xom, Elcorillord.co, Elcorilord.com, Elcoillord.com, 4lcorillord.com, Elcoriloord.com, Elcorillorr.com, Eldorillord.com, Elcorillord..com, Elcorrillord.com, http://www.elcorillord.com http://www.elcorillord.com, El orillord.com, E.corillord.com, Elcorillord. om, Elcorilllrd.com, Elcorillrod.com, Elcoriklord.com, Elcorillorrd.com, Elcorillordcom, Elcorillkrd.com, Elcorillord.om, Elcorlilord.com, Elco4illord.com, Elcorillrd.com, Elcprillord.com, Elcodillord.com, Elcorillordc.om, Ecorillord.com, Elcoorillord.com, Slcorillord.com, Elcorillorx.com, Elcorill9rd.com, Elcorilpord.com, Elcorillord.cpm, Elcorillord.fom, Elco5illord.com, Elc9rillord.com, Elcorillird.com, Elcirillord.com, Elcorillord.clm, Elcorillors.com, Elcorillord.vom, Elcorullord.com, Elcorillord.comm, Elcorillord.c9m, Eocorillord.com, Elcorilloord.com, Elcourillourd.com, E,corillord.com, Elcorkllord.com, Elcorillodr.com, Elcorillodd.com, Elcorillord,com, Elcorillotd.com, Elcorillod.com, Elcorillor.dcom, Elcor9llord.com, Elc0rillord.com, Elcoril,ord.com, Elcorilllord.com, Elcorillo5d.com, EElcorillord.com, Elxorillord.com, E;corillord.com, Elcori;lord.com, Elcorllord.com, Elccorillord.com, Elcrillord.com, Elcoril.ord.com, Elcorilkord.com, Elcorillord.cmo, Ellcorillord.com, Eclorillord.com, Elcorillo4d.com, Rlcorillord.com, wwwelcorillord.com, Elclrillord.com, ElcorillordLcom, Dlcorillord.com, Elcorillofd.com, Elcorillore.com, Elcorillord;com, lcorillord.com, Elcorillorc.com, Elcorillord.c0m, Elcorillord.dom, Elcorillord.ocm." Surely you have to see where many people would consider that either keyword stuffing, gibberish or typo spam.
xpose2000 16y ago
Maybe this can help . . . I have a few websites that automatically make new posts. As of 10/14, they all show 0 pages indexed in Google. Previously they would get a few thousand visitors per day. I guess Google feels as though they violate their terms and removed them. It seems to me it was a manual removal. I received no emails in webmaster tools about the removal.
- Matt_Cutts 16y ago
  "I have a few websites that automatically make new posts." Making a bunch of autogenerated sites has its risks. For example, if you were just taking a bunch of MP3 names or Hot Trends queries and then scraping twitter for mentions of those phrases and slapping that all up on a website with scripts, that tends to cruft up our index with autogenerated content that users complain about and that violates our quality guidelines. Likewise, if all you were doing was scraping Twitter for phrases like sad or heartbroken or heartless and throwing that scraped Twitter content up on a webpage with a script, users would also complain about that autogenerated content and it would violate our guidelines. Would that be helpful insight?
  - epi0Bauqu 16y ago
    As a concrete case to discuss, what about something like http://poeet.com/ http://poeet.com/ I made this over a weekend. And the people whose poetry is being captured love it. But it is auto-generated in the sense you're talking about. It actually went down for a bit and I got a bunch of complaints, enough that I got it back up relatively quickly.
    - Rogan 16y ago
      Theres a clear difference between your site http://poeet.com http://poeet.com and a clean cut case of auto generated spam. Your site is actually quite creative where it is aggregating content from a twitter hashtag and indexing short poems that may otherwise go unnoticed, you are also showing the users original tweet and @user and not manipulating anything. The original poster was likely scraping content, not providing citation and for the means of having the duplicate content wrapped around ads.

Why did Google blacklist all of your Tldscan sites? Was it just because your sites' content was updated automatically? Or was it because you did something wrong for SEO?

epi0Bauqu 16y ago

Here's my (completely unsubstantiated) theory. It happend literally the day after crossing $500 & 50K views in adsense. I'm guessing one of those was a trigger for manual review by some contractor, perhaps overseas. They looked at my sites for 3 sec, found them to be cookie-cutter and decided to blacklist the account. I get the impression they shoot first, ask questions later. I didn't feel like dealing with it all or starting over so I just moved on to other things.

jacquesm 16y ago
I wonder how many otherwise viable little businesses fall victim to this heavy handed approach. Is this one of the reasons DDG got started?
- epi0Bauqu 16y ago
  Well, it did push me onto other things, though I wasn't (at least consciously) looking to get back at Google.
stakent 16y ago
In my case some clicks worth over $2 (if I remember correctly) triggered manual review. They sent me canned email about tos etc. The funny part is that the domain in question was already expired when email arrived because I've decided to stop this venture.
asmithmd1 16y ago
I talked to someone from Google at I/O who should know and he claimed they don't play "Whack a mole" with websites. They will tweak their ranking algorithm to punish the behavior they see in a web site they don't want to be ranked.
- bobds 16y ago
  Lies, damned lies.
- epi0Bauqu 16y ago
  Interesting, but in this case it wasn't that it was ranked lower, but that one day I had 25 domains indexed fine and the next day they were no where to be found, i.e. not in the index at all. And I had plenty of other domains (not in that adsense account) that were still ranked fine.
- bigiain 16y ago
  Matt Cutts often tells us this - but he talks specifically from a search web spam perspective, I suspect the Adsense team have different rules and probably can (and do) "play Whack a mole" when appropriate.
- Matt_Cutts 16y ago
  That was probably me. We have two sides to the webspam team at Google: engineering and manual. We definitely prefer to write algorithms so that we avoid dealing with individual websites--the idea is that you strive to fix the root cause of an issue, not to tackle specific sites. However, if we see a website that violates our guidelines and that gets past the algorithms, we are willing to take manual action. Where possible, we use the output of the manual team not only to reduce spam itself, but to train the next iteration of algorithms. For example, one of the big issues in blackhat spam this past year was illegally hacked sites. Our algorithms weren't doing the best job on hacked sites, so the manual team kept an eye out for hacked sites to remove them (and often to alert the website owners that they'd been hacked). The data generated by the manual team helped us build and deploy multiple new algorithms to detect hacked sites, leading to a 90% reduction in the number of hacked sites showing up in Google's search results in the past few months. That decrease in hacked spam in turn frees up the manual team to tackle the next bleeding-edge technique the spammers use. I suspect every major search engine uses similar approaches: try to stop the majority of spam with algorithms, but be willing to take action in the mean time while engineers work to improve the algorithms.
  - 16y ago
Matt_Cutts 16y ago
I don't think that was the issue. The fact is that if you've got dozens of websites, each of which has lists of domains/IPs like http://www.mattcutts.com/images/verypopularwebsites-com.png http://www.mattcutts.com/images/verypopularwebsites-com.png , that is the sort of thing that users complain about and don't want showing up when they do a search. Especially if sites have autogenerated boilerplate content for each one of those links. I mean, if you're auto-generating a page that has this text: "Elcorillord.com Common misspellings and typos: Elcroillord.com, Elcorilolrd.com, Elcori.lord.com, Elcofillord.com, www.elcorillord.com, lEcorillord.com, Elcotillord.com, Elcorillor.com, Elcoeillord.com, Elcori,lord.com, Wlcorillord.com, Elcorjllord.com, Elcorillord.coom, Elocrillord.com, Elcor8llord.com, Elvorillord.com, Elcorillprd.com, Elcorillord.cim, Elcorillorf.com, Elcorilloed.com, Elorillord.com, Elckrillord.com, Elcoriplord.com, Elcorillord.ckm, Elcorillord.cm, Elcorillord.ccom, Epcorillord.com, Elcoril;ord.com, Elcoirllord.com, Elcoriillord.com, Elforillord.com, 3lcorillord.com, Elcorollord.com, Elcorillordd.com, Elcorill0rd.com, Elcorillord/com, Elcoriolord.com, Ekcorillord.com, Elcorillord.xom, Elcorillord.co, Elcorilord.com, Elcoillord.com, 4lcorillord.com, Elcoriloord.com, Elcorillorr.com, Eldorillord.com, Elcorillord..com, Elcorrillord.com, http://www.elcorillord.com http://www.elcorillord.com, El orillord.com, E.corillord.com, Elcorillord. om, Elcorilllrd.com, Elcorillrod.com, Elcoriklord.com, Elcorillorrd.com, Elcorillordcom, Elcorillkrd.com, Elcorillord.om, Elcorlilord.com, Elco4illord.com, Elcorillrd.com, Elcprillord.com, Elcodillord.com, Elcorillordc.om, Ecorillord.com, Elcoorillord.com, Slcorillord.com, Elcorillorx.com, Elcorill9rd.com, Elcorilpord.com, Elcorillord.cpm, Elcorillord.fom, Elco5illord.com, Elc9rillord.com, Elcorillird.com, Elcirillord.com, Elcorillord.clm, Elcorillors.com, Elcorillord.vom, Elcorullord.com, Elcorillord.comm, Elcorillord.c9m, Eocorillord.com, Elcorilloord.com, Elcourillourd.com, E,corillord.com, Elcorkllord.com, Elcorillodr.com, Elcorillodd.com, Elcorillord,com, Elcorillotd.com, Elcorillod.com, Elcorillor.dcom, Elcor9llord.com, Elc0rillord.com, Elcoril,ord.com, Elcorilllord.com, Elcorillo5d.com, EElcorillord.com, Elxorillord.com, E;corillord.com, Elcori;lord.com, Elcorllord.com, Elccorillord.com, Elcrillord.com, Elcoril.ord.com, Elcorilkord.com, Elcorillord.cmo, Ellcorillord.com, Eclorillord.com, Elcorillo4d.com, Rlcorillord.com, wwwelcorillord.com, Elclrillord.com, ElcorillordLcom, Dlcorillord.com, Elcorillofd.com, Elcorillore.com, Elcorillord;com, lcorillord.com, Elcorillorc.com, Elcorillord.c0m, Elcorillord.dom, Elcorillord.ocm." Surely you have to see where many people would consider that either keyword stuffing, gibberish or typo spam.

xpose2000 16y ago

Maybe this can help . . . I have a few websites that automatically make new posts. As of 10/14, they all show 0 pages indexed in Google. Previously they would get a few thousand visitors per day. I guess Google feels as though they violate their terms and removed them. It seems to me it was a manual removal. I received no emails in webmaster tools about the removal.

Matt_Cutts 16y ago
"I have a few websites that automatically make new posts." Making a bunch of autogenerated sites has its risks. For example, if you were just taking a bunch of MP3 names or Hot Trends queries and then scraping twitter for mentions of those phrases and slapping that all up on a website with scripts, that tends to cruft up our index with autogenerated content that users complain about and that violates our quality guidelines. Likewise, if all you were doing was scraping Twitter for phrases like sad or heartbroken or heartless and throwing that scraped Twitter content up on a webpage with a script, users would also complain about that autogenerated content and it would violate our guidelines. Would that be helpful insight?
- epi0Bauqu 16y ago
  As a concrete case to discuss, what about something like http://poeet.com/ http://poeet.com/ I made this over a weekend. And the people whose poetry is being captured love it. But it is auto-generated in the sense you're talking about. It actually went down for a bit and I got a bunch of complaints, enough that I got it back up relatively quickly.
  - Rogan 16y ago
    Theres a clear difference between your site http://poeet.com http://poeet.com and a clean cut case of auto generated spam. Your site is actually quite creative where it is aggregating content from a twitter hashtag and indexing short poems that may otherwise go unnoticed, you are also showing the users original tweet and @user and not manipulating anything. The original poster was likely scraping content, not providing citation and for the means of having the duplicate content wrapped around ads.