6 ms·

1. http://bit.ly/gTADhE http://bit.ly/gTADhE 2. Click Install, close page 3. Open each of the links below in a new tab, click block on the first result 4. Wi

by dsl 16y ago

1. http://bit.ly/gTADhE http://bit.ly/gTADhE 2. Click Install, close page 3. Open each of the links below in a new tab, click block on the first result 4. Win. http://www.google.com/search?q=Mahalo http://www.google.com/search?q=Mahalo http://www.google.com/search?q=ehow http://www.google.com/search?q=ehow http://www.google.com/search?q=experts-exchange http://www.google.com/search?q=experts-exchange http://www.google.com/search?q=livestrong.com http://www.google.com/search?q=livestrong.com http://www.google.com/search?q=answerbag http://www.google.com/search?q=answerbag http://www.google.com/search?q=bills.com http://www.google.com/search?q=bills.com http://www.google.com/search?q=chacha.com http://www.google.com/search?q=chacha.com http://www.google.com/search?q=associated+content http://www.google.com/search?q=associated+content http://www.google.com/search?q=efreedom http://www.google.com/search?q=efreedom http://www.google.com/search?q=questionhub http://www.google.com/search?q=questionhub http://www.google.com/search?q=squidoo.com http://www.google.com/search?q=squidoo.com http://www.google.com/search?q=about.com http://www.google.com/search?q=about.com http://www.google.com/search?q=yellowpages.com http://www.google.com/search?q=yellowpages.com ---- Edits: fixed formatting, added suggestions This method is fine. The actual data sent to Google when you block a domain does not contain the search query (or the referrer). This is what gets sent when you block a domain: http://www.google.com/gen_204?atyp=i&oi=site_blocker&ct=addToBlocklist&ei=[CSRF-cookie]&cad=mattcutts.com and unblock: http://www.google.com/gen_204?atyp=i&oi=site_blocker&ct=deleteFromBlocklist&ei=undefined&cad=mattcutts.com (Interestingly the CSRF token is broken when unblocking.)

jonursenbach 16y ago
Don't forget about http://www.google.com/search?q=about.com http://www.google.com/search?q=about.com
- lurker17 16y ago
  What's wrong with about.com? I generally find about.com has high-quality content giving an overview of a topic, much more in-depth than mahalo or eHow.
  - seabee 16y ago
    I can see where he's coming from: it has information on a lot of different topics and it's full of ads, therefore it's an evil content farm. Just like how someone that weighs the same as a duck must be made of wood, and therefore is a witch.
    - jonursenbach 16y ago
      They may not do it now, but they used to be as bad as Associated Content.
billybob 16y ago
Doesn't this tell Google "Mahalo is a spam result if I search for 'Mahalo'"? Seems like you should do this when you get one of these and you wanted Stackoverflow.
- jmillikin 16y ago
  I don't think it records which search first resulted in the block, only that a block exists
- deleted 16y ago
  [deleted]
davepeck 16y ago
I'd put 'osdir' on that list...
- dminor 16y ago
  Very first site I blocked. (Would have been eFreedom, but I haven't seen them in the results for awhile. Thanks Google!)
- comforteagle 16y ago
  [I run osdir.com] Just to be clear we're not scraping content nor SEOing to death. We've been running our mailing list archive for almost 8 years. To quote Matt_Cutts somewhere else in this thread: 'because USENET/mailing lists don't always have a centralized/canonical location on the web, which makes dupe content more of a potential issue. It's not the usual "Website X copied my website" scenario.' If you dislike us that's cool, but just know we're not showing in your SERPs because of scraping & SEO trickery. A lot of people, maybe not you, really are finding their answer there. There's a lot of solid answers in inboxes.
TREYisRAD 16y ago
I used to write for eHow and the like, and I fully endorse blocking them. Thanks for the list!
- Natsu 16y ago
  On the subject of eHow, here's another reason to consider blocking them, Google for "no text to spin" (include quotes). That's an error message from bots that are spamming random wikis. I wrote more about it here: http://news.ycombinator.com/item?id=2215993 http://news.ycombinator.com/item?id=2215993
  - patio11 16y ago
    To be fair, there isn't a scintilla of evidence that they're involved with that. Some black hat is just using them as feedstock for content spinning. Happens to Wiki and my blog all the time.
    - Natsu 16y ago
      Very possible. I gave more mention of that possibility in the writeup. There are some weird things, too. I see what might be links between mystery numbers in the spam pages and parts of their URL, and I find it odd that they're the only site listed in those error messages, and there aren't a lot of those errors. Which may indicate that they're not deliberately blocking the spambots. But I will concede that it's weak evidence so far at best. On another note, who is using your blog as feedstock? Can you give a URL to a spun copy of your stuff?
rbancroft 16y ago
about.com actually has some interesting information from time to time. Secondly, the first link that appears for me when you search for experts-exchange is wikipedia.
- btipling 16y ago
  About's atheism site is pretty good. I wont block about.com they have good information.
ohkine 16y ago
The hiding functionality doesn't interest me, this feature has been available for ages and ages in GreaseMonkey scripts and the like. But sending the information to Google, with the hope that they will eventually use it for something, interests me greatly. Unfortunately, i don't and can't use Chrome. If all this does though is send a URL request when a site is blocked, it should be extremely trivial (for someone who is smarter and has more free time than me, probably) to re-implement this in a Firefox-compatible form, right?
benologist 16y ago
You forgot bytes.com and bigresource.com, holy fuck I hate them.
tristanperry 16y ago
A good list :) A couple more: http://www.google.com/search?q=wisegeek.com http://www.google.com/search?q=wisegeek.com http://www.google.com/search?q=bizrate.com http://www.google.com/search?q=bizrate.com ---- My reasons for suggesting that these are blocked: "What are Some Animals Commonly Mistaken for Dinosaurs?" (http://www.wisegeek.com/what-are-some-animals-commonly-mistaken-for-dinosaurs.htm http://www.wisegeek.com/what-are-some-animals-commonly-mista...) "How Do I Care for a Rose Breasted Cockatoo?" (http://www.wisegeek.com/how-do-i-care-for-a-rose-breasted-cockatoo.htm http://www.wisegeek.com/how-do-i-care-for-a-rose-breasted-co...) "Why Does Alaska Have Really Long Days During Some Times of the Year and Really Short Days During Other Times?" (http://www.wisegeek.com/why-does-alaska-have-really-long-days-during-some-times-of-the-year-and-really-short-days-during-other-times.htm http://www.wisegeek.com/why-does-alaska-have-really-long-day...) - should be fairly self explanatory. Plus it only took be ~45 seconds to find these examples of journalistic <sarcasm>brilliance</sarcasm> (there's MANY more such examples). Plus they are - IMO - breaking AdSense guidelines since the Google link unit ads are EXACTLY the same (in color/style and :hover behaviour) as the in-content navigation links, which is a direct breach of AdSense policy. You'd think the AdSense team would be aware of such a big violation.. -- And as for BizRate, it's a 99% autogenerated site with little (if any) unique content. Plus most pages have at least one keyword stuffing area (such as http://www.bizrate.com/mens-shoes/dress-shoes/ http://www.bizrate.com/mens-shoes/dress-shoes/ - see the 'related searches' bit on the left, for a quick example).
- EEvangelist 16y ago
  Just a remark - Experts-Exchange user generated content. If something is copied from elsewhere and not attributed, it will be attributed or deleted and the poster told off.
  - 9ec4c12949a4f3 16y ago
    EE tends to have poor quality that goes nowhere, and mods come by and make people accept answers anyway because it makes their business model look good.
Kadrith 16y ago
For the extension developers: How about a way to import lists of sites like this?
elsewhen 16y ago
If these sites are as bad as you claim, hacker news users will easily be able to spot them and nuke them from their results... I have a sneaking suspicion that those of you passing around lists of sites have ulterior motives.
- benologist 16y ago
  My ulterior motive is I'm sick of seeing crap sites like bigresource.com and bytes.com. All they are is scraped/stolen content wrapped around AdSense, and I don't even know how many times I've clicked on them without thinking. bigresource is the most annoying, one time those assholes put a fricking CAPTCHA in place so you had to prove you were human to get to the source link they STOLE the content from.
9ec4c12949a4f3 16y ago
I thought the CSRF was part of it, I really did, here's the extent to which I've been trying to hack this plugin to let me add sites en-mass. I modified the source code of common.js in the plugin after getting frustrated when not getting results in a less hackish-way. http://pastebin.com/VbCnk5Et http://pastebin.com/VbCnk5Et I now goto google search, search for qwerty, click 'block en.wikipedia.com', and even though I've hard-coded the CAD= value to .asdf.com, my block filter still ends up blocking wikipedia, not asdf.com. I was confused, so I tossed in a URL alert, which confirmed the URL modification had taken place. Then I opened wireshark and watched the HTTP stream to google, which also contained a request and response. http://pastebin.com/raw.php?i=pwVmiLM3 http://pastebin.com/raw.php?i=pwVmiLM3 Basically, something else is going on here. The CSRF may be a UUID.
redstar 16y ago
Ask.com and Answers.com are bigger culprits than all of your list put together. google this: site:answers.com "can you answer this question" 32,200,000 of their 103,000,000 answers pages DO NOT HAVE AN ANSWER! They are spamming google’s users to get them to build answer.com’s database of answers. (and surrounding the non-content with heavy ads!) For the pages that do have an answer, answers.com has gone the way of answers.yahoo.com. (I guess you forgot answers.yahoo.com too) And, BTW, ask.com is just regurgitating answers.com!! google this: site:ask.com answers.com or just surf around ask.com to see that they just regurgitate ALL of the content farm results, which makes ask.com a "content farm" farm your list was a good start, but you missed the biggest (by far) offenders on the internet.

by dsl 16y ago

jonursenbach 16y ago

Don't forget about http://www.google.com/search?q=about.com http://www.google.com/search?q=about.com

lurker17 16y ago
What's wrong with about.com? I generally find about.com has high-quality content giving an overview of a topic, much more in-depth than mahalo or eHow.
- seabee 16y ago
  I can see where he's coming from: it has information on a lot of different topics and it's full of ads, therefore it's an evil content farm. Just like how someone that weighs the same as a duck must be made of wood, and therefore is a witch.
  - jonursenbach 16y ago
    They may not do it now, but they used to be as bad as Associated Content.

billybob 16y ago

Doesn't this tell Google "Mahalo is a spam result if I search for 'Mahalo'"? Seems like you should do this when you get one of these and you wanted Stackoverflow.

jmillikin 16y ago
I don't think it records which search first resulted in the block, only that a block exists
deleted 16y ago
[deleted]

davepeck 16y ago

I'd put 'osdir' on that list...

dminor 16y ago
Very first site I blocked. (Would have been eFreedom, but I haven't seen them in the results for awhile. Thanks Google!)
comforteagle 16y ago
[I run osdir.com] Just to be clear we're not scraping content nor SEOing to death. We've been running our mailing list archive for almost 8 years. To quote Matt_Cutts somewhere else in this thread: 'because USENET/mailing lists don't always have a centralized/canonical location on the web, which makes dupe content more of a potential issue. It's not the usual "Website X copied my website" scenario.' If you dislike us that's cool, but just know we're not showing in your SERPs because of scraping & SEO trickery. A lot of people, maybe not you, really are finding their answer there. There's a lot of solid answers in inboxes.

TREYisRAD 16y ago

I used to write for eHow and the like, and I fully endorse blocking them. Thanks for the list!

Natsu 16y ago
On the subject of eHow, here's another reason to consider blocking them, Google for "no text to spin" (include quotes). That's an error message from bots that are spamming random wikis. I wrote more about it here: http://news.ycombinator.com/item?id=2215993 http://news.ycombinator.com/item?id=2215993
- patio11 16y ago
  To be fair, there isn't a scintilla of evidence that they're involved with that. Some black hat is just using them as feedstock for content spinning. Happens to Wiki and my blog all the time.
  - Natsu 16y ago
    Very possible. I gave more mention of that possibility in the writeup. There are some weird things, too. I see what might be links between mystery numbers in the spam pages and parts of their URL, and I find it odd that they're the only site listed in those error messages, and there aren't a lot of those errors. Which may indicate that they're not deliberately blocking the spambots. But I will concede that it's weak evidence so far at best. On another note, who is using your blog as feedstock? Can you give a URL to a spun copy of your stuff?

rbancroft 16y ago

about.com actually has some interesting information from time to time. Secondly, the first link that appears for me when you search for experts-exchange is wikipedia.

btipling 16y ago
About's atheism site is pretty good. I wont block about.com they have good information.

ohkine 16y ago

The hiding functionality doesn't interest me, this feature has been available for ages and ages in GreaseMonkey scripts and the like. But sending the information to Google, with the hope that they will eventually use it for something, interests me greatly. Unfortunately, i don't and can't use Chrome. If all this does though is send a URL request when a site is blocked, it should be extremely trivial (for someone who is smarter and has more free time than me, probably) to re-implement this in a Firefox-compatible form, right?

benologist 16y ago

You forgot bytes.com and bigresource.com, holy fuck I hate them.

tristanperry 16y ago

A good list :) A couple more: http://www.google.com/search?q=wisegeek.com http://www.google.com/search?q=wisegeek.com http://www.google.com/search?q=bizrate.com http://www.google.com/search?q=bizrate.com ---- My reasons for suggesting that these are blocked: "What are Some Animals Commonly Mistaken for Dinosaurs?" (http://www.wisegeek.com/what-are-some-animals-commonly-mistaken-for-dinosaurs.htm http://www.wisegeek.com/what-are-some-animals-commonly-mista...) "How Do I Care for a Rose Breasted Cockatoo?" (http://www.wisegeek.com/how-do-i-care-for-a-rose-breasted-cockatoo.htm http://www.wisegeek.com/how-do-i-care-for-a-rose-breasted-co...) "Why Does Alaska Have Really Long Days During Some Times of the Year and Really Short Days During Other Times?" (http://www.wisegeek.com/why-does-alaska-have-really-long-days-during-some-times-of-the-year-and-really-short-days-during-other-times.htm http://www.wisegeek.com/why-does-alaska-have-really-long-day...) - should be fairly self explanatory. Plus it only took be ~45 seconds to find these examples of journalistic <sarcasm>brilliance</sarcasm> (there's MANY more such examples). Plus they are - IMO - breaking AdSense guidelines since the Google link unit ads are EXACTLY the same (in color/style and :hover behaviour) as the in-content navigation links, which is a direct breach of AdSense policy. You'd think the AdSense team would be aware of such a big violation.. -- And as for BizRate, it's a 99% autogenerated site with little (if any) unique content. Plus most pages have at least one keyword stuffing area (such as http://www.bizrate.com/mens-shoes/dress-shoes/ http://www.bizrate.com/mens-shoes/dress-shoes/ - see the 'related searches' bit on the left, for a quick example).

EEvangelist 16y ago
Just a remark - Experts-Exchange user generated content. If something is copied from elsewhere and not attributed, it will be attributed or deleted and the poster told off.
- 9ec4c12949a4f3 16y ago
  EE tends to have poor quality that goes nowhere, and mods come by and make people accept answers anyway because it makes their business model look good.

Kadrith 16y ago

For the extension developers: How about a way to import lists of sites like this?

elsewhen 16y ago

If these sites are as bad as you claim, hacker news users will easily be able to spot them and nuke them from their results... I have a sneaking suspicion that those of you passing around lists of sites have ulterior motives.

benologist 16y ago
My ulterior motive is I'm sick of seeing crap sites like bigresource.com and bytes.com. All they are is scraped/stolen content wrapped around AdSense, and I don't even know how many times I've clicked on them without thinking. bigresource is the most annoying, one time those assholes put a fricking CAPTCHA in place so you had to prove you were human to get to the source link they STOLE the content from.

9ec4c12949a4f3 16y ago

I thought the CSRF was part of it, I really did, here's the extent to which I've been trying to hack this plugin to let me add sites en-mass. I modified the source code of common.js in the plugin after getting frustrated when not getting results in a less hackish-way. http://pastebin.com/VbCnk5Et http://pastebin.com/VbCnk5Et I now goto google search, search for qwerty, click 'block en.wikipedia.com', and even though I've hard-coded the CAD= value to .asdf.com, my block filter still ends up blocking wikipedia, not asdf.com. I was confused, so I tossed in a URL alert, which confirmed the URL modification had taken place. Then I opened wireshark and watched the HTTP stream to google, which also contained a request and response. http://pastebin.com/raw.php?i=pwVmiLM3 http://pastebin.com/raw.php?i=pwVmiLM3 Basically, something else is going on here. The CSRF may be a UUID.

redstar 16y ago

Ask.com and Answers.com are bigger culprits than all of your list put together. google this: site:answers.com "can you answer this question" 32,200,000 of their 103,000,000 answers pages DO NOT HAVE AN ANSWER! They are spamming google’s users to get them to build answer.com’s database of answers. (and surrounding the non-content with heavy ads!) For the pages that do have an answer, answers.com has gone the way of answers.yahoo.com. (I guess you forgot answers.yahoo.com too) And, BTW, ask.com is just regurgitating answers.com!! google this: site:ask.com answers.com or just surf around ask.com to see that they just regurgitate ALL of the content farm results, which makes ask.com a "content farm" farm your list was a good start, but you missed the biggest (by far) offenders on the internet.