5 ms·

Because, for reasons known to the engineers toiling at Google, and not me, these sites will show in the search results, and the actual StackOverflow answer will

Because, for reasons known to the engineers toiling at Google, and not me, these sites will show in the search results, and the actual StackOverflow answer will not be shown at all.

geekfactor 16y ago
That's been my experience as well. These sites have been a royal PITA and I'm glad to see this extension.
Encosia 16y ago
SEO is non-deterministic enough that I hesitate to even speculate, but the clean markup on sites like eFreedom is probably a lot of their advantage. Stack Overflow buries its content in quite a bit of presentational markup. Everything else being equal, it makes sense that the tighter, more focused page should rank better than the larger, more convoluted page containing the same content.
earl 16y ago
I don't know what the deal is, but it's weird on SO's part. If the issue is, as a sibling comment says, presentation html / js confusing G's crawler, then you email your account manager at Google and it will almost certainly be completely kosher to show the G scraper different material than is displayed, as long as the difference is true to the intent of the searcher. So you can present the G crawler just questions and answers with no javascript and no additional markup.
- codinghorror 16y ago
  Good idea, but I have no idea who our account manager is at Google. Do we even have an 'account manager'? what do they manage exactly, we don't do any adsense. I'll try to find out. but I'm kinda skeptical this approach (serve googlebot one set of content, everyone else something else) is something that Google would actually want to encourage at scale on any site.
  - Aaronontheweb 16y ago
    Obvious question, but have you reached out to Matt Cutts yet?
    - codinghorror 16y ago
      I'll try, but Matt is a busy guy.
      - Aaronontheweb 16y ago
        He is, but he's exceptionally helpful and generous with his time. Has my utmost respect.
    - Matt_Cutts 16y ago
      I'm looking into this a bit now. It's not a clearcut situation, in that SO has a license that allows copying. Jeff, drop me an email and let's talk about it more (I followed you on Twitter so you can DM me). The specific example at http://webmasters.stackexchange.com/questions/5385/page-appears-indexed-in-google-but-not-findable-for-any-search-terms http://webmasters.stackexchange.com/questions/5385/page-appe... is ranking SO at #1 for me now, but if you send some more examples I'm happy to chat with people in the search quality team about this.
  - slaven 16y ago
    Cloaking (the name for faking out the google-bot) is NOT cool by Google. There are ways to accomplish the same thing without cloaking, for example making sure content loads first, keeping markup and presentation away from the main HTML, etc.
    - sysop073 16y ago
      Are you sure? I thought they were generally ok with it as long as it was the same information the user sees, just presented differently to help Google index it better
      - 16y ago
codinghorror 16y ago
yes, it's very very bizarre and it drives me crazy. We've tried 3 or 4 different things to fix it and nothing seems to take. Note that our attribution terms do require a link back primarily for this reason, and even the sites who attribute back to us totally legally still have this problem. It truly does feel like a Google bug, honestly. See related discussion at http://webmasters.stackexchange.com/questions/5385/page-appears-indexed-in-google-but-not-findable-for-any-search-terms http://webmasters.stackexchange.com/questions/5385/page-appe... . I'm all ears if anyone knows of a way to fix this.
- dminor 16y ago
  Do what you can to get links to your questions with the question title as anchor text. 1) Is it possible to require the question title as anchor text in the creative commons link that points to the question (and not a nofollow link obviously)? 2) Make a widget for embedding a question and chosen answer on a web page, along with a link back to the question, a la google maps. 3) Make a wordpress plugin that takes a stack overflow link and turns it into the question and chosen answer, with a link back to stack overflow.
- brc 16y ago
  Why not just yank the creative commons licence and replace with one that explicitly does not allow scraping? That would be the fastest way in my book. I've never worked out why SO allows it in the first place. Is it just to appear open and web 2.0-y, or is there a business reason. It's a proper business now, users are loyal, cancel the licence. I can't think of a single person who would say 'oh, but I much preferred to read those spam sites'.
  - Encosia 16y ago
    Yes, absolutely. As someone who has contributed a fair amount of content to SO, I would wholeheartedly support modifying the CC license on the content I've contributed. I much prefer the idea of that to the idea of allowing my answers to help build dens 'o spam like eFreedom. Now that I think of it, my answers going straight-to-spam is a nontrivial detriment to my contributing content to SO.

Because, for reasons known to the engineers toiling at Google, and not me, these sites will show in the search results, and the actual StackOverflow answer will not be shown at all.

geekfactor 16y ago

That's been my experience as well. These sites have been a royal PITA and I'm glad to see this extension.

Encosia 16y ago

SEO is non-deterministic enough that I hesitate to even speculate, but the clean markup on sites like eFreedom is probably a lot of their advantage. Stack Overflow buries its content in quite a bit of presentational markup. Everything else being equal, it makes sense that the tighter, more focused page should rank better than the larger, more convoluted page containing the same content.

earl 16y ago

I don't know what the deal is, but it's weird on SO's part. If the issue is, as a sibling comment says, presentation html / js confusing G's crawler, then you email your account manager at Google and it will almost certainly be completely kosher to show the G scraper different material than is displayed, as long as the difference is true to the intent of the searcher. So you can present the G crawler just questions and answers with no javascript and no additional markup.

codinghorror 16y ago
Good idea, but I have no idea who our account manager is at Google. Do we even have an 'account manager'? what do they manage exactly, we don't do any adsense. I'll try to find out. but I'm kinda skeptical this approach (serve googlebot one set of content, everyone else something else) is something that Google would actually want to encourage at scale on any site.
- Aaronontheweb 16y ago
  Obvious question, but have you reached out to Matt Cutts yet?
  - codinghorror 16y ago
    I'll try, but Matt is a busy guy.
    - Aaronontheweb 16y ago
      He is, but he's exceptionally helpful and generous with his time. Has my utmost respect.
  - Matt_Cutts 16y ago
    I'm looking into this a bit now. It's not a clearcut situation, in that SO has a license that allows copying. Jeff, drop me an email and let's talk about it more (I followed you on Twitter so you can DM me). The specific example at http://webmasters.stackexchange.com/questions/5385/page-appears-indexed-in-google-but-not-findable-for-any-search-terms http://webmasters.stackexchange.com/questions/5385/page-appe... is ranking SO at #1 for me now, but if you send some more examples I'm happy to chat with people in the search quality team about this.
- slaven 16y ago
  Cloaking (the name for faking out the google-bot) is NOT cool by Google. There are ways to accomplish the same thing without cloaking, for example making sure content loads first, keeping markup and presentation away from the main HTML, etc.
  - sysop073 16y ago
    Are you sure? I thought they were generally ok with it as long as it was the same information the user sees, just presented differently to help Google index it better
    - 16y ago

codinghorror 16y ago

yes, it's very very bizarre and it drives me crazy. We've tried 3 or 4 different things to fix it and nothing seems to take. Note that our attribution terms do require a link back primarily for this reason, and even the sites who attribute back to us totally legally still have this problem. It truly does feel like a Google bug, honestly. See related discussion at http://webmasters.stackexchange.com/questions/5385/page-appears-indexed-in-google-but-not-findable-for-any-search-terms http://webmasters.stackexchange.com/questions/5385/page-appe... . I'm all ears if anyone knows of a way to fix this.

dminor 16y ago
Do what you can to get links to your questions with the question title as anchor text. 1) Is it possible to require the question title as anchor text in the creative commons link that points to the question (and not a nofollow link obviously)? 2) Make a widget for embedding a question and chosen answer on a web page, along with a link back to the question, a la google maps. 3) Make a wordpress plugin that takes a stack overflow link and turns it into the question and chosen answer, with a link back to stack overflow.
brc 16y ago
Why not just yank the creative commons licence and replace with one that explicitly does not allow scraping? That would be the fastest way in my book. I've never worked out why SO allows it in the first place. Is it just to appear open and web 2.0-y, or is there a business reason. It's a proper business now, users are loyal, cancel the licence. I can't think of a single person who would say 'oh, but I much preferred to read those spam sites'.
- Encosia 16y ago
  Yes, absolutely. As someone who has contributed a fair amount of content to SO, I would wholeheartedly support modifying the CC license on the content I've contributed. I much prefer the idea of that to the idea of allowing my answers to help build dens 'o spam like eFreedom. Now that I think of it, my answers going straight-to-spam is a nontrivial detriment to my contributing content to SO.