4 ms·

When we find out about these cases, we ask them to conform to our creative commons license agreement by linking back to stackoverflow.com, in hopes that ultimat

by spolsky 16y ago

When we find out about these cases, we ask them to conform to our creative commons license agreement by linking back to stackoverflow.com, in hopes that ultimately this will teach Google that they are non-canonical and rank them lower than us.

brc 16y ago
That might be so, but right now there's about three of them and they clog up the search results with essentially the same answer but lower quality for the reasons mentioned above. So far I've not seen any that outrank stackoverflow, but it's only a matter of time before one does some blackhat and jumps for certain questions. You say that they link back, but in the cases I've seen, they link back with a slightly different Url form to the one you use. For example : http://stackoverflow.com/questions/1037925/recreate-stack-trace-with-line-numbers-from-user-bug-report-in-net http://stackoverflow.com/questions/1037925/recreate-stack-tr... this question appears on scraping sites (not going to name them for fear of feeding them traffic), but they link back with this Url: http://stackoverflow.com/questions/1037925 http://stackoverflow.com/questions/1037925 - which doesn't resolve back to the longer version. I know there's a canonical link on it (for the longer url) but I'm wondering if this strategy is working or not. Maybe it's time to start rejecting some bots crawling the site, or at least to slow them right down. It's a problem for Google and for SO, so someone from both places should probably be talking about it. Because in the end, SO starts looking spammy because the others are spammy and scrape the content, which is exactly the opposite case of how things should be. The spam sites are just cloning SO and shoving crappy ads everywhere, it's not high quality content. When you ask a question, you want 5 different answers from 5 different site, not the same answer spammed 5 times back at you.

by spolsky 16y ago

brc 16y ago

That might be so, but right now there's about three of them and they clog up the search results with essentially the same answer but lower quality for the reasons mentioned above. So far I've not seen any that outrank stackoverflow, but it's only a matter of time before one does some blackhat and jumps for certain questions. You say that they link back, but in the cases I've seen, they link back with a slightly different Url form to the one you use. For example : http://stackoverflow.com/questions/1037925/recreate-stack-trace-with-line-numbers-from-user-bug-report-in-net http://stackoverflow.com/questions/1037925/recreate-stack-tr... this question appears on scraping sites (not going to name them for fear of feeding them traffic), but they link back with this Url: http://stackoverflow.com/questions/1037925 http://stackoverflow.com/questions/1037925 - which doesn't resolve back to the longer version. I know there's a canonical link on it (for the longer url) but I'm wondering if this strategy is working or not. Maybe it's time to start rejecting some bots crawling the site, or at least to slow them right down. It's a problem for Google and for SO, so someone from both places should probably be talking about it. Because in the end, SO starts looking spammy because the others are spammy and scrape the content, which is exactly the opposite case of how things should be. The spam sites are just cloning SO and shoving crappy ads everywhere, it's not high quality content. When you ask a question, you want 5 different answers from 5 different site, not the same answer spammed 5 times back at you.