4 ms·

You can unfortunately see this with DuckDuckGo: people are clearly targeting more than just Google. Python results, for example, are so infested by SEO-speciali

by cge 5y ago

You can unfortunately see this with DuckDuckGo: people are clearly targeting more than just Google. Python results, for example, are so infested by SEO-specialist spam that searching for standard library functions will return spam above the actual standard library documentation, particularly for more popular functions. Searching for "python datetime.now" or "python json.loads" will both return spam above documentation. This problem heavily impacts anything 'data scientist' spammers see as important as well; it's actually worse on DDG than Google. What's frustrating is that these often seem to be a handful of domains, like 'geeksforgeeks' and 'towardsdatascience'; for a while, of course, there was also the "gitmemory" spammer who seemed to be able to push out Github results on both DDG and Google. Yet I think Google removed reporting and blacklisting domains from searches long ago, and I think DDG never had it, leaving the only option for removing them client-side scripts and extensions that work poorly. Likewise, no search engine appears to be manually blacklisting them. Yet as you point out, if one did, then the spammers would probably just move to using many domains, which would probably be worse.

Nextgrid 5y ago
Domain age & quantity/breadth of content should be taken into account in ranking. A fresh domain that suddenly has a ton of content should be viewed with suspicion and downranked as it's likely a spammer (copying GitHub/Stackoverflow/official docs). Legitimate sites that are starting out shouldn't be affected as they are unlikely to have a ton of content from the start. Of course, this isn't perfect, but it should take care of the majority of spam copycat sites.
- drivebycomment 5y ago
  You think all the existing search engineers haven't thought of using the domain age for the signal and tried it yet? And if you did, you think SEO people wouldn't figure it out and take advantage of it? This cycle of adversarial game on this particular signal has already gone through the full cycle - https://www.searchenginejournal.com/ranking-factors/domain-age/ https://www.searchenginejournal.com/ranking-factors/domain-a... And what makes you think search engines don't take quantity and breadth of content ? And, why do you think SEO won't be able to take advantage of that ? Most people commenting and lamenting about search quality need to understand how much of search quality is an adversarial game, that neither side (search engine vs SEO people who want to manipulate the result in their favor) can win decisively forever. And generally it's not for the lack of trying the quality hasn't improved as much - it's just some areas it's more difficult to make progress due to more money spent on making it difficult for the search engines. What I am most disappointed about Google and search engines in general isn't so much about ranking simplistic things like generic recipe or product reviews and stuff like that - most recipes for general items are all similar, and product review is inherently commercial that reviews always have been biases (short of using Consumer Reports, almost all magazine review articles are biases and influenced by manufacturers). What I am most disappointed is the lack of improvements in UI and retrieval/recall. Interactive UI that can narrow down search space iteratively and interactively so that I can describe what I want in more precise way is lacking. Where is the virtual librarian that I can talk to to narrow down what I want in a step by step way ? Instead of a single search query being the only input, why shouldn't search engine ask for clarifications and let users more precisely describe what they want ? Human languages and human brain just don't produce a single phrase to describe what we want - it takes sentences and questioning back and forth to determine what we want, and we often need more input to realize and describe what we actually want. And all search engines have failed to provide such an interface. And my bet is that whoever can crack that would beat all other search engines, since users can give better, higher quality input for the search to find.
codeisawesome 5y ago
I have to wonder just what is the justification for Google to not let users decide what sites they never want to see results from? I'm sure it can't be storage or compute limitations. This one feature alone would make so many people (me included) happier to use the engine.

by cge 5y ago

Nextgrid 5y ago

Domain age & quantity/breadth of content should be taken into account in ranking. A fresh domain that suddenly has a ton of content should be viewed with suspicion and downranked as it's likely a spammer (copying GitHub/Stackoverflow/official docs). Legitimate sites that are starting out shouldn't be affected as they are unlikely to have a ton of content from the start. Of course, this isn't perfect, but it should take care of the majority of spam copycat sites.

drivebycomment 5y ago
You think all the existing search engineers haven't thought of using the domain age for the signal and tried it yet? And if you did, you think SEO people wouldn't figure it out and take advantage of it? This cycle of adversarial game on this particular signal has already gone through the full cycle - https://www.searchenginejournal.com/ranking-factors/domain-age/ https://www.searchenginejournal.com/ranking-factors/domain-a... And what makes you think search engines don't take quantity and breadth of content ? And, why do you think SEO won't be able to take advantage of that ? Most people commenting and lamenting about search quality need to understand how much of search quality is an adversarial game, that neither side (search engine vs SEO people who want to manipulate the result in their favor) can win decisively forever. And generally it's not for the lack of trying the quality hasn't improved as much - it's just some areas it's more difficult to make progress due to more money spent on making it difficult for the search engines. What I am most disappointed about Google and search engines in general isn't so much about ranking simplistic things like generic recipe or product reviews and stuff like that - most recipes for general items are all similar, and product review is inherently commercial that reviews always have been biases (short of using Consumer Reports, almost all magazine review articles are biases and influenced by manufacturers). What I am most disappointed is the lack of improvements in UI and retrieval/recall. Interactive UI that can narrow down search space iteratively and interactively so that I can describe what I want in more precise way is lacking. Where is the virtual librarian that I can talk to to narrow down what I want in a step by step way ? Instead of a single search query being the only input, why shouldn't search engine ask for clarifications and let users more precisely describe what they want ? Human languages and human brain just don't produce a single phrase to describe what we want - it takes sentences and questioning back and forth to determine what we want, and we often need more input to realize and describe what we actually want. And all search engines have failed to provide such an interface. And my bet is that whoever can crack that would beat all other search engines, since users can give better, higher quality input for the search to find.

codeisawesome 5y ago

I have to wonder just what is the justification for Google to not let users decide what sites they never want to see results from? I'm sure it can't be storage or compute limitations. This one feature alone would make so many people (me included) happier to use the engine.