2 ms·
> Websites are, and always have been, allowed to decline being crawled and indexed by google at any time. One can argue about the appropriateness of it. I am
by JadeNB 2y ago
> Websites are, and always have been, allowed to decline being crawled and indexed by google at any time.
One can argue about the appropriateness of it. I am taking no position here, just saying it's not new to AI summaries.
- semking 2y agoGoogle performed one of the biggest web scrapping experiences in the history of the internet! And there's something completely new with AI summaries. Google went beyond crawling and indexing. Google decided to generate content, based on what was crawled/indexed. Instead of listing traditional search results that lead to websites, Google now steals content and engagement. The only way to get out of AI summaries? Blocking Googlebot. Which means disappearing from Google Search. Google leaves you no choice.