3 ms·
Websites are, and always have been, allowed to decline being crawled and indexed by google at any time.
by fallingknife 2y ago
Websites are, and always have been, allowed to decline being crawled and indexed by google at any time.
- JadeNB 2y ago> Websites are, and always have been, allowed to decline being crawled and indexed by google at any time. One can argue about the appropriateness of it. I am taking no position here, just saying it's not new to AI summaries.
- semking 2y agoGoogle performed one of the biggest web scrapping experiences in the history of the internet! And there's something completely new with AI summaries. Google went beyond crawling and indexing. Google decided to generate content, based on what was crawled/indexed. Instead of listing traditional search results that lead to websites, Google now steals content and engagement. The only way to get out of AI summaries? Blocking Googlebot. Which means disappearing from Google Search. Google leaves you no choice.
- tivert 2y ago> Websites are, and always have been, allowed to decline being crawled and indexed by google at any time. That may be so, but they never had a real choice. Google has also lost an antitrust case that found its search engine was a monopoly, which proves it. https://apnews.com/article/google-antitrust-ad-tech-virginia-opening-7a19f525287f782609a5316b1fdb08f0 https://apnews.com/article/google-antitrust-ad-tech-virginia...