> A website could block Google’s crawlers in its robots.txt file — OpenAI, fresh from scraping every website in the world to build ChatGPT, just allowed its cra

> A website could block Google’s crawlers in its robots.txt file — OpenAI, fresh from scraping every website in the world to build ChatGPT, just allowed its crawler to be blocked in this way — but blocking Google’s crawlers means deindexing your site from search, which is, bluntly, suicidal. And someone still doesn't think Google is constantly abusing their monopoly position ? This is basically textbook example, and it's not the first time they do something like this.