3 ms·
No, but AFAIK they pulled some shenanigans with "bundling" Gemini scraping and search engine scraping. Almost everybody wants to appear in search, so disallowi
by miki123211 10mo ago
No, but AFAIK they pulled some shenanigans with "bundling" Gemini scraping and search engine scraping.
Almost everybody wants to appear in search, so disallowing the entirety of Google is far more costly than E.G. disallowing Openai, who even differentiates between content scraped for training and content accessed to respond to a user request.
- inkysigma 10mo agoWhile there isn't a way to differentiate between scraping for training data and content accessed in response to a user request, I think you can block Googlebot-extended to block training access.