> I don’t understand why people are ok with Google scraping their site (when it is called indexing), fine with users scraping their site (when it is called RSS

> I don’t understand why people are ok with Google scraping their site (when it is called indexing), fine with users scraping their site (when it is called RSS reading), but suddenly not ok with AI startups scraping their site. I’m pretty open‐minded about AI, and have no visceral objection to AI scraping in theory. However… the rise of AI scrapers is the first time in twenty years of running tiny websites that my servers—all my servers, consistently—hit CPU and bandwidth usage levels far above baseline, and even brush against my hosts’ monthly limits. When I check server logs, the traffic is largely bots repeatedly trawling infinite variations of version control history pages. Googlebot never crawled these enough to become a problem; Internet Archive never crawled these enough to become a problem. But the current scrapers are, and literally everyone I talk to who hosts their own VCS webservers is having the same problem right now. Many are choosing to reduce the problem by injecting Cloudflare or Anubis, which I as a user hate, and refuse to do myself—but what alternative can I propose to them? People running these servers are just users of Gitea and CGit and whatnot, they’re not capable of rewriting the underlying tools to reduce the CPU usage of diff generation.