4 ms·Google ignores robots.txt and botnets residential addresses to crawl anyway? (LLM startups already do this.)by archagon 5mo agoGoogle ignores robots.txt and botnets residential addresses to crawl anyway? (LLM startups already do this.)