3 ms·His robots.txt explicitly allows bots including LLM bots to scrape his siteby janmo 6mo agoHis robots.txt explicitly allows bots including LLM bots to scrape his sitejefposk 6mo agoThe LLM scraper bots ignore robots.txtxigoi 6mo agoPermission to scrape is not permission to DDoS.