4 ms·
Given that the scraping doesn't do any rate limiting and pisses on robots.txt, yes it is abuse
by PunchyHamster 7mo ago
Given that the scraping doesn't do any rate limiting and pisses on robots.txt, yes it is abuse
- singpolyma3 7mo agoIs there any evidence OpenAI has been ignoring robots.txt for scraping purposes? AFAIK the main sources of that traffic are still unknown.