4 ms·
There is an interesting article about AI tarpits that addresses a similar issue: https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and
by EPendragon 1y ago
There is an interesting article about AI tarpits that addresses a similar issue: https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and-trick-ai-scrapers-that-ignore-robots-txt/ https://arstechnica.com/tech-policy/2025/01/ai-haters-build-...
The argument is basically to have them scrape your website indefinitely wasting their resources for the bots that decide to ignore your robots.txt (or any bot if you desire)