2 ms·
They need to stop or else make their crawlers easy to identify and block. However I have no faith that AI companies will play by the rules. They already cost p
by agentultra 2y ago
They need to stop or else make their crawlers easy to identify and block. However I have no faith that AI companies will play by the rules.
They already cost people time, money, and their mental health by using adversarial tactics to evade blocking and ignoring robots.txt
https://drewdevault.com/2025/03/17/2025-03-17-Stop-externalizing-your-costs-on-me.html https://drewdevault.com/2025/03/17/2025-03-17-Stop-externali...