3 ms·
You ignore the robots.txt file, you will be fed garbage....All is fair in War and Love...
by belter 2y ago
You ignore the robots.txt file, you will be fed garbage....All is fair in War and Love...
- spiderfarmer 2y agoI want to do this, but it will cost me bandwidth and CPU cycles. What is the most efficient way to pollute these nasty AI profiteers?
- user432678 2y agoHow do you determine they ignore it? Honeypots?