4 ms·
Isn't Perplexity the AI company that has been accused of ignoring robots.txt and requests not to data harvest?
by trentnix 1y ago
Isn't Perplexity the AI company that has been accused of ignoring robots.txt and requests not to data harvest?
- Hamuko 1y agoIt is. They do go further than just ignoring robots.txt too. https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/ https://blog.cloudflare.com/perplexity-is-using-stealth-unde...