3 ms·

As a site owner, how does one opt out of this, since it obviously ignores robots.txt?

• chadwebscraper 8mo ago
  Shoot me your site and I can blacklist it
  • xnx 8mo ago
    That's what robots.txt is for.