4 ms·Well this scraper honours robots.txt so I'm sure most AI crawlers will find it useless.by andrepd 7mo agoWell this scraper honours robots.txt so I'm sure most AI crawlers will find it useless.