3 ms·I read robots.txt, but I'm not a massive corporation.by c3534l 7y agoI read robots.txt, but I'm not a massive corporation.