2 ms·You could maybe still only follow robots.txt rules for Googlebot.by wilg 2y agoYou could maybe still only follow robots.txt rules for Googlebot.deleted 2y ago[deleted]