2 ms·Robots.txt files are directions to site-crawling robots, not to humans. It's certainly not an ad.by LorenPechtel 5y agoRobots.txt files are directions to site-crawling robots, not to humans. It's certainly not an ad.