5 ms·
It's indicating the root of the site. Documentation for further reading at http://www.robotstxt.org/robotstxt.html http://www.robotstxt.org/robotstxt.html
by michaelbuckbee 16y ago
It's indicating the root of the site. Documentation for further reading at http://www.robotstxt.org/robotstxt.html http://www.robotstxt.org/robotstxt.html
- joe_the_user 16y agoIndeed, It seemed self-explanatory here. I was being perhaps a bit sarcastic...