4 ms·
A "Disallow:" (without anything after it) is saying allow all robots access to the entire server. (If you need something beyond my word on the topic, there's a
by ecaron 14y ago
A "Disallow:" (without anything after it) is saying allow all robots access to the entire server. (If you need something beyond my word on the topic, there's a thorough explanation at http://www.robotstxt.org/robotstxt.html http://www.robotstxt.org/robotstxt.html)