3 ms·
Wikipedia's robots.txt is quite verbose: http://en.wikipedia.org/robots.txt http://en.wikipedia.org/robots.txt
by oevi 13y ago
Wikipedia's robots.txt is quite verbose:
http://en.wikipedia.org/robots.txt http://en.wikipedia.org/robots.txt
- iso8859-1 13y agoBecause it's part of the Wiki: https://en.wikipedia.org/w/index.php?title=MediaWiki:Robots.txt&action=history https://en.wikipedia.org/w/index.php?title=MediaWiki:Robots....
- MattHeard 13y agoMy favourite part: > # Folks get annoyed when XfD discussions end up the number 1 google hit for > # their name.