26 ms·
Robots.txt is a suicide note. http://www.archiveteam.org/index.php?title=Robots.txt http://www.archiveteam.org/index.php?title=Robots.txt My personal server r
by sbierwagen 14y ago
Robots.txt is a suicide note.
http://www.archiveteam.org/index.php?title=Robots.txt http://www.archiveteam.org/index.php?title=Robots.txt
My personal server returns a 410 to robots.txt requests.
- tomjen3 14y agoI have no clue as to why the author of that shit is as angry as he is, but I have zero interest in his opinion until such time as he learns to show me the issues, and not just blindly assume that anybody who is not as enlightened as him is a blind idiot.
- sbierwagen 14y agoOkay.