> robots.txt wasn't created for security but it can have security implications if you publish a list of Disallow paths with the intention of hiding sensitive co

> robots.txt wasn't created for security but it can have security implications if you publish a list of Disallow paths with the intention of hiding sensitive content Using robots.txt to secure your server from bots is the equivalent of attempting to secure your house from robbery by planting a sign that says "please,don't rob my house". Surprisingly it may works from time to time, by if you're into attempting security by wishful thinking maybe don't be too surprised when it fails about as much as security by chance.