It's also worth pointing out that only the likes of Google and Yahoo will even look at robots.txt. Some other crawler may walk in there, then create pages that

It's also worth pointing out that only the likes of Google and Yahoo will even look at robots.txt. Some other crawler may walk in there, then create pages that link your files, and then Google could link those pages, and it's game over.