3 ms·
That's not what nofollow is for anyway. Just update your robots.txt file.
by pkulak 16y ago
That's not what nofollow is for anyway. Just update your robots.txt file.
- ck2 16y agoPutting it in robots.txt is like advertising to bad bots exactly where to go. On some sites what I do is make a fake virtual directory and just ban the directory in robots.txt while prefixing the links with the fake directory path. Oh and I swear google sometimes follows stuff in robots.txt, it just doesn't index it. Then there's the "alternate" user agents they use like pretending to be IE (from a real Google IP).
- jrockway 16y agoThe solution is to password-protect the parts of your site that you don't want bots to see. Javascript obfuscation is silly. Robots.txt is advisory. Passwords are simple and failsafe.
- tomjen3 16y agoPutting it in robots.txt is like advertising to bad bots exactly where to go. Exactly. Right to your special page that earns a 24 hour ip ban. It doesn't take that long to code and should keep any bot away.
- pkulak 16y agoWho said anything about using robot.txt for security? I just use it for keeping Google from indexing my print pages, for example. Anything that I don't want the world to see, can't be seen without authorization.