3 ms·
robots.txt can certainly keep google away from your search box, but it won't defend you against the lower-case probes, the extra letter probes, the delete-a-cha
by greglindahl 8y ago
robots.txt can certainly keep google away from your search box, but it won't defend you against the lower-case probes, the extra letter probes, the delete-a-character probes, etc.
- dwild 8y agoYet he didn't try it. That's seriously weird that he doesn't mentions trying that (he updated his post to say they doesn't support regex, yet does support wildcard, which is all what's needed for a search). He removed the search before using a simple robots.txt...