2 ms·
Would adding robots.txt help? I cannot blanket ban the page as it needs to display to the user with 0 or more results and my understanding is that whilst robot
by scalesolved 8y ago
Would adding robots.txt help? I cannot blanket ban the page as it needs to display to the user with 0 or more results and my understanding is that whilst robots.txt would take precedence for index settings as I have it defined in the header for that page it ought to achieve the same.
- wejick 8y agoI think adding rule for /search* to your robot is easier way to block google bot
- dougunplugged 8y agoDon't do this until Google sees your new robots meta directives (or canonical tag) on these pages, that way they will drop from the index. Then you can add this to your robots.txt to prevent it being crawled again.