3 ms·
I don't think you actually want to do that. Blocking in robots.txt will prevent Google from crawling the URL - not from indexing it. You actually want them to c
by troels 12y ago
I don't think you actually want to do that. Blocking in robots.txt will prevent Google from crawling the URL - not from indexing it. You actually want them to crawl the URL and then respond with a 404 or 410.
If there are inbound links pointing to that URL, you should disavow them in GWT.
- eli 12y agoa "noindex" robots meta tag would work too