2 ms·
No. It prevents pages from being crawled, but references to uncrawled pages can still be shown. See http://www.mattcutts.com/blog/robots-txt-remove-url/ http://
by Matt_Cutts 16y ago
No. It prevents pages from being crawled, but references to uncrawled pages can still be shown. See http://www.mattcutts.com/blog/robots-txt-remove-url/ http://www.mattcutts.com/blog/robots-txt-remove-url/ for how it works.