3 ms·

Wouldn't a proper robots.txt rule fix this?

• mkjones 14y ago
  A robots.txt file disallowing crawling on the sites that display the contents of user email would help fix this. However, as some of the discussion below points out, I don't believe that disallowing crawling of these URLs in our robots.txt would keep them from the index if a search engine finds reference to them elsewhere; I think it simply keeps them from being crawled.