3 ms·
Just define a robot.txt file instructing the crawler to not index your website. See: http://www.robotstxt.org/ http://www.robotstxt.org/
by mileycyrusXOXO 9y ago
Just define a robot.txt file instructing the crawler to not index your website.
See: http://www.robotstxt.org/ http://www.robotstxt.org/
- seanwilson 9y agoThis is a common misconception: https://support.google.com/webmasters/answer/7424835?hl=en#h17 https://support.google.com/webmasters/answer/7424835?hl=en#h... > robots.txt Disallow does not guarantee that a page will not appear in results: Google may still decide, based on external information such as incoming links, that it is relevant. If you wish to explicitly block a page from being indexed, you should instead use the noindex robots meta tag or X-Robots-Tag HTTP header. If it's something like a staging site, you'd be better putting it behind a login screen to prevent mistakes anyway.