3 ms·

Then the webserver admin should work out how to stop google crawling his site and stop his site responding as any DNS name.

• hartator 8y ago
  Default Nginx and Apache configs serve all IPs. If you don't share your server IP, it's reasonable to think your content won't be indexed by Google.
  • Qwertie 8y ago
    Google seems to be very good at finding new never seen before content. If you don't want it public then put some kind of authentication on it.
    • dx034 8y ago
      But Google also always respects robots.txt. If you don't want your content to appear just put a deny all for all domains. You can easily set that up in nginx by returning static content, you don't even need to create a file in the right directories.
• human 8y ago
  Unless directory listing is turned on, I can’t see how Google would be able to index these PDFs given that no external links point to them.
  • fjsolwmv 8y ago
    And presumably it doesn't guess PATHs because Larry Page isn't in jail.