2 ms·

robots can ignore your /robots.txt.

• Karunamon 9y ago
  Googlebot, and every other search engine robot of note. Putting a page up on the public internet includes the risk that it may be visited.