3 ms·

What about the blocking google bot by their IPs, also combined with user-agent wouldn't that stop the crawlers Google crawlers IPs https://www.lifewire.com/wha

• benatkin 5y ago
  That will stop the crawlers but you could still show up in the search results, because of other web pages. From GP: > If other pages point to your page with descriptive text, Google could still index the URL without visiting the page