2 ms·
Since robots.txt alone is not enough to keep a site away from being indexed by Google, I have previously used to include <meta name="robots" content="noind
by codetrotter 5y ago
Since robots.txt alone is not enough to keep a site away from being indexed by Google, I have previously used to include
<meta name="robots" content="noindex">
instead when I don't want a temporary site to appear in search results.
However, it can be a drag to make sure you have that header tag included while not including it in a production site.
Today I learned that there's a HTTP header that we can use instead to get the same effect. And in my case this simplifies things a ton. So I am posting this link about it here for others that are unaware about the X-Robots-Tag HTTP header.