3 ms·
Using the X-Robots-Tag HTTP header
- codetrotter 5y agoSince robots.txt alone is not enough to keep a site away from being indexed by Google, I have previously used to include <meta name="robots" content="noindex"> instead when I don't want a temporary site to appear in search results. However, it can be a drag to make sure you have that header tag included while not including it in a production site. Today I learned that there's a HTTP header that we can use instead to get the same effect. And in my case this simplifies things a ton. So I am posting this link about it here for others that are unaware about the X-Robots-Tag HTTP header.