3 ms·
I ran into this the other day. Google will disregard robots.txt and will instead honour the noindex meta tag; BUT if you have a robots.txt that disallows the fi
by mxxx 6y ago
I ran into this the other day. Google will disregard robots.txt and will instead honour the noindex meta tag; BUT if you have a robots.txt that disallows the file with the noindex meta tag then it won’t honour either. Thanks google.