3 ms·
Are they actually blocking Google somehow? No robots.txt, no 403 when using a Googlebot user agent to fetch the site page...
by ipsin 7y ago
Are they actually blocking Google somehow?
No robots.txt, no 403 when using a Googlebot user agent to fetch the site page...
- sieabahlpark 7y agoGooglebot uses the Google ip range
- einr 7y agoYes. <meta name="googlebot" content="noindex"> Googlebot looks for this meta tag.