3 ms·
Maybe I'm missing something, but in http://stackoverflow.com/robots.txt http://stackoverflow.com/robots.txt I see: Sitemap: /sitemap.xml But http://stacko
by staunch 16y ago
Maybe I'm missing something, but in http://stackoverflow.com/robots.txt http://stackoverflow.com/robots.txt I see:
Sitemap: /sitemap.xml
But http://stackoverflow.com/sitemap.xml http://stackoverflow.com/sitemap.xml is a 404.
Perhaps he submitted a different sitemap URL to Google directly. Maybe that's the problem though?
All the examples on http://www.sitemaps.org/protocol.php#submit_robots http://www.sitemaps.org/protocol.php#submit_robots use a fully qualified URL not an absolute URL relative to the domain. I've always used that.
- carson 16y agoThey are probably blocking everyone except for Google from grabbing the file.
- jonknee 16y agoPerhaps Google checks this... They are typically not cool with cloaking. If you want to hide your site map, just make it a non-guessable URL and submit it to Google the Webmaster CP.
- deleted 16y ago[deleted]
- nostromo 16y agoIt still 404s with a GoogleBot user agent.
- staunch 16y agoYou can do a PTR/A lookup to check if an IP is legit Googlebot or not though. It's possible they're doing that.
- codinghorror 16y agocorrect, we are doing this
- kj12345 16y agoYeah here's Google's cached version: http://webcache.googleusercontent.com/search?q=cache:TN1KGcAqBQwJ:stackoverflow.com/sitemap.xml+http://stackoverflow.com/sitemap.xml&cd=1&hl=en&ct=clnk&gl=us http://webcache.googleusercontent.com/search?q=cache:TN1KGcA...