3 ms·
Look at github.com/robots.txt and you'll see there is no bingbot listed there. I'm not sure if this affects the result in your query, but it might well just be
by simonz05 14y ago
Look at github.com/robots.txt and you'll see there is no bingbot listed there. I'm not sure if this affects the result in your query, but it might well just be a case of bing respecting a robots.txt file.
- chucknthem 14y agoActually, it has msnbot listed, and according to this http://www.bing.com/community/site_blogs/b/webmaster/archive/2010/06/28/bing-crawler-bingbot-on-the-horizon.aspx http://www.bing.com/community/site_blogs/b/webmaster/archive... The bingbot will respect msnbot permissions when it doesn't see bingbot, unless of course github has explicitly blocked unrecognized user agents too.