5 ms·
Seems like they could easily fix this with robots.txt or something similar, I really doubt it's oversight on their part either. Any ideas why they're doing thi
by Igglyboo 12y ago
Seems like they could easily fix this with robots.txt or something similar, I really doubt it's oversight on their part either.
Any ideas why they're doing this?
- andybalholm 12y agoI assume that some site has hostgator-related links with two slashes instead of one. Due to the two slashes, the GoogleBot doesn't realize that it's indexing their own results pages.
- guan 12y agoThey probably want to index some pages on google.com, but not search results. To exclude search results, someone wrote something to exclude URLs that start with /search, and forgot that //search works the same way.
- spindritf 12y agoThey disallow /search User-agent: * Disallow: /search but maybe //search slipped thorough?
- johnmu 12y agoThis is just a bug, sorry.