3 ms·
In my experience, sitemaps aren't for URL discovery, but more for URL prioritization. Google will pretty much crawl you whole site whether or not you include a
by keltex 16y ago
In my experience, sitemaps aren't for URL discovery, but more for URL prioritization. Google will pretty much crawl you whole site whether or not you include a sitemap. Where the sitemap becomes important is whether google includes in the supplementary index or the main index. With a sitemap, you can specify a priority for each page and basically hint to google that some pages are more important than others.
http://sitemaps.org/protocol.php#prioritydef http://sitemaps.org/protocol.php#prioritydef
- codinghorror 16y agothe consensus is that sitemap.xml works best for rapid discovery of new content -- it's not very good for discovery of deep content the crawler can't get to because of the aforementioned paradox (that is, if Google can't see you linking to your own page, it is disinclined to let the sitemap link matter)