3 ms·
Have not found the crawl rates changed for sites where everything is well linked together. Then again, these have been just small sites with a few million pages
by jkaljundi 9y ago
Have not found the crawl rates changed for sites where everything is well linked together. Then again, these have been just small sites with a few million pages.
- franze 9y agomeasurability crawling != indexing with a segmented sitemap (i.e.: one per pagetype, per namespace, ..) you get important communicated to indexed ratios you otherwise do not get. if crawling and indexing works, it works but if it doesn't you should know it (see it in google search console) and debug it (narrow it down via segmented sitemaps).
- andrei_says_ 9y agoCurious if you have examples of the types of sitemaps you mentioned: segmentation, namespacing, one per pagetype etc. and the reasoning behind the variations.
- franze 9y agoi.e.: tourradar.com/robots.txt