5 ms·
A few questions that come to mind off the bat: Do you have a sitemap.xml file? Does it include accurate last updated dates or suggest realistic crawl schedules
by Minor49er 4y ago
A few questions that come to mind off the bat:
Do you have a sitemap.xml file? Does it include accurate last updated dates or suggest realistic crawl schedules (eg: only index static pages every month, but frequently updated pages every day)?
Have you run your site through an SEO checker (or at least some of the ignored pages) to make sure that they are accessible and not being blocked/ignored by a crawler? Any server responses or canonical links that may signal the wrong thing to a bot?
Do you have any deep links to your more overlooked pages on the homepage (or at least on your more popular pages)?
Do you have backlinks in the wild that point to the pages that you want to have crawled? Are they marked as nofollow? Do you post links to these pages on any social media accounts?
- brntsllvn 4y agoWe have a sitemap index and several dozen sitemaps based on how we segment our pages (and each sitemap has a limit of 50k). We use SEMRush and that's partly what's driving my questions: Their guidance is largely irrelevant for us: We're a niche search engine. We have several thousand backlinks and I wish I had a better strategy here, but our content is broadly unshareable (versus something like Stack Overflow, which is very shareable).
- Minor49er 4y agoDo you also use something like Google Analytics or Bing Webmaster Tools on your site?
- brntsllvn 4y agoDefinitely. Daily tools include Google analytics, Google search console, mouseflow and SEMRush. Bing Webmaster is surprisingly good.
- Minor49er 4y agoHave you tried Google's Index Coverage report? It may reveal why those pages are not yet crawled, if they are even known about https://support.google.com/webmasters/answer/7440203 https://support.google.com/webmasters/answer/7440203
- brntsllvn 4y agoAbsolutely. But the reasons are vague at best. The key reason seems to boil down to limited crawl budget simply because we don't have reputation. In which case, we need to rethink if our business has the ability to generate shareable content. It might not.
- Minor49er 4y agoWhat is the business, if you don't mind me asking?
- brntsllvn 4y agoAggregated used car listings. CarGraph.com
- RobSm 4y agoPeople don't care about such content and so does Google.
- brntsllvn 4y agoFirstly thank you for your time and effort on our behalf. We're grateful for your generosity. Next, I totally agree we should probably have an internal version of listing info so users stay on our platform versus bouncing and then having to come back. Good insight. I'll talk with the team about richer data as it furthers our SEO goals. I think the idea here would be that better engagement will improve our ranking? Is that correct? The tricky part is getting impressions and clicks but this could help.