3 ms·
I'm interested in trying out this technique for a site we built but your comments about Googlebot and the associated downside with long-tail pages gives me paus
by adriand 15y ago
I'm interested in trying out this technique for a site we built but your comments about Googlebot and the associated downside with long-tail pages gives me pause. Do you have any suggestions on how to evaluate whether or not a technique like this would bring advantages, or is it better to simply give it a try and see how it works out?
In our case, the site features real estate listings, and at any given time there are about 50,000 listings. Each listing has its own page. There are numerous other pages, some of which are just index pages (lists of listings), a couple of search pages, etc. But the majority of pages are the single listing view pages, and then following that, the lists of listings. Let's call it around 60,000 pages or so.
The total number of pageviews per day (not counting search engine crawlers) is less than the total number of pages, but I would be willing to be that some individual listing pages are accessed at a much higher rate than others (namely, the newest ones). The listings themselves are updated on a daily basis, so the cache stores would get invalidated at least once a day.
In this situation, how do I determine the appropriate caching strategy?
- cstejerean 15y agoIf you have enough cache capacity to keep every individual listing in cache then you could simply write a script that periodically hits the detail page for each listing to make sure the latest version of each listing is always available in cache.