5 ms·

Pagination Tunnels – An Experiment in Crawlability and Click Depth

43 points by adamcarson 9y ago

rjmunro 9y ago
In some circumstances a "random" button might be a nice thing to have, rather than a predictable mid-point, as it would encourage real users to sample different things from deep in the long tail. The crawler could also use this to find jumping off points. You could also then leave out the "last" option, and therefore you wouldn't need a "first" option to look symmetrical for human users.
dzhiurgis 9y ago
How about putting all titles in one “index” page so I (and a bot) can easily ctrl+F skim thru them?
- dpcx 9y ago
  That works until you have (like the example) 20k titles. That's a performance hit to the (assuming) database to send all of that data at once, to the web server for having to parse all of those results and generate the HTML, to the client browser for having to parse all of that HTML and render it. It's possible, of course, that it's better for the user. It just doesn't happen to be that way very often, from my own anecdotal experience.
  - deleted 9y ago
    [deleted]
  - dzhiurgis 9y ago
    Is it tho? It's probably half the size of any JS framework or library out there. Yes, it can be slow if you wrap each line in tons nested HTML, inefficient CSS and tons of JS handlers, but it doesn't have to.
lioeters 9y ago
The visualization of "crawlability" is great, the graphs were intuitive to see how some methods are more effective than others in reducing long sequential branches of pages. After finishing the article, I had a feeling that none of the pagination methods were "perfect" with a good balance of crawlability and user-friendliness. It made me think, for sites with a large number of pages, there needs to be more than just paginataion for navigating them - for example, navigating/sorting by categories would probably produce a different graph, closer to the ideal level of branches and depths.
tveita 9y ago
For crawlability you may be better of with an explicit site map. Pagination is very important and your focus should be on UX, not bots. https://en.wikipedia.org/wiki/Sitemaps https://en.wikipedia.org/wiki/Sitemaps
spyder 9y ago
Google's recommendation: - Do nothing. - Specify a View All page. - Use rel="next" and rel="prev" links https://support.google.com/webmasters/answer/1663744?hl=en https://support.google.com/webmasters/answer/1663744?hl=en + sitemap: https://support.google.com/webmasters/answer/183668?hl=en https://support.google.com/webmasters/answer/183668?hl=en

43 points by adamcarson 9y ago

In some circumstances a "random" button might be a nice thing to have, rather than a predictable mid-point, as it would encourage real users to sample different things from deep in the long tail. The crawler could also use this to find jumping off points. You could also then leave out the "last" option, and therefore you wouldn't need a "first" option to look symmetrical for human users.

dzhiurgis 9y ago

How about putting all titles in one “index” page so I (and a bot) can easily ctrl+F skim thru them?

dpcx 9y ago
That works until you have (like the example) 20k titles. That's a performance hit to the (assuming) database to send all of that data at once, to the web server for having to parse all of those results and generate the HTML, to the client browser for having to parse all of that HTML and render it. It's possible, of course, that it's better for the user. It just doesn't happen to be that way very often, from my own anecdotal experience.
- deleted 9y ago
  [deleted]
- dzhiurgis 9y ago
  Is it tho? It's probably half the size of any JS framework or library out there. Yes, it can be slow if you wrap each line in tons nested HTML, inefficient CSS and tons of JS handlers, but it doesn't have to.

lioeters 9y ago

The visualization of "crawlability" is great, the graphs were intuitive to see how some methods are more effective than others in reducing long sequential branches of pages. After finishing the article, I had a feeling that none of the pagination methods were "perfect" with a good balance of crawlability and user-friendliness. It made me think, for sites with a large number of pages, there needs to be more than just paginataion for navigating them - for example, navigating/sorting by categories would probably produce a different graph, closer to the ideal level of branches and depths.

tveita 9y ago

For crawlability you may be better of with an explicit site map. Pagination is very important and your focus should be on UX, not bots. https://en.wikipedia.org/wiki/Sitemaps https://en.wikipedia.org/wiki/Sitemaps

spyder 9y ago

Google's recommendation: - Do nothing. - Specify a View All page. - Use rel="next" and rel="prev" links https://support.google.com/webmasters/answer/1663744?hl=en https://support.google.com/webmasters/answer/1663744?hl=en + sitemap: https://support.google.com/webmasters/answer/183668?hl=en https://support.google.com/webmasters/answer/183668?hl=en