3 ms·
“Next page” link goes to the same page with entirely different set of addresses. So, it’s practically infinite for a crawler.
by sedatk 2y ago
“Next page” link goes to the same page with entirely different set of addresses. So, it’s practically infinite for a crawler.
- pdimitar 2y agoOhhhh I see, thanks for clarifying! Brain fart. Indeed, I'd love to use your stuff.