3 ms·
Just consider how you click around HN versus how your crawler would behave if you wanted to crawl every page of HN starting from the homepage.
by hombre_fatal 2y ago
Just consider how you click around HN versus how your crawler would behave if you wanted to crawl every page of HN starting from the homepage.