3 ms·
Breadth-first search means crawling all of the links on the page and adding all of the links on the child pages to the queue at once rather than drilling down o
by frisco 15y ago
Breadth-first search means crawling all of the links on the page and adding all of the links on the child pages to the queue at once rather than drilling down on one link-path first before moving to the other links on the first page. There's probably some highly non-obvious reason for crawling by url length.
- xyzzyz 15y agoThe thing is, the deeper you are, the longer are the urls, so if you do a breadth first search, you are more likely to visit short urls first.