3 ms·
Maybe because that's how they are sorted in the hashtable/database they use to queue urls? Or maybe because they want to index the shortest pages first, so that
by ignifero 15y ago
Maybe because that's how they are sorted in the hashtable/database they use to queue urls? Or maybe because they want to index the shortest pages first, so that they are processed before any duplicates with longer urls (i.e. get /articles/ before /articles/index.php)
- georgemcbay 15y agoI suspect you're on the right track with your first guess. Most people posting here are looking for some sort of deep meaning in this when IMO it is more likely just due to a localized side-effect of doing something such as storing the urls in a trie-like structure and then iterating over it breadth-first.