3 ms·

Google started with something, probably by scanning the Yahoo directory and starting from there. If they just pointed Googlebot to one site they would never get

by gscott 16y ago

Google started with something, probably by scanning the Yahoo directory and starting from there. If they just pointed Googlebot to one site they would never get the whole web, they had to start from some source with a lot of sites that they could then branch out from.

lazugod 16y ago
And depending on the state of Yahoo's (or Lycos' or Altavista's) robots.txt at the time, that may be defendable.