4 ms·
History. It’s been documented that Google no longer tries to index everything on the Web and is allowing older material to disappear. Tim Bray has a blog post a
by lkrubner 5y ago
History. It’s been documented that Google no longer tries to index everything on the Web and is allowing older material to disappear. Tim Bray has a blog post about this. A full history would be useful, also the ability to run a search with the Page Rank of a given day. For instance, what site was the most linked node on July 19 of 2011?
- marginalia_nu 5y agoTo be fair, you can't index everything on the web, due to dynamic page generation and wildcard subdomains, web is effectively boundless. You can have an URL http://integer.example.com/N http://integer.example.com/N only that contains a link to http://integer.example.com/N+1 http://integer.example.com/N+1 All search engines need to make concessions. The tricky part is identifying the sorts of content that humans want to find.