Brewster Kahle (Archive.org) made a small fortune by selling WAIS (Wide Area Information Server), which made each machine a search engine, and used a protocol t

Brewster Kahle (Archive.org) made a small fortune by selling WAIS (Wide Area Information Server), which made each machine a search engine, and used a protocol to request from all engines. Developed in the 80s. Unfortunately, the Web has gone down this road of having centralised search engines which somehow know their way through a maze. It would be better to contact several sites in turn (NYT, FT, Archive, etc) and pull information. If you have to pay, you go through the credit card paywall. Then you don't have to worry about NYT's dark kilobytes. Why aren't we using it? I guess that's what happens when you sell something like that to AOL. ;-) I should also point out that Google could have a stream for the NYT, where the NYT feeds all its stories to Google on creation, and Google doesn't enable cache for the stuff people pay for. But for all I know, that's already being done. But service-push-to-server is better than Google's pull-service-to-server.