3 ms·
It's silly in that the author means, by search engine, a clone of Google's architecture: "A crawler gets the Web pages off of that pesky Web and onto your beau
by corentin 19y ago
It's silly in that the author means, by search engine, a clone of Google's architecture:
"A crawler gets the Web pages off of that pesky Web and onto your beautiful disks. You'll need lots of disks.
Then you need to index these pages--say which page has which words. This will tell you that Janet Jackson was found on the www.superbowl.com page. Usually, indexing happens locally on the disks where your crawler dumped these Web pages. Hey, why move them?"
It's just one possibility. An alternative (not necessarily the best one, though) could be a personal spider that starts somewhere and is intelligent enough to find what you're looking for without crawling the entire web. Or a peer-to-peer search engine, to share the infrastructure between the users. Or a search engine that only crawl in the subset of the web you're interested in (i.e. you manage a list of domains per user profile).
- papersmith 19y agoA peer-to-peer search engine sounds good. The difficult part is convincing users to donate the resources for your cause.
- deleted 19y ago[deleted]
- euccastro 19y agoHow do Skype and bittorrent achieve that?