3 ms·
Your idea doesn't go far enough. What good are a couple of stale indexes of Wikipedia and Stackoverflow going to be? Instead, I am wondering why there isnt a f
by visarga 9y ago
Your idea doesn't go far enough. What good are a couple of stale indexes of Wikipedia and Stackoverflow going to be?
Instead, I am wondering why there isnt a federated open source search engine. It would be a cloud of nodes, each node spidering and indexing a small hash bucket of urls. With a million such nodes we could have a live updated, distributed search engine to replace Google. We could run millions of queries without paying someone - we'd pay back by serving as a node, just like in BitTorrent. With all the interest in privacy, I wanted to see more discussion of replacing Google with an open, non-censured and private protocol for search.
- finnh 9y agoWho would host the index? You can't contact one million peers to run a search, so fanout-on-read doesn't work. If the index is also distributed (by search term?), then the search+indexing nodes will need to fanout-on-write, which is less latency-sensitive but still pretty onerous.
- tastythrowaway 9y agoThis is a brilliant idea. Let's work on that. Seriously. Do you want to work on this together?
- boramalper 9y ago> Instead, I am wondering why there isnt a federated open source search engine. Not sure if federated by they call themselves decentralised: https://yacy.net/en/index.html https://yacy.net/en/index.html
- runeb 9y agoIn the same vein, what good is an index of the web going to be? Googles value isn't their index, its how they translate your search terms into good results from that index. Anyone can index the web, not everyone can make sense of it from human questions.
- visarga 9y agoThere has been a lot of research in search engines since AltaVista collapsed and Google won. Much of it is in public domain.
- deleted 9y ago[deleted]