3 ms·
I'm working in an architecture for a semi-decentralized, some-of-the-web crawler and search engine to try to address some of the problems with Google. Basicall
by alangibson 6y ago
I'm working in an architecture for a semi-decentralized, some-of-the-web crawler and search engine to try to address some of the problems with Google.
Basically affinity groups maintain lists of links (sort of like social bookmarking) that are the crawl root urls. Crawling comes from Common Crawl plus updates for recency. Groups can pool resources to run their own indexes and receive a share of the profits from text display ads.
- lhnz 6y agoIs there anything that I can follow? I'm very interested in this -- I was considering making something similar myself.
- alangibson 6y agoI'll give you a shout when I publish the first draft. From your profile you seem like a pretty easy guy to reach. :)