3 ms·
yacy has support for additional crawlers. But I was thinking of something more radical ... let the indexes be peer-to-peer with an IPFS storage back-end, but b
by smoyer 7y ago
yacy has support for additional crawlers.
But I was thinking of something more radical ... let the indexes be peer-to-peer with an IPFS storage back-end, but build at least the generic crawler as a browser plugin that also performs searches against the index. For "normal" web pages, each visit results in a reindexing. Of course you have to solve the chicken and egg problem of pages that aren't visited because nobody has indexed them so you don't find them in search.
- iovrthoughtthis 7y agoAllow people to submit their sites to the index. People who want to be found have an incentive to do the work.
- Polylactic_acid 7y agoPeople already do this. You can submit your sitemap to bing and google. But I doubt you will get anyone to bother with a small player or p2p service.