4 ms·

I wonder if it would be possible to build a decentralized search engine with a decentralized crawl on top of it.

• hosh 6y ago
  I've been wondering the same thing too! Maybe if there is a way to share indexes, or some way to put out a call for searches through the DHT, and then the requestor aggregates the results. A couple examples: Web crawler -- seeded through a curated or personal bookmark list. Knowledge base -- the permaculture community have amassed a knowledge base on growing plants: USDA hardiness zones, heat zones, humidity, soil requirements, companion planting relationships, etc. That's something I can see people adding on to, particularly with non-standardized homegrown cultivars and how well they do in their local area.
  • beaconstudios 6y ago
    Limewire had an interesting protocol for exactly this back in the day, over its Gnutella network. I don't know if it scales to search engine sizes (most things don't, especially distributed) but there's at least some prior production grade work in this space. Edit: having just double checked it looks like Gnutella predates limewire; that's just the avenue through which I came across it.
    • deleted 6y ago
      [deleted]
  • lovemenot 6y ago
    >> Maybe if there is a way to share indexes... What does a SE index look like? Any designs in the open?
• emaro 6y ago
  There is one: https://yacy.net/ https://yacy.net/ However it's slow and the last release is from 2016.