3 ms·
There's also yacy which is the distributed search effort. Tiny by size, but worth mentioning.
by viraptor 8y ago
There's also yacy which is the distributed search effort. Tiny by size, but worth mentioning.
- thekyle 8y agoI really like YaCy (I run a node), but a big problem with it that will prevent it from ever competing with Google/Bing is the lack of a verifiable link graph for ranking results. Right now it basically just uses full text search and a local link graph which is problematic.
- misterman0 8y ago>> the lack of a verifiable link graph I hadn't heard about YaCy even though I'm in their game. I too run a search engine that doesn't use a link graph for scoring relevance. My bet is on semantic relevance over PageRank, because I think our understanding of language models have become so sophisticated we can start relying on them solely. Also, it's harder I think to game a language model when compared to a page rank model. What do you think is lacking from non-link graph search engines? EDIT: maybe I misunderstood b/c it says here YaCy _has_ a link graph: https://yacy.net/en/API.html https://yacy.net/en/API.html
- thekyle 8y ago> maybe I misunderstood b/c it says here YaCy _has_ a link graph: https://yacy.net/en/API.html https://yacy.net/en/API.html That link graph being referenced is the local nodes one, not a global one. So it only yields links that have been discovered by the node you are sending the request to. You could send a request to each node individually and aggregate the results, however, you have no way to verify the nodes are being honest without actually crawling those pages yourself. If I may ask, what is the project that you are working on?
- misterman0 8y ago>> it only yields links that have been discovered by the node you are sending the request to Wait! YaCy is a little bit hard to understand. I don't mean that in any way other than in an extremely encouraging way. What do you mean exactly? A YaCy node will interrogate the "swarm" of data and that interrogation might yield more links compared to if you interrogate another node? What's that all about?
- misterman0 8y agoIc. Thx! I'm working on a project similar to the YaCy one. The main difference is that I do not want or even require that each node trust each other. I let the user decide who they trust. https://github.com/kreeben/resin https://github.com/kreeben/resin
- elcomet 8y agoWhat is your search engine?