3 ms·

>> the lack of a verifiable link graph I hadn't heard about YaCy even though I'm in their game. I too run a search engine that doesn't use a link graph for sco

>> the lack of a verifiable link graph I hadn't heard about YaCy even though I'm in their game. I too run a search engine that doesn't use a link graph for scoring relevance. My bet is on semantic relevance over PageRank, because I think our understanding of language models have become so sophisticated we can start relying on them solely. Also, it's harder I think to game a language model when compared to a page rank model. What do you think is lacking from non-link graph search engines? EDIT: maybe I misunderstood b/c it says here YaCy _has_ a link graph: https://yacy.net/en/API.html https://yacy.net/en/API.html

thekyle 8y ago
> maybe I misunderstood b/c it says here YaCy _has_ a link graph: https://yacy.net/en/API.html https://yacy.net/en/API.html That link graph being referenced is the local nodes one, not a global one. So it only yields links that have been discovered by the node you are sending the request to. You could send a request to each node individually and aggregate the results, however, you have no way to verify the nodes are being honest without actually crawling those pages yourself. If I may ask, what is the project that you are working on?
- misterman0 8y ago
  >> it only yields links that have been discovered by the node you are sending the request to Wait! YaCy is a little bit hard to understand. I don't mean that in any way other than in an extremely encouraging way. What do you mean exactly? A YaCy node will interrogate the "swarm" of data and that interrogation might yield more links compared to if you interrogate another node? What's that all about?
- misterman0 8y ago
  Ic. Thx! I'm working on a project similar to the YaCy one. The main difference is that I do not want or even require that each node trust each other. I let the user decide who they trust. https://github.com/kreeben/resin https://github.com/kreeben/resin
elcomet 8y ago
What is your search engine?

thekyle 8y ago

> maybe I misunderstood b/c it says here YaCy _has_ a link graph: https://yacy.net/en/API.html https://yacy.net/en/API.html That link graph being referenced is the local nodes one, not a global one. So it only yields links that have been discovered by the node you are sending the request to. You could send a request to each node individually and aggregate the results, however, you have no way to verify the nodes are being honest without actually crawling those pages yourself. If I may ask, what is the project that you are working on?

misterman0 8y ago
>> it only yields links that have been discovered by the node you are sending the request to Wait! YaCy is a little bit hard to understand. I don't mean that in any way other than in an extremely encouraging way. What do you mean exactly? A YaCy node will interrogate the "swarm" of data and that interrogation might yield more links compared to if you interrogate another node? What's that all about?
misterman0 8y ago
Ic. Thx! I'm working on a project similar to the YaCy one. The main difference is that I do not want or even require that each node trust each other. I let the user decide who they trust. https://github.com/kreeben/resin https://github.com/kreeben/resin

elcomet 8y ago

What is your search engine?