3 ms·
As others have commented, the problem here is the ranking algorithm and how it can be gamed. Essentially, trust. 'Web of trust' has its flaws too: a sufficient
by desc 7y ago
As others have commented, the problem here is the ranking algorithm and how it can be gamed. Essentially, trust.
'Web of trust' has its flaws too: a sufficiently large number of malicious nodes cooperating can subvert the network.
However, maybe we can exploit locality in the graph? If the user has an easy way to indicate the quality of results, and we cluster the graph of relevance sources, the barrier to subverting the network can be raised significantly.
Let's say that each ranking server indicates 'neighbours' which it considers relatively trustworthy. When a user first performs a search their client will pick a small number of servers at random, and generate results based on them.
* If the results are good, those servers get a bit more weight in future. We can assume that the results are good if the user finds what they're looking for in the top 5 or so hits (varying depending on how specific their query is; this would need some extra smarts).
* If the results are poor (the user indicates such, or tries many pages with no luck) those servers get downweighted.
* If the results are actively malicious (indicated by the user) then this gets recorded too...
There would need to be some way of distributing the weightings based on what the servers supplied, too. If someone's shovelling high weightings at us for utter crap, they need to get the brunt of the downweighting/malice markers.
Servers would gain or lose weighting and malice based on their advertised neighbours too. Something like PageRank? The idea is to hammer the trusting server more than the trusted, to encourage some degree of self-policing.
Users could also chose to trust others' clients, and import their weighting graph (but with a multiplier).
Every search still includes random servers, to try to avoid getting stuck in an echo chamber. The overall server graph could be examined for clustering and a special effort made to avoid selecting more than X servers in a given cluster. This might help deal with malicious groups of servers, which would eventually get isolated. It would be necessary to compromise a lot of established servers in order to get enough connections.
Of course, then we have the question of who is going to run all these servers, how the search algorithm is going to shard efficiently and securely, etc etc.
Anyone up for a weekend project? >_>