3 ms·
Fact-checking sets up a seemingly-circular definition where "a trustworthy source is a source trusted by trustworthy sources" It reminds me of a similarly recu
by unholiness 5y ago
Fact-checking sets up a seemingly-circular definition where "a trustworthy source is a source trusted by trustworthy sources"
It reminds me of a similarly recursive definition in Google's original pagerank algorithm: "A popular site is a site linked prominently by other popular sites". for pagerank, there was a nice way around the issue: apply popularity iteratively.
You can imagine starting with some initial guess (perhaps that every site is equally popular), then score sites which are heavily linked higher to update the score. Then, use those scores to weigh the value of their links (unpopular sites' links matter less, popular sites' links matter more) and evaluate popularity again, and again...
Of course, this is simply a matrix multiplication applied infinitely many times. If the matrix is not singular (which, if your system is sufficiently connected, is vanishingly unlikely), then the final solution will converge to the eigenvector with the largest eigenvalue no matter what your initial guess was. The definition of popularity which seemed which recursive, wasn't. It doesn't matter who you assume is popular to start with, if you apply the definition enough times, you will arrive at the same result.
It makes me wonder if there is a possible computational aid to some of these fact-checking dilemmas, where we don't need to trust anyone a priori to have trust in the whole network. In areas where we can't look at truth as some absolutely thing, that doesn't mean we should we throw our hands up and say "everything depends on who you ask". We can still have a goal of evaluating things that are incredibly likely to be true, false, misleading, etc.
Society has lots of imperfect tools for evaluating information from those who know more than us: We can look at their past performance, we can understand their incentives (e.g. brand harm if discovered), we can check if others disagree, we can use quality heuristics (e.g. superficial flaws can be a sign of deeper flaws), we can gain some small expertise to help evaluate the greater experts. None of these things are perfect signals but they are strong signals.
Currently fact-checking is cobbling together these signals informally, with responsibility spread out culturally among the organizations' internal hierarchy, the fact-checked victims complaining, some outsiders who bother to fact-check the fact-checkers, everyone's subjective trust of brands, etc. I wonder if there could be a more formal way to do this, where sources fact-check each other in a decentralized way. The ultimate score of trust would come not from one single source of authority, but from a global evaluation, which anyone could calculate, of this recursive definition of trust.
It would come with its own set of problems (google-bombing was a common thing in that era for rarely-searched terms) but it still seems like a massively useful tool, and one I've never seen an attempt to formalize.