4 ms·

Who fact-checks the fact-checkers?

md_ 6y ago
The point here seems not to have independent professional fact-checkers, but to crowd-source it. On the one hand, I certainly struggle to see how more crowds will improve upon what is fundamentally a problem with crowds to begin with. On the other, Wikipedia seems to work reasonably well--and purely as a result of the community mores, not due to any sophisticated moderation algorithm or ranking structure or similar. I'm not holding my breath, but it is interesting to see if a community-driven effort imbued with a Wikipedian-like spirit might succeed where more automated efforts have not.
- briantakita 6y ago
  The underlying question is philosophical freedom vs authoritarianism. The same tactics can be utilized by different actors who have different worldviews. The Stasi utilized Crowdsourcing to enforce authoritarianism for example. One person's terrorist is another's freedom fighter. One's misinformation is another's truth particularly when it comes to how information is interpreted & what is focused upon.
  - fellowniusmonk 6y ago
    Once a proposition is falsified a rational community would rejected it. Once your community or protocol "philosophically" rejects checksums it is only a matter of time before your system breaks catastrophically. This philosophical freedom vs authoritarianism thing you're proposing is a view that is incomplete and small to the point of being false. The actual underlying question is do you accept that grounded truth exists and are you willing to have a large and nuanced enough view to include all non false propositions and reject all false propositions.
  - md_ 6y ago
    Sure, and if you squint hard enough, there are similarities between having a warning label slapped on your Tweets and being thrown in a gulag for decades. But if you're squinting that hard, not a lot of light is getting through.
- ardy42 6y ago
  > On the one hand, I certainly struggle to see how more crowds will improve upon what is fundamentally a problem with crowds to begin with. When you only have a hammer, every problem looks like a nail. I believe that the social networks have built technologies that probably can't actually be made to work in a socially beneficial way, since they're utterly dependent on crowdsourcing and automation for economic reasons, but those tools aren't actually up the the problems they face.
- seqizz 6y ago
  Wikipedia works because you can remove content (even remove accounts afaik) once you're sure it's providing wrong information. On Twitter, people will interpret this as censoring.
  - md_ 6y ago
    That's true. And the alternative (contextual warnings) is of unclear value: https://link.springer.com/content/pdf/10.3758/MC.38.8.1087.pdf https://link.springer.com/content/pdf/10.3758/MC.38.8.1087.p....
- collyw 6y ago
  Personally I have found Wikipedias quality to be declining for anything with a political slant to it. Articles about any public figure that has a right wing opinion looks more like a smear piece than an objective piece. Often phrases like "has been described as" are used in place of objective facts.
deleted 6y ago
[deleted]
bena 6y ago
The problem with this statement is that it assumes that "fact-checkers" are a single entity when that is not the case. The answer to "who fact-checks the fact-checkers" is "the fact checkers". And we can use consensus, reputation, and checks against known facts to validate the checks themselves. If group A, group B, and group C all agree and group D disagrees, then we can say with relative confidence that D is probably wrong in this case. It is safe to go with the consensus. However, if A, B, C are notoriously unreliable and have a history of distorting claims, then it is beneficial to see why D disagrees with all three. Or even not use A, B, and C as sources. We can also judge a fact-checker's veracity by how they evaluate a claim. Do they provide sources for their claim. It's one thing to say "This will murder 40 people a minute" and quite another to say "This claim disagrees with a recent study by the CDC [link to study] that says it will only murder 39 people a minute". The only reason to cast doubt on the process or activity of fact-checking is if you don't want people checking your facts.

md_ 6y ago

The point here seems not to have independent professional fact-checkers, but to crowd-source it. On the one hand, I certainly struggle to see how more crowds will improve upon what is fundamentally a problem with crowds to begin with. On the other, Wikipedia seems to work reasonably well--and purely as a result of the community mores, not due to any sophisticated moderation algorithm or ranking structure or similar. I'm not holding my breath, but it is interesting to see if a community-driven effort imbued with a Wikipedian-like spirit might succeed where more automated efforts have not.

briantakita 6y ago
The underlying question is philosophical freedom vs authoritarianism. The same tactics can be utilized by different actors who have different worldviews. The Stasi utilized Crowdsourcing to enforce authoritarianism for example. One person's terrorist is another's freedom fighter. One's misinformation is another's truth particularly when it comes to how information is interpreted & what is focused upon.
- fellowniusmonk 6y ago
  Once a proposition is falsified a rational community would rejected it. Once your community or protocol "philosophically" rejects checksums it is only a matter of time before your system breaks catastrophically. This philosophical freedom vs authoritarianism thing you're proposing is a view that is incomplete and small to the point of being false. The actual underlying question is do you accept that grounded truth exists and are you willing to have a large and nuanced enough view to include all non false propositions and reject all false propositions.
- md_ 6y ago
  Sure, and if you squint hard enough, there are similarities between having a warning label slapped on your Tweets and being thrown in a gulag for decades. But if you're squinting that hard, not a lot of light is getting through.
ardy42 6y ago
> On the one hand, I certainly struggle to see how more crowds will improve upon what is fundamentally a problem with crowds to begin with. When you only have a hammer, every problem looks like a nail. I believe that the social networks have built technologies that probably can't actually be made to work in a socially beneficial way, since they're utterly dependent on crowdsourcing and automation for economic reasons, but those tools aren't actually up the the problems they face.
seqizz 6y ago
Wikipedia works because you can remove content (even remove accounts afaik) once you're sure it's providing wrong information. On Twitter, people will interpret this as censoring.
- md_ 6y ago
  That's true. And the alternative (contextual warnings) is of unclear value: https://link.springer.com/content/pdf/10.3758/MC.38.8.1087.pdf https://link.springer.com/content/pdf/10.3758/MC.38.8.1087.p....
collyw 6y ago
Personally I have found Wikipedias quality to be declining for anything with a political slant to it. Articles about any public figure that has a right wing opinion looks more like a smear piece than an objective piece. Often phrases like "has been described as" are used in place of objective facts.

deleted 6y ago

[deleted]

bena 6y ago

The problem with this statement is that it assumes that "fact-checkers" are a single entity when that is not the case. The answer to "who fact-checks the fact-checkers" is "the fact checkers". And we can use consensus, reputation, and checks against known facts to validate the checks themselves. If group A, group B, and group C all agree and group D disagrees, then we can say with relative confidence that D is probably wrong in this case. It is safe to go with the consensus. However, if A, B, C are notoriously unreliable and have a history of distorting claims, then it is beneficial to see why D disagrees with all three. Or even not use A, B, and C as sources. We can also judge a fact-checker's veracity by how they evaluate a claim. Do they provide sources for their claim. It's one thing to say "This will murder 40 people a minute" and quite another to say "This claim disagrees with a recent study by the CDC [link to study] that says it will only murder 39 people a minute". The only reason to cast doubt on the process or activity of fact-checking is if you don't want people checking your facts.