24 ms·

Show HN: I made a platform for journalists to “open source” their fact checking

593 points by YazIAm 8y ago

YazIAm 8y ago
Been experimenting with building tools to make fact checking more transparent and reusable for a few years now, this is the latest iteration of that project. Feedback appreciated!
- robinduckett 8y ago
  Perhaps I am just being nitpicky, and I have no comments on the content or intent of the site (not my industry / area) but grey/blue text on the night sky blue background with white for emphasis really threw me whilst trying to read the text on the site. It may also have to do with Linux's crappy font rendering but I found it unreasonably hard to look at. In my experience, there are few good colours that remain readable on text, and I personally find serif fonts visually displeasing in most cases, although there are some exceptions (https://www.nytimes.com/ https://www.nytimes.com/) and even then, the light grey on white background is pretty displeasing. I wish you all the best in your endeavour!
  - YazIAm 8y ago
    The goal is to enable journalists/bloggers to annotate their content with their fact checking. Allowing their readers to audit to and to contribute to the fact checking for themselves. In other words, enabling journalists to "show their work". Thanks for the feedback about the colours, I'll play around with it!
    - v_lisivka 8y ago
      It doesn't look like "fact checking". It looks like "citation checking". More popular but wrong opinion will take over less popular but correct one. Fact checking need a court like system, with judges. Contact me, if you interested. I have lot of real world experience with that.
      - YazIAm 8y ago
        So this differs from regular citations in that: 1) only primary sources are accepted 2) the claim and primary source must be verified by a community of fact checkers Definitely happy to chat though, will send you an email!
- Vinnl 8y ago
  Is the primary idea for journalists to share the checks they did, or to have contributors help them check whether the things they mention are factual? The former is what I'd imagine when I hear "open source fact checking", the latter would be called "crowd-sourced fact checking", I think. Just some feedback that might or might not be useful for your messaging, depending on your goals :)
- oblib 8y ago
  Looks like you've done great work on this. It's clean and easy to use and I don't necessarily see a huge problem with "noise" diluting the "facts". I certainly appreciate both the goal and work you've done to accomplish it.
- scott_s 8y ago
  For this to be useful at scale, you will need to not just design this for good-faith users, but also against bad-faith actors. If this catches on, it will become a target of coordinated disinformation campaigns. Have you been thinking about what to do against that?
  - YazIAm 8y ago
    Indeed, currently all "verified" reviewers are manually vetted. Long term the goal is to have an open reputation score for every fact checker. I'm currently experimenting with different implementations of this, and seeing good promise.
    - scott_s 8y ago
      How would the open reputation score mitigate bad-faith actors trying to exploit the system? For example, I believe HN has a vote-ring detector, but I think there's also a decent amount of manual intervention from the mods. Amazon's reputation system is under constant attack through a variety of means (paying teams of people for fake reviews, going to the length of having real transactions on Amazon between fake-reviewer and the seller buying the review).
fabiandesimone 8y ago
Very interesting and very well done!
mshkreli 8y ago
Neat tool and well done, but why would a journalist use this? In the day and age of 'fake news', it's pretty clear that journalists don't highly value checking facts.
- YazIAm 8y ago
  The idea is that good journalists can stand out by making their fact checking open and auditable
  - xondono 8y ago
    While I quite like the concept, I think there's something of a catch 22 here. 'Fact-based' generally implies 'lesser impact', it's way easier to catch eyeballs with fakes and click-bait than with better stories, and part of the fake news issues is that professional success in journalism is dependent on impact, not on accuracy (a similar thing goes on in academia). You'll have to figure out a way to increase the impact of writers with more accuracy (less negative fact checks maybe?) by promoting them in some shape or form. Have you considered top writers scoreboards or something similar?
queercode 8y ago
The hover state reminds me of the lyric annotation system Genius (https://genius.com/ https://genius.com/) has, which is a good thing.
ahnick 8y ago
IMHO reputation is important to build into the base level for a platform like this. Allowing anon registrations and commenting will eventually lead to a lot of noise and abuse of the platform for misinformation. I think some sort of verification of a person's qualifications on particular subject areas and then allowing them to only "fact check" on those qualified areas is required here to have an effective platform that will scale well.
- YazIAm 8y ago
  Currently (for the prototype), you can start contributing, but your reviews only become public once I've vetted you and given you a fact checking test. Long term the goal is to have an open reputation score for every fact checker. I'm currently experimenting with different implementations of this, and seeing good promise.
  - hamitron 8y ago
    What credentials do you have to properly vet people?
    - YazIAm 8y ago
      Well first, the point of this type of open fact checking is you can always verify everything for yourself, so you don't need to trust my credentials. Working on this over the last few years though, I've personally fact checked over 500 assertions.
      - adamsea 8y ago
        I would respectfully disagree. Introduce enough noise into the system and the cost of verification becomes so high that doing so is unrealistic for many or most people. Also, on first glance the app looks great, well-polished, and the UX of reading an article and looking at sources for claim is nice :). Thanks for working on and sharing this!
      - SamBam 8y ago
nahimn 8y ago
Finally something viable to combat fake news
- v_lisivka 8y ago
  Russian government backed trolls will use this system to spread fakes instead. Wikipedia has similar system and Russians abuse it a lot.
  - Fnoord 8y ago
    Can you back this up with examples?
    - v_lisivka 8y ago
      Examples of hoaxes backed by Russian government trolls, or examples of abuse of Wikipedia and other sites by Russian government trolls? I have both.
      - Fnoord 8y ago
        The latter.
- murph-almighty 8y ago
  I'm cautiously optimistic. I like that this emphasizes use of primary sources, but 1) Some primary sources are dubious, or manipulated. See the multiple versions of Acosta getting kicked out of a White House press briefing. 2) What happens when you have an article that repeatedly cites dubious secondary opinion pieces, but the reader accepts these pieces as fact?
  - YazIAm 8y ago
    1) Now that video deepfake is a thing, I currently only allow videos as primary sources in rare circumstances (such as the video being published by a verified account of the subject). 2) Only primary sources are accepted. Currently anyone can start contributing, but every claim is still reviewed by a team of verified fact checkers. I hope to scale this long term with a fact checking reputation score.
jaredwiener 8y ago
How would a reporter use this? Do you need to have an article already written/published? Does it have to be on your site? It seems like its less for fact checking, which would happen prior to publication, than for fact proof?
- YazIAm 8y ago
  I'm happy to provide any journalist/blogger who wants to include this style of annotation on their site a js plug-in to do just that. The whole fact checking process actually happens within the system, the facts you see on there were collaboratively fact checked by a team of 7 to 11 vetted fact checkers. So that may also provide an incentive for journalists.
jasonlotito 8y ago
How does this handle anonymous sources? Not only do you have to account for what those sources say, but also, a reporter reporting what an anonymous source said can be 100% factual (Yes, the anonymous, valid source was indeed reported accurately), but maybe the anonymous source is incorrect. The reporting is accurate. The source is not. Basically, a lot of grey area. And with that comes the issue of anything not "fact checked" by a service like this could be dismissed. What harm does that do? Not a dismissal of a service like this. Rather, just open questions.
- YazIAm 8y ago
  Only public primary sources are allowed as evidence on SourcedFact. Anonymous sources certainly have value within journalism, but the goal of SourcedFact is to make facts that can be proven beyond a reasonable doubt with public primary sources stand out from everything else.
  - jasonlotito 8y ago
    Okay. I think as long as that distinction is clear, sounds good!
johnm 8y ago
How are you going to deal with underlying "sources" where the content disappears or is changed out from under the fact checkers?
- YazIAm 8y ago
  Every source document submitted to the platform is automatically saved to the webarchive, and the webarchive snapshot is used :)
  - 7373737373 8y ago
    Definitely have a look at https://tlsnotary.org/ https://tlsnotary.org/ !
teleclimber 8y ago
Nice to see more work in this area. However as others have said a reputation system for participants is absolutely key. Any platform like this has to assume it is going to be flooded by bots trying to influence opinion on behalf of corporations, foreign governments and activists. A reputation where users gain reputation slowly (through work, by verifying sources, etc..) become more able to influence opinion, but the reputation drops quickly if they are found to be misleading people. How this actually works is harder than it seems if you want a truly robust system. When I think of platforms like this I think of Galileo[0]. He was perceived as a troll or heretic in his time, but of course he was right. How do you create a platform that allows correct ideas to flourish even when they go against conventional wisdom? [0] https://en.wikipedia.org/wiki/Galileo_Galilei https://en.wikipedia.org/wiki/Galileo_Galilei
- YazIAm 8y ago
  I 100% agree that the reputation system will be they key to scaling this. I'm currently seeing promise with my experiments with reputation scoring. As for "How do you create a platform that allows correct ideas to flourish even when they go against conventional wisdom?" Great question, however my goal here is a bit more modest, it's to create a platform that allows facts which are provable by public primary sources to stand out from everything else. I think a platform that focuses on simply that can add a lot of value to political discourse.
  - SamBam 8y ago
    The bots would be a serious issue if this ever became big enough, since they can add to each other's reputation. Reddit probably has thousands of code-hours devoted to this, and bots are still a problem. There is also the problem that humans will probably tend to upvote "facts" that they agree with, and visa-versa. I'm not sure that reputation is the solution. And yet "expert" verification introduces trust. I'm not sure what the solution is.
    - teleclimber 8y ago
      Yes it's a super tough problem, not to be underestimated by anybody working on such a platform. If there is any easily automatable way gaining reputation, then the bots will use that, and if you can easily pass reputation too, then the rep system won't hold. > I'm not sure what the solution is. Ditto, and I've ben pondering this on and off for years :/
elliekelly 8y ago
This reminds me of Grasswire: https://news.ycombinator.com/item?id=7954327 https://news.ycombinator.com/item?id=7954327
jellicle 8y ago
So, your examples are all "reporting about court cases" and you use court documents as your citations for facts. Court cases are as good as we have for determining facts, I suppose. (Though the error rate in court cases is quite high!) But 99.9999% of reporting isn't about court cases. Most reporting isn't about issues that will ever be litigated, and even those things that will be litigated, haven't been at the time of the reporting. "Mitch McConnell had oatmeal for breakfast this morning, according to sources." There, that's a fact. Now, tell me which court case I can refer to in order to verify that fact? Fact-checking is an impossible endeavor. The essence of journalism is reporting things no one knows. That's what news is! This is completely at odds with "fact-checking". A good news story CANNOT be fact-checked by the public because if the public knew it, it wouldn't be news. There is an extremely limited genre of "news by trawling public but little-known documents", such as reporting on old court cases. Almost zero news fits into that category.
- jpttsn 8y ago
  Call me a pedant, but I'd say OP is enabling "consensus checking" I find the present day's fact checking wave is a misnomer. In a world where something is false, but sourced, most of what we call fact-checking would only reinforce mistruths. e.g. The ratio of circumference to diameter is exactly three to one [1][2][3] [1]: the Bible. [2]: pious scholars. [3]: low-tech experiments. Would be cool to see someone think carefully about what "real" "fact checking" would be. First-hand sources? Links to instructions for replicating findings? Consensus checking is of course also an interesting problem, albeit a pretty different one.
  - NateEag 8y ago
    Offtopic, but the Bible does not actually use a value of 3 for pi: https://www.purplemath.com/modules/bibleval.htm https://www.purplemath.com/modules/bibleval.htm
m-i-l 8y ago
See also https://fullfact.org https://fullfact.org which sponsored an Apache Solr hackathon a couple of years back.
deepstream 8y ago
I'm surprised that FAGAM haven't got into the crowd-sourced fact checking space. Seems like a great way to build a knowledge engine / AI, by getting humans to "connect the facts" while motivating (herding) them with the idea they are doing it to "preserve truth in an era of fake narratives", when actually they are just helping build an AI.
rohan1024 8y ago
We need this decentralised
- YazIAm 8y ago
  That's the long term goal!
sidhantgandhi 8y ago
By the way, clicking on a fact on Chrome iOS is not loading anything.
- YazIAm 8y ago
  Will investigate, thanks for letting me know!
- jason_slack 8y ago
  Safari on iOS isn’t working for me.
thebeefytaco 8y ago
This is pretty awesome. Any time I open any article, the first thing I do is try to follow the trail of links/sources back to the primary sources and read that instead. If I can't find one, I don't trust the article.
- YazIAm 8y ago
  I'm the exact same way, that's why I built this :)
eeZah7Ux 8y ago
"open source" is not the right term and also is not a verb. "transparency" is the word you are looking for. It's more general and predates "open source" by far.
- 0xffff2 8y ago
  "Open source" is absolutely a verb. "I open sourced my latest work project this month". It's also a term that seems to be used similarly to the post title in contexts outside of software. For example, I subscribe to a podcast that covers arms control, where they frequently use the term "open source" to describe analysis derived publicly available sources and methods.
idlewords 8y ago
What problem does this solve that existing reputation systems don't? When I read a New Yorker article, for example, part of what my subscription is paying for is considerable skilled labor by a group of in-house fact checkers. How does moving this work to an outside free platform improve the situation? Moreover, how do you replicate work like calling sources to verify statements before publication? I wrote a short piece for Wired once and had to go back and forth with their fact checker on the most picayune details. In an article about balancing online work and travel I had mentioned in passing searching under a hotel bed for an outlet, and she wanted to know what hotel it was, what country it was in, when I stayed there, etc. The process was exhaustive and not something I would go through with a bunch of Internet randos. It was also done before publication and involved a fairly high level of trust.
- YazIAm 8y ago
  The goal is to capture the fact checking process and enable any reader to review it for themselves. Certainly some publications currently have an excellent fact checking process, but very few make their fact checking open and transparent. As for "how do you replicate work like calling sources to verify statements before publication?" Good question. Currently this is only designed to open up the fact checking which involves public primary sources. I think that a platform that focuses on doing that subset of fact checking really really well can add a lot of value to the public discourse.
  - jessaustin 8y ago
    Congratulations on this; it looks great. Currently this is only designed to open up the fact checking which involves public primary sources. Limiting the scope to this is wise. It's a tractable problem and it's better journalism. The types of journalists who write "this unnamed source implied that this other unnamed source had implied to another unnamed source..." articles won't like this sort of tool, but those journalists should be encouraged to change careers. Some might not realize that articles like that are fairly common. For an example, I went to NYT homepage and this was the first article: https://www.nytimes.com/2019/04/03/us/politics/william-barr-mueller-report.html https://www.nytimes.com/2019/04/03/us/politics/william-barr-... First sentence: "Some of Robert S. Mueller III’s investigators have told associates that Attorney General William P. Barr failed to adequately portray the findings of their inquiry and that they were more troubling for President Trump than Mr. Barr indicated, according to government officials and others familiar with their simmering frustrations." This sort of crap gets swept under the rug within a couple of weeks, but we're certainly paying attention to it now.
IronWolve 8y ago
Wikipedia talk pages are a perfect example of facts ignored and removed from articles to push an agenda. You can see how a narrative can be pushed with half-truths and selective editing. Same thing is happening in News, its agenda driven, not truth driven. Entertainment opinion pieces that pretends to be news. People are just bending the facts to support their personal politics. If Wikipedia can't be factually accurate due to editors personal politics, how do you keep those personal politics out of a fact checker?
- YazIAm 8y ago
  You're 100% correct in that one can cite completely valid facts and still push an agenda by omitting other facts. I'm actually working on a "fact comment section" feature where any reader can pin facts to an article (facts that still go through the same rigorous and open fact checking process as everything else). Facts which readers deem are important extra context. However, I'd argue that even without such a feature, having an open standard for facts openly verified from primary sources will add a lot of value to current public discourse.
hoaxcracker 8y ago
I have been thinking about something similar but stuck on a different project. Great work!
rootusrootus 8y ago
Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.
- sanbor 8y ago
  But it might be worth a try. Wikipedia does pretty well imho but if someone would told me the idea I would never thought that it would work.
  - dfgert 8y ago
    Wikipedia has flaws,too. 1. Your account need to have good history or social repo for your edits to be seriously considered by Wikipedia admins. 2. Admin can have their own set of bias and can decide to ignore edits. Remember Wikipedia is a centralized and under control of single entity, fact that they rely on donation doesn't change anything.
    - stingraycharles 8y ago
      Just because something has flaws doesn’t mean it’s not a good idea to do it.
      - jtuente 8y ago
        There wasn't anything said about Wikipedia being a bad idea, but rather that it shouldn't be your sole source for fact-checking. I've learned that people only trust Wikipedia to the point at which it confirms their biases.
        penagwin 8y ago
        Part of the problem is that Wikipedia is generally pretty darn accurate. Everybody always talks about funny vandalism stories, but generally it's pretty good. The issue of course is that there's parts of it that definitely aren't correct. This makes it easy to say "Well if it's accurate 99% of the time then it must confirm what I thought!".
nilskidoo 8y ago
How we've existed for so long without this is beyond me. Thank you, to the creators of this.
dsjoerg 8y ago
YazIAm, you should check out https://www.kialo.com https://www.kialo.com and https://web.hypothes.is/journalism/ https://web.hypothes.is/journalism/ and https://www.procon.org/ https://www.procon.org/
jshowa3 8y ago
While I like how the software ties together the sourcing, I feel like the following is lacking (and I realize this is an alpha, but its things to consider) 1. Anyone who answers a question regarding whether the claim is "fully" proven by a source or whether the source is valid for the claim must or should (debatable) provide proof of their own interpretation by citing either specific instances in the source in question supporting it. For example, in the Net Neutrality article, Yatz states this: The adopted principles in this statement are at the top of page 3 in a response to does the source fully prove the claim. It would be nice if that was linked to the actual location in the document or identified through highlighting or some other mechanism to get full context for the response. There also should be uses for outside sources to be attached when evaluating these questions that are suppose to validate a source. This is because someone could submit a response to these source validity questions and cite an external source or internal quote of the source in question for their reasoning that doesn't support their answer. 2. Having some sort of expertise verification is extremely important and should be weighted either separately as in Experts: Yes Experts: No category and should be more important. It distinguishes this from random anon answers that have no training in how the source should be interpreted. 3. Having anonymous people do this seems rather dubious and the current platform seems like it would make it prone to Youtube comment syndrome. Maybe have some reputation system to gate keep like StackOverflow? 4. Sources usually require appropriate interpretation in order to be taken in the right context and be considered correct. Sometimes there's no single source or any source, philosophically, that fully proves a claim. I'm guessing that's what supporting documents are for? 5. Why is the original articles claim allowed to be edited?
catchmeifyoucan 8y ago
For Mobile, I didn’t know what I was looking for. The Navigation bar should be fixed to the top.
snambi 8y ago
This is really great idea. Today I don't trust the journalists, but if they provide references atleast there is a bit a scope for trusting a particular article. ofcourse, provided the references can stand the scrutiny.
Swquenzer 8y ago
Cool platform. One suggestion - create a unique url/route for each article. I wanted to send a particular article to a friend, but realized there's no way to provide a unique link.
- YazIAm 8y ago
  Thanks :) Indeed they have unique urls, I'll see if I can make that clearer on the landing page: Net Neutrality - https://sourcedfact.com/a/net-neutrality-how-we-got-here https://sourcedfact.com/a/net-neutrality-how-we-got-here Cell Phone Privacy - https://sourcedfact.com/a/cell-location-data-without-warrant https://sourcedfact.com/a/cell-location-data-without-warrant Israel Boycotts - https://sourcedfact.com/a/government-measures-against-israel-boycotts https://sourcedfact.com/a/government-measures-against-israel...
hkdobrev 8y ago
Is this pravduh? :D Source threads: - https://twitter.com/elonmusk/status/1000072092769964033 https://twitter.com/elonmusk/status/1000072092769964033 - https://twitter.com/elonmusk/status/1000080833678528512 https://twitter.com/elonmusk/status/1000080833678528512 https://pravduh.com https://pravduh.com
- YazIAm 8y ago
  I actually got a friend who works at Tesla to email Musk about SourcedFact when he was tweeting about that haha, busy guy though ;)
geekfreakdev 8y ago
give beer to this guy!
vonnik 8y ago
This is really interesting. It reminds me a bit of peer-reviewing journalism via Web annotation: https://web.hypothes.is/ https://web.hypothes.is/ I have a couple comments to make as a former newspaper reporter: * Many important stories use anonymous sources that are impossible to verify via crowd-sourcing or by linking to official documents. Much of what happens in the world is not publicly documented nor is it widely known (until a reporter publishes a story). That gap between events and any record of those events is one of the structural problems that allow "fake news" to occur. * Timing matters. Reporters working for the dwindling number of publications that employ fact-checkers or editors have their facts checked before they publish. For that small group of outlets, this diminishes the number of falsehoods they are responsible for unleashing online. Fact-checking a story after its release is useful, but the cat's already out of the bag. The lie is already speeding around the world at the speed of viral outrage. * As with any platform where better information leads people to make different choices, there are lots of incentives for publications not engage in crowd-sourced fact-checking. Why do people lie or distort the facts on their dating profiles and job applications? Why do companies put forward their most utopian face? They are trying to shape the world with information. And so are media outlets such as Fox News, to name just the most egregious. * Fact-checking is a lot of work, and can descend into epistemological quibbles. Most people don't have time do engage in it, or even to understand the debate around a given fact. This is why, in the past, we outsourced this work to the editorial staff of the fourth estate. While this would bring more transparency to the process, I am not sure who would have time to take advantage of that transparency, anymore than the normal reader will closely follow the "talk" tab on a Wikipedia article.
chw9e 8y ago
Would be nice to avoid having to open another tab to do fact checking - having some kind of a summary or something beyond "10 verifications" would at least let me know if opening the new tab is really worth it. Ideally the article and all verifications would live in one tab.
- zachguo 8y ago
  Also, instead of a dedicated site, a browser extension allowing you to select text then right-click to get facts would work better. https://web.hypothes.is/journalism/ https://web.hypothes.is/journalism/
beat 8y ago
Awkward question here... can anyone cite an example of a major-media news story that was actually factually wrong in a way that "open source fact checking" could have detected it? Just because a bunch of people believe it's a problem doesn't mean it's a problem. I find the bias, when it exists, is in the presentation, not the facts themselves. And mainstream journalism is really committed to getting the facts right.
- jimmy1 8y ago
  The most recent and obvious case in my mind: Covington Catholic Incident: almost every major mainstream media outlet ran with the narrative that the boys were harassing the Native Americans which was based solely on one video clip. No traditional investigative work was done. No one was interviewed (until the damage had been done). Nothing was fact checked. It was just one video, and boom: instant headlines.
  - beat 8y ago
    I disagree. Whether they were harassing is a matter of opinion. Whether there was a confrontation is a matter of fact, confirmed by the video and photos. If you want to claim the "narrative", you need a link to an actual article with actual quotes. That's how fact-checking works, and your response is failing it badly.
    - ajvs 8y ago
      This has been debunked thoroughly. That guy went up to the kids and started beating his drum right in their faces to intimidate them and somehow the media painted the kids as the harasser.
      - beat 8y ago
        Was his intent to intimidate? Why do you say that? How is that a "fact"? Where did you even hear that story? Geez, y'all don't understand fact checking at all.
amirouche 8y ago
well done, thanks for sharing.
_mitch 8y ago
Just a couple quick feedback points: I'd prefer if the currently selected fact text remained highlighted when the mouse is not hovered over it. That way you know what the active "Investigate >" link is referring to. Also, for accessibility and convenience, it would be nice if you could tab through the instances of the fact text blocks.
- YazIAm 8y ago
  Thanks for the feedback, the second (accessibility) should be a quick fix. I definitely also want to address the first issue, the tricky part is coming up with a solution that also works when there are multiple assertions on one line, but I'll give it more thought.
devinplatt 8y ago
Hey everybody, I just want to point out that I'm working on a similar project to document evidence for and against claims: https://www.wikiclaim.org https://www.wikiclaim.org The tech is dumb (MediaWiki on Heroku), but it's been sufficient to allow me to spend several hundred hours creating probably about 200 hundred wiki pages so far.
- smenis 8y ago
  Thanks for posting this, I really enjoyed browsing Wikiclaim and it feels to me like I've never seen anywhere on the Internet that centralises this kind of content (commonly heard claims with info for/against)... Having said that, the usual strategy when I hear a new claim I'm not sure about is to google it; so is the Google search engine itself the competition for this site? It does try to give an "answer" at the top of search results these days. Then there is Quora. The main advantage Wikiclaim has is that, like Wikipedia, the info can be moderated and updated, and hopefully can become an improving source of truth. Godspeed, devinplatt!
  - devinplatt 8y ago
    Thanks Smenis! In time I hope that Google will surface Wikiclaim higher in its search results (whether URL or even answer box), so in my mind the competitor is also the ally :) I don't think the strong suit of Quora (information based personal experiences) overlaps with what I'm trying to do with Wikiclaim (information from numerous reliable sources, editable by anyone), so no issues there either.
uberdru 8y ago
Coz that's what we hire journalists for. . . .
volt4ire 8y ago
Sadly this "platform to open source factchecking" is not itself open source, as the site's source code is not available and the user content license restricts commercial use (see point 3 https://sourcedfact.com/terms_of_service https://sourcedfact.com/terms_of_service and https://blog.okfn.org/2010/06/24/why-share-alike-licenses-are-open-but-non-commercial-ones-arent/ https://blog.okfn.org/2010/06/24/why-share-alike-licenses-ar...)
- vortico 8y ago
  That's fine with me. The two concepts of "open-source" at hand are drastically different. The lack of ability to verify the website's source code doesn't prevent me in any way from seeing its content which helps me verify journalist claims. I hope the author is able to grow their service through marketing, licensing, and contracts with media agencies. Open-sourcing the service's software would be a distraction from that goal and would create fragmentation of its database when clones of the service are created. What's important for a service like this is open data, not necessarily an open-source stack.
  - nautilus12 8y ago
    Except we are talking about journalism which is prone to things like censorship, shadow banning and the like which would defeat the purpose of it being open. The source code would reveal if there were such a functionality.
    - YazIAm 8y ago
      I intend to move user contributed fact verifications to some sort of distributed ledger model to ensure transparency. Still in alpha though, so lots of work ahead.
iandanforth 8y ago
This is reminiscent of how some scientific papers are written. You might be able to pick up some tips from those processes as well. For example: - Write each sentence on a different line. This allows for better change tracking, commentary etc in a number of tools including github. - Inline sources and compile into a references section. I want to see that a claim is backed by more than one piece of supporting work whenever possible. - Have a compiled / formatted version for holistic evaluation. After you've done the hard work of writing factual statements the piece also needs to be readable. This is an orthogonal view into the same information however and should be presented as such.
rfeather 8y ago
I like the idea and genuinely hope that fact checking becomes more mainstream. Seriously, kudos to the author for building something. But what I'm not getting is why a news platform would use this, unless this is intending on becoming a news platform itself? I think there is an underlying assumption that the general world cares about things like "primary sources" and "logical fallacies" enough to bother hovering over the text or viewing the content, and that there isn't an intentional manipulation of these things by media organizations to fit a narrative. Maybe if this was a browser plugin that came with <major browser> by default and automatically highlighted fact checked statements in exiting articles. That way folks wouldn't have to opt in, but as it is, whats the incentive for either the populace or the media to participate? Think of it this way- how did the fact checking in the examples get done? Someone spent 2 minutes on their favorite search engine. People willing to do that will do it if they have the time, and people not willing probably aren't interested in information that would contradict their opinions anyway - unless it comes from a source they already trust, like someone in their bubble. My own experience with this is that the folks who will care exist but are rare. Others, you can literally watch their eyes glaze over the moment you introduce a little cognitive dissonance.
Zoo3y 8y ago
Where are you pulling the PDFs of FCC documents from?
indymike 8y ago
First, love the project. Hope it works and makes journalism better. Even if it doesn't it will be interesting to see how much the news industry sees value in crowd-sourced fact-checking. Good luck!
anigbrowl 8y ago
Promising!
abathur 8y ago
Interesting work, YazIAm. Conceptually this reminds me a little bit of the idea behind Kialo. Down the road, there might be some interesting synergy potential between fact attestation and debate. I've had a few related thoughts, on the off chance you find them useful: 1. It would be interesting, variously for journalists and the reading public, if it was possible for journalists to generate, disclose, and research cryptographic sourcing identifiers that enable them to figure out when they share a source. This could, critically, help journalists identify sources with a record of feeding other journalists bad information. It'd be nice if the same work could help the rest of us unravel citation chains based on a small number of unique sources, but I'm not sure there's a way to achieve that without it being fairly simple to re-identify sources via brute-force. 2. If we can drill enough to find some bedrock, I think it might be useful to have virtuous-cycle user tools, like browser extensions that warn users when visiting an article by writers/publications (and potentially even sources, per above) with repeated sourcing/attestation problems. I don't think of this as purely a journalistic thing. In the sciences it could apply to methodology, data-collection/statistical integrity. With some bedrock, tooling in place, and established trust/community practice in place here, it might also be possible to expand the scope a little and address things like headlines that aren't at all moored to evidence.
mikedilger 8y ago
I appreciate the recognition that there is a problem in journalism. But I'm afraid that the fundamental problems are problems in the nature of humans and aren't ameniable to this sort of solution. Those problems include: 1) Many people choose to believe what feels right to them, rather than dispassionately seeing things for what they are. Unfortunately there are a great many of people that do this regularly, and they have practiced this their entire lives. Richard Feynman cautioned us in saying that "The first principle is that you must not fool yourself - and you are the easiest person to fool." That caution was aimed at scientists who are already among the most objective of us, yet they can still suffer from this ailment. 2) When people talk about the facts, they rarely just lay out the facts dispassionately and without judgement. Humans need a motivation to tell a story, and most people's motivations include convincing other people to support their ideology, or else berating those who dont. Facts are organized within a mental framework of the ideology of the teller, wrapped up in beliefs, desires, and biases andcarry additional information about which ideological position you ought to hold -- sometimes they are even intended to mislead and manipulate the listener. The modern news media is composed of businesses in search of profits. That goal is often not aligned with a straight reporting of facts. By pandering to an audiences ideology they increase ratings. By reporting on salacious and exciting news, even if incorrect, they get ratings. Buyer beware.
- Razengan 8y ago
  3) Whenever there is a chance to gain wealth and/or influence without immediate disadvantages, humans will usually take it.
- ARandomerDude 8y ago
  Indeed. The heart is deceitful above all things, and desperately sick; who can understand it?
  - Jenz 8y ago
    > desperately sick How so?
    - 8y ago
jammygit 8y ago
What would you say if a well known, analytical journalist approached you to be able to perform fact checking, but you knew they were openly biased on many topics?
aleister_777 8y ago
The main fault I see is how do we augment your "facts" with other "facts". For instance, I clicked on Net Neutrality. It's tough to truly appreciate the "net neutrality" dynamic of the past few years without understanding the backstory of Reed Hastings battling the ILECs (https://www.dslreports.com/shownews/Netflix-CEO-Comcast-Wants-Whole-Internet-to-Pay-Them-129154 https://www.dslreports.com/shownews/Netflix-CEO-Comcast-Want...), buddying up to Obama (https://www.businessinsider.com/house-of-cards-obama-2013-12 https://www.businessinsider.com/house-of-cards-obama-2013-12), getting the law changed in his favor (https://www.nytimes.com/2015/03/13/technology/fcc-releases-net-neutrality-rules.html https://www.nytimes.com/2015/03/13/technology/fcc-releases-n...), and now rewarding Obama in kind: https://www.nytimes.com/2018/05/21/us/politics/barack-obama-netflix-show.html https://www.nytimes.com/2018/05/21/us/politics/barack-obama-.... It's almost as if it had nothing to with actual "net neutrality" at all since none of the horrible things have come to pass that were predicted. Broadband isn't as good as it could or should be, but it isn't getting worse.
yantra_ml 8y ago
This is really cool!!
enriquto 8y ago
But journalism is not only stating true facts, but choosing a limited set of facts to present. You can pretty well do fake news by stating a list of true facts (and omitting a few other, fundamental, true facts). Example: Published news : person A killed person B by pushing a knife on his stomach, at a time when person B was unarmed. Non-published statements : just before being killed, person B and a group of three friends armed with knifes attacked person A, which was unarmed at the time, and started cutting him. During the struggle, person A managed to get hold of B's knife and fought back, harming B. The friends of B were scared and ran away. Person A finally survived his injuries, but person B didn't. I suppose everybody agrees that publishing the first statement alone is a clear example of fake news.
GistNoesis 8y ago
Little feedback : Nice. Please add a vertical splitter in the verify page so that the verification is not constrained to a small margin on the right, which make it not easily readable. It would also be nice to have some color coding when reading articles, so you can know what's the majority opinion on the validity of the piece. Hopefully once you have gathered enough data, you can get one of these NLP bots to check the fact in an even more neutral way.
vegangaijin 8y ago
I <3 this.
alvalentini 8y ago
I think that the primary value of this platform would reside on making the fact-checking available and shareable to non-journalists.
JoelMcCracken 8y ago
This reminds me a bit of an application I wanted to build many years ago: A platform for collections of arguments. For example, the question "are humans contributing to global warming?" Allow crowd-submitted answers. Collect information about facts cited (similar to how this works). Mostly-rational people can be swayed by evidence and arguments, yet often disagree with each other because they doubt the truth of the other side's statements (fact checking), they believe the other side has missed important context in their argument, the other side has implicit assumptions that they disagree with (this is something that should be explored as a sub issue, then). I'm very happy to see this. I'd love to see more of this kind of thing in the future.
- devinplatt 8y ago
  Hi Joel. In case you didn't see my top level comment (it's buried quite low), I'm working on a similar project to document evidence for and against claims: https://www.wikiclaim.org https://www.wikiclaim.org The tech is dumb (just MediaWiki on Heroku), but it's been sufficient to allow me to spend several hundred hours creating about 200 wiki pages so far.
cphoover 8y ago
Love it... Good job.
Siira 8y ago
The site doesn’t work in iOS safari? I can’t get sources for the claims.
VvR-Ox 8y ago
This is a very good idea but I'm afraid it won't play along the human condition. People construct reality with things they want to believe. Most people aren't interested and/or skilled enough to be reasonable and think logically. How could someone like that orange ape become POTUS???? This is just one major indicator for the fall of reason. The enlightenment has failed and instead of a mass of mature citizens who are able to govern the world via consensus like the democratic system needs it to be we are driven by emotions. Keep on doing this and I hope it may still have some impact.

593 points by YazIAm 8y ago

Been experimenting with building tools to make fact checking more transparent and reusable for a few years now, this is the latest iteration of that project. Feedback appreciated!

robinduckett 8y ago
Perhaps I am just being nitpicky, and I have no comments on the content or intent of the site (not my industry / area) but grey/blue text on the night sky blue background with white for emphasis really threw me whilst trying to read the text on the site. It may also have to do with Linux's crappy font rendering but I found it unreasonably hard to look at. In my experience, there are few good colours that remain readable on text, and I personally find serif fonts visually displeasing in most cases, although there are some exceptions (https://www.nytimes.com/ https://www.nytimes.com/) and even then, the light grey on white background is pretty displeasing. I wish you all the best in your endeavour!
- YazIAm 8y ago
  The goal is to enable journalists/bloggers to annotate their content with their fact checking. Allowing their readers to audit to and to contribute to the fact checking for themselves. In other words, enabling journalists to "show their work". Thanks for the feedback about the colours, I'll play around with it!
  - v_lisivka 8y ago
    It doesn't look like "fact checking". It looks like "citation checking". More popular but wrong opinion will take over less popular but correct one. Fact checking need a court like system, with judges. Contact me, if you interested. I have lot of real world experience with that.
    - YazIAm 8y ago
      So this differs from regular citations in that: 1) only primary sources are accepted 2) the claim and primary source must be verified by a community of fact checkers Definitely happy to chat though, will send you an email!
Vinnl 8y ago
Is the primary idea for journalists to share the checks they did, or to have contributors help them check whether the things they mention are factual? The former is what I'd imagine when I hear "open source fact checking", the latter would be called "crowd-sourced fact checking", I think. Just some feedback that might or might not be useful for your messaging, depending on your goals :)
oblib 8y ago
Looks like you've done great work on this. It's clean and easy to use and I don't necessarily see a huge problem with "noise" diluting the "facts". I certainly appreciate both the goal and work you've done to accomplish it.
scott_s 8y ago
For this to be useful at scale, you will need to not just design this for good-faith users, but also against bad-faith actors. If this catches on, it will become a target of coordinated disinformation campaigns. Have you been thinking about what to do against that?
- YazIAm 8y ago
  Indeed, currently all "verified" reviewers are manually vetted. Long term the goal is to have an open reputation score for every fact checker. I'm currently experimenting with different implementations of this, and seeing good promise.
  - scott_s 8y ago
    How would the open reputation score mitigate bad-faith actors trying to exploit the system? For example, I believe HN has a vote-ring detector, but I think there's also a decent amount of manual intervention from the mods. Amazon's reputation system is under constant attack through a variety of means (paying teams of people for fake reviews, going to the length of having real transactions on Amazon between fake-reviewer and the seller buying the review).

fabiandesimone 8y ago

Very interesting and very well done!

mshkreli 8y ago

Neat tool and well done, but why would a journalist use this? In the day and age of 'fake news', it's pretty clear that journalists don't highly value checking facts.

YazIAm 8y ago
The idea is that good journalists can stand out by making their fact checking open and auditable
- xondono 8y ago
  While I quite like the concept, I think there's something of a catch 22 here. 'Fact-based' generally implies 'lesser impact', it's way easier to catch eyeballs with fakes and click-bait than with better stories, and part of the fake news issues is that professional success in journalism is dependent on impact, not on accuracy (a similar thing goes on in academia). You'll have to figure out a way to increase the impact of writers with more accuracy (less negative fact checks maybe?) by promoting them in some shape or form. Have you considered top writers scoreboards or something similar?

queercode 8y ago

The hover state reminds me of the lyric annotation system Genius (https://genius.com/ https://genius.com/) has, which is a good thing.

ahnick 8y ago

IMHO reputation is important to build into the base level for a platform like this. Allowing anon registrations and commenting will eventually lead to a lot of noise and abuse of the platform for misinformation. I think some sort of verification of a person's qualifications on particular subject areas and then allowing them to only "fact check" on those qualified areas is required here to have an effective platform that will scale well.

YazIAm 8y ago
Currently (for the prototype), you can start contributing, but your reviews only become public once I've vetted you and given you a fact checking test. Long term the goal is to have an open reputation score for every fact checker. I'm currently experimenting with different implementations of this, and seeing good promise.
- hamitron 8y ago
  What credentials do you have to properly vet people?
  - YazIAm 8y ago
    Well first, the point of this type of open fact checking is you can always verify everything for yourself, so you don't need to trust my credentials. Working on this over the last few years though, I've personally fact checked over 500 assertions.
    - adamsea 8y ago
      I would respectfully disagree. Introduce enough noise into the system and the cost of verification becomes so high that doing so is unrealistic for many or most people. Also, on first glance the app looks great, well-polished, and the UX of reading an article and looking at sources for claim is nice :). Thanks for working on and sharing this!
    - SamBam 8y ago

nahimn 8y ago

Finally something viable to combat fake news

v_lisivka 8y ago
Russian government backed trolls will use this system to spread fakes instead. Wikipedia has similar system and Russians abuse it a lot.
- Fnoord 8y ago
  Can you back this up with examples?
  - v_lisivka 8y ago
    Examples of hoaxes backed by Russian government trolls, or examples of abuse of Wikipedia and other sites by Russian government trolls? I have both.
    - Fnoord 8y ago
      The latter.
murph-almighty 8y ago
I'm cautiously optimistic. I like that this emphasizes use of primary sources, but 1) Some primary sources are dubious, or manipulated. See the multiple versions of Acosta getting kicked out of a White House press briefing. 2) What happens when you have an article that repeatedly cites dubious secondary opinion pieces, but the reader accepts these pieces as fact?
- YazIAm 8y ago
  1) Now that video deepfake is a thing, I currently only allow videos as primary sources in rare circumstances (such as the video being published by a verified account of the subject). 2) Only primary sources are accepted. Currently anyone can start contributing, but every claim is still reviewed by a team of verified fact checkers. I hope to scale this long term with a fact checking reputation score.

jaredwiener 8y ago

How would a reporter use this? Do you need to have an article already written/published? Does it have to be on your site? It seems like its less for fact checking, which would happen prior to publication, than for fact proof?

YazIAm 8y ago
I'm happy to provide any journalist/blogger who wants to include this style of annotation on their site a js plug-in to do just that. The whole fact checking process actually happens within the system, the facts you see on there were collaboratively fact checked by a team of 7 to 11 vetted fact checkers. So that may also provide an incentive for journalists.

jasonlotito 8y ago

How does this handle anonymous sources? Not only do you have to account for what those sources say, but also, a reporter reporting what an anonymous source said can be 100% factual (Yes, the anonymous, valid source was indeed reported accurately), but maybe the anonymous source is incorrect. The reporting is accurate. The source is not. Basically, a lot of grey area. And with that comes the issue of anything not "fact checked" by a service like this could be dismissed. What harm does that do? Not a dismissal of a service like this. Rather, just open questions.

YazIAm 8y ago
Only public primary sources are allowed as evidence on SourcedFact. Anonymous sources certainly have value within journalism, but the goal of SourcedFact is to make facts that can be proven beyond a reasonable doubt with public primary sources stand out from everything else.
- jasonlotito 8y ago
  Okay. I think as long as that distinction is clear, sounds good!

johnm 8y ago

How are you going to deal with underlying "sources" where the content disappears or is changed out from under the fact checkers?

YazIAm 8y ago
Every source document submitted to the platform is automatically saved to the webarchive, and the webarchive snapshot is used :)
- 7373737373 8y ago
  Definitely have a look at https://tlsnotary.org/ https://tlsnotary.org/ !

teleclimber 8y ago

Nice to see more work in this area. However as others have said a reputation system for participants is absolutely key. Any platform like this has to assume it is going to be flooded by bots trying to influence opinion on behalf of corporations, foreign governments and activists. A reputation where users gain reputation slowly (through work, by verifying sources, etc..) become more able to influence opinion, but the reputation drops quickly if they are found to be misleading people. How this actually works is harder than it seems if you want a truly robust system. When I think of platforms like this I think of Galileo[0]. He was perceived as a troll or heretic in his time, but of course he was right. How do you create a platform that allows correct ideas to flourish even when they go against conventional wisdom? [0] https://en.wikipedia.org/wiki/Galileo_Galilei https://en.wikipedia.org/wiki/Galileo_Galilei

YazIAm 8y ago
I 100% agree that the reputation system will be they key to scaling this. I'm currently seeing promise with my experiments with reputation scoring. As for "How do you create a platform that allows correct ideas to flourish even when they go against conventional wisdom?" Great question, however my goal here is a bit more modest, it's to create a platform that allows facts which are provable by public primary sources to stand out from everything else. I think a platform that focuses on simply that can add a lot of value to political discourse.
- SamBam 8y ago
  The bots would be a serious issue if this ever became big enough, since they can add to each other's reputation. Reddit probably has thousands of code-hours devoted to this, and bots are still a problem. There is also the problem that humans will probably tend to upvote "facts" that they agree with, and visa-versa. I'm not sure that reputation is the solution. And yet "expert" verification introduces trust. I'm not sure what the solution is.
  - teleclimber 8y ago
    Yes it's a super tough problem, not to be underestimated by anybody working on such a platform. If there is any easily automatable way gaining reputation, then the bots will use that, and if you can easily pass reputation too, then the rep system won't hold. > I'm not sure what the solution is. Ditto, and I've ben pondering this on and off for years :/

elliekelly 8y ago

This reminds me of Grasswire: https://news.ycombinator.com/item?id=7954327 https://news.ycombinator.com/item?id=7954327

jellicle 8y ago

So, your examples are all "reporting about court cases" and you use court documents as your citations for facts. Court cases are as good as we have for determining facts, I suppose. (Though the error rate in court cases is quite high!) But 99.9999% of reporting isn't about court cases. Most reporting isn't about issues that will ever be litigated, and even those things that will be litigated, haven't been at the time of the reporting. "Mitch McConnell had oatmeal for breakfast this morning, according to sources." There, that's a fact. Now, tell me which court case I can refer to in order to verify that fact? Fact-checking is an impossible endeavor. The essence of journalism is reporting things no one knows. That's what news is! This is completely at odds with "fact-checking". A good news story CANNOT be fact-checked by the public because if the public knew it, it wouldn't be news. There is an extremely limited genre of "news by trawling public but little-known documents", such as reporting on old court cases. Almost zero news fits into that category.

jpttsn 8y ago
Call me a pedant, but I'd say OP is enabling "consensus checking" I find the present day's fact checking wave is a misnomer. In a world where something is false, but sourced, most of what we call fact-checking would only reinforce mistruths. e.g. The ratio of circumference to diameter is exactly three to one [1][2][3] [1]: the Bible. [2]: pious scholars. [3]: low-tech experiments. Would be cool to see someone think carefully about what "real" "fact checking" would be. First-hand sources? Links to instructions for replicating findings? Consensus checking is of course also an interesting problem, albeit a pretty different one.
- NateEag 8y ago
  Offtopic, but the Bible does not actually use a value of 3 for pi: https://www.purplemath.com/modules/bibleval.htm https://www.purplemath.com/modules/bibleval.htm

m-i-l 8y ago

See also https://fullfact.org https://fullfact.org which sponsored an Apache Solr hackathon a couple of years back.

deepstream 8y ago

I'm surprised that FAGAM haven't got into the crowd-sourced fact checking space. Seems like a great way to build a knowledge engine / AI, by getting humans to "connect the facts" while motivating (herding) them with the idea they are doing it to "preserve truth in an era of fake narratives", when actually they are just helping build an AI.

rohan1024 8y ago

We need this decentralised

YazIAm 8y ago
That's the long term goal!

sidhantgandhi 8y ago

By the way, clicking on a fact on Chrome iOS is not loading anything.

YazIAm 8y ago
Will investigate, thanks for letting me know!
jason_slack 8y ago
Safari on iOS isn’t working for me.

thebeefytaco 8y ago

This is pretty awesome. Any time I open any article, the first thing I do is try to follow the trail of links/sources back to the primary sources and read that instead. If I can't find one, I don't trust the article.

YazIAm 8y ago
I'm the exact same way, that's why I built this :)

eeZah7Ux 8y ago

"open source" is not the right term and also is not a verb. "transparency" is the word you are looking for. It's more general and predates "open source" by far.

0xffff2 8y ago
"Open source" is absolutely a verb. "I open sourced my latest work project this month". It's also a term that seems to be used similarly to the post title in contexts outside of software. For example, I subscribe to a podcast that covers arms control, where they frequently use the term "open source" to describe analysis derived publicly available sources and methods.

idlewords 8y ago

What problem does this solve that existing reputation systems don't? When I read a New Yorker article, for example, part of what my subscription is paying for is considerable skilled labor by a group of in-house fact checkers. How does moving this work to an outside free platform improve the situation? Moreover, how do you replicate work like calling sources to verify statements before publication? I wrote a short piece for Wired once and had to go back and forth with their fact checker on the most picayune details. In an article about balancing online work and travel I had mentioned in passing searching under a hotel bed for an outlet, and she wanted to know what hotel it was, what country it was in, when I stayed there, etc. The process was exhaustive and not something I would go through with a bunch of Internet randos. It was also done before publication and involved a fairly high level of trust.

YazIAm 8y ago
The goal is to capture the fact checking process and enable any reader to review it for themselves. Certainly some publications currently have an excellent fact checking process, but very few make their fact checking open and transparent. As for "how do you replicate work like calling sources to verify statements before publication?" Good question. Currently this is only designed to open up the fact checking which involves public primary sources. I think that a platform that focuses on doing that subset of fact checking really really well can add a lot of value to the public discourse.
- jessaustin 8y ago
  Congratulations on this; it looks great. Currently this is only designed to open up the fact checking which involves public primary sources. Limiting the scope to this is wise. It's a tractable problem and it's better journalism. The types of journalists who write "this unnamed source implied that this other unnamed source had implied to another unnamed source..." articles won't like this sort of tool, but those journalists should be encouraged to change careers. Some might not realize that articles like that are fairly common. For an example, I went to NYT homepage and this was the first article: https://www.nytimes.com/2019/04/03/us/politics/william-barr-mueller-report.html https://www.nytimes.com/2019/04/03/us/politics/william-barr-... First sentence: "Some of Robert S. Mueller III’s investigators have told associates that Attorney General William P. Barr failed to adequately portray the findings of their inquiry and that they were more troubling for President Trump than Mr. Barr indicated, according to government officials and others familiar with their simmering frustrations." This sort of crap gets swept under the rug within a couple of weeks, but we're certainly paying attention to it now.

IronWolve 8y ago

Wikipedia talk pages are a perfect example of facts ignored and removed from articles to push an agenda. You can see how a narrative can be pushed with half-truths and selective editing. Same thing is happening in News, its agenda driven, not truth driven. Entertainment opinion pieces that pretends to be news. People are just bending the facts to support their personal politics. If Wikipedia can't be factually accurate due to editors personal politics, how do you keep those personal politics out of a fact checker?

YazIAm 8y ago
You're 100% correct in that one can cite completely valid facts and still push an agenda by omitting other facts. I'm actually working on a "fact comment section" feature where any reader can pin facts to an article (facts that still go through the same rigorous and open fact checking process as everything else). Facts which readers deem are important extra context. However, I'd argue that even without such a feature, having an open standard for facts openly verified from primary sources will add a lot of value to current public discourse.

hoaxcracker 8y ago

I have been thinking about something similar but stuck on a different project. Great work!

rootusrootus 8y ago

Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.

sanbor 8y ago
But it might be worth a try. Wikipedia does pretty well imho but if someone would told me the idea I would never thought that it would work.
- dfgert 8y ago
  Wikipedia has flaws,too. 1. Your account need to have good history or social repo for your edits to be seriously considered by Wikipedia admins. 2. Admin can have their own set of bias and can decide to ignore edits. Remember Wikipedia is a centralized and under control of single entity, fact that they rely on donation doesn't change anything.
  - stingraycharles 8y ago
    Just because something has flaws doesn’t mean it’s not a good idea to do it.
    - jtuente 8y ago
      There wasn't anything said about Wikipedia being a bad idea, but rather that it shouldn't be your sole source for fact-checking. I've learned that people only trust Wikipedia to the point at which it confirms their biases.
      - penagwin 8y ago
        Part of the problem is that Wikipedia is generally pretty darn accurate. Everybody always talks about funny vandalism stories, but generally it's pretty good. The issue of course is that there's parts of it that definitely aren't correct. This makes it easy to say "Well if it's accurate 99% of the time then it must confirm what I thought!".

nilskidoo 8y ago

How we've existed for so long without this is beyond me. Thank you, to the creators of this.

dsjoerg 8y ago

YazIAm, you should check out https://www.kialo.com https://www.kialo.com and https://web.hypothes.is/journalism/ https://web.hypothes.is/journalism/ and https://www.procon.org/ https://www.procon.org/

jshowa3 8y ago

While I like how the software ties together the sourcing, I feel like the following is lacking (and I realize this is an alpha, but its things to consider) 1. Anyone who answers a question regarding whether the claim is "fully" proven by a source or whether the source is valid for the claim must or should (debatable) provide proof of their own interpretation by citing either specific instances in the source in question supporting it. For example, in the Net Neutrality article, Yatz states this: The adopted principles in this statement are at the top of page 3 in a response to does the source fully prove the claim. It would be nice if that was linked to the actual location in the document or identified through highlighting or some other mechanism to get full context for the response. There also should be uses for outside sources to be attached when evaluating these questions that are suppose to validate a source. This is because someone could submit a response to these source validity questions and cite an external source or internal quote of the source in question for their reasoning that doesn't support their answer. 2. Having some sort of expertise verification is extremely important and should be weighted either separately as in Experts: Yes Experts: No category and should be more important. It distinguishes this from random anon answers that have no training in how the source should be interpreted. 3. Having anonymous people do this seems rather dubious and the current platform seems like it would make it prone to Youtube comment syndrome. Maybe have some reputation system to gate keep like StackOverflow? 4. Sources usually require appropriate interpretation in order to be taken in the right context and be considered correct. Sometimes there's no single source or any source, philosophically, that fully proves a claim. I'm guessing that's what supporting documents are for? 5. Why is the original articles claim allowed to be edited?

catchmeifyoucan 8y ago

For Mobile, I didn’t know what I was looking for. The Navigation bar should be fixed to the top.

snambi 8y ago

This is really great idea. Today I don't trust the journalists, but if they provide references atleast there is a bit a scope for trusting a particular article. ofcourse, provided the references can stand the scrutiny.

Swquenzer 8y ago

Cool platform. One suggestion - create a unique url/route for each article. I wanted to send a particular article to a friend, but realized there's no way to provide a unique link.

YazIAm 8y ago
Thanks :) Indeed they have unique urls, I'll see if I can make that clearer on the landing page: Net Neutrality - https://sourcedfact.com/a/net-neutrality-how-we-got-here https://sourcedfact.com/a/net-neutrality-how-we-got-here Cell Phone Privacy - https://sourcedfact.com/a/cell-location-data-without-warrant https://sourcedfact.com/a/cell-location-data-without-warrant Israel Boycotts - https://sourcedfact.com/a/government-measures-against-israel-boycotts https://sourcedfact.com/a/government-measures-against-israel...

hkdobrev 8y ago

Is this pravduh? :D Source threads: - https://twitter.com/elonmusk/status/1000072092769964033 https://twitter.com/elonmusk/status/1000072092769964033 - https://twitter.com/elonmusk/status/1000080833678528512 https://twitter.com/elonmusk/status/1000080833678528512 https://pravduh.com https://pravduh.com

YazIAm 8y ago
I actually got a friend who works at Tesla to email Musk about SourcedFact when he was tweeting about that haha, busy guy though ;)

geekfreakdev 8y ago

give beer to this guy!

vonnik 8y ago

This is really interesting. It reminds me a bit of peer-reviewing journalism via Web annotation: https://web.hypothes.is/ https://web.hypothes.is/ I have a couple comments to make as a former newspaper reporter: * Many important stories use anonymous sources that are impossible to verify via crowd-sourcing or by linking to official documents. Much of what happens in the world is not publicly documented nor is it widely known (until a reporter publishes a story). That gap between events and any record of those events is one of the structural problems that allow "fake news" to occur. * Timing matters. Reporters working for the dwindling number of publications that employ fact-checkers or editors have their facts checked before they publish. For that small group of outlets, this diminishes the number of falsehoods they are responsible for unleashing online. Fact-checking a story after its release is useful, but the cat's already out of the bag. The lie is already speeding around the world at the speed of viral outrage. * As with any platform where better information leads people to make different choices, there are lots of incentives for publications not engage in crowd-sourced fact-checking. Why do people lie or distort the facts on their dating profiles and job applications? Why do companies put forward their most utopian face? They are trying to shape the world with information. And so are media outlets such as Fox News, to name just the most egregious. * Fact-checking is a lot of work, and can descend into epistemological quibbles. Most people don't have time do engage in it, or even to understand the debate around a given fact. This is why, in the past, we outsourced this work to the editorial staff of the fourth estate. While this would bring more transparency to the process, I am not sure who would have time to take advantage of that transparency, anymore than the normal reader will closely follow the "talk" tab on a Wikipedia article.

chw9e 8y ago

Would be nice to avoid having to open another tab to do fact checking - having some kind of a summary or something beyond "10 verifications" would at least let me know if opening the new tab is really worth it. Ideally the article and all verifications would live in one tab.

zachguo 8y ago
Also, instead of a dedicated site, a browser extension allowing you to select text then right-click to get facts would work better. https://web.hypothes.is/journalism/ https://web.hypothes.is/journalism/

beat 8y ago

Awkward question here... can anyone cite an example of a major-media news story that was actually factually wrong in a way that "open source fact checking" could have detected it? Just because a bunch of people believe it's a problem doesn't mean it's a problem. I find the bias, when it exists, is in the presentation, not the facts themselves. And mainstream journalism is really committed to getting the facts right.

jimmy1 8y ago
The most recent and obvious case in my mind: Covington Catholic Incident: almost every major mainstream media outlet ran with the narrative that the boys were harassing the Native Americans which was based solely on one video clip. No traditional investigative work was done. No one was interviewed (until the damage had been done). Nothing was fact checked. It was just one video, and boom: instant headlines.
- beat 8y ago
  I disagree. Whether they were harassing is a matter of opinion. Whether there was a confrontation is a matter of fact, confirmed by the video and photos. If you want to claim the "narrative", you need a link to an actual article with actual quotes. That's how fact-checking works, and your response is failing it badly.
  - ajvs 8y ago
    This has been debunked thoroughly. That guy went up to the kids and started beating his drum right in their faces to intimidate them and somehow the media painted the kids as the harasser.
    - beat 8y ago
      Was his intent to intimidate? Why do you say that? How is that a "fact"? Where did you even hear that story? Geez, y'all don't understand fact checking at all.

amirouche 8y ago

well done, thanks for sharing.

_mitch 8y ago

Just a couple quick feedback points: I'd prefer if the currently selected fact text remained highlighted when the mouse is not hovered over it. That way you know what the active "Investigate >" link is referring to. Also, for accessibility and convenience, it would be nice if you could tab through the instances of the fact text blocks.

YazIAm 8y ago
Thanks for the feedback, the second (accessibility) should be a quick fix. I definitely also want to address the first issue, the tricky part is coming up with a solution that also works when there are multiple assertions on one line, but I'll give it more thought.

devinplatt 8y ago

Hey everybody, I just want to point out that I'm working on a similar project to document evidence for and against claims: https://www.wikiclaim.org https://www.wikiclaim.org The tech is dumb (MediaWiki on Heroku), but it's been sufficient to allow me to spend several hundred hours creating probably about 200 hundred wiki pages so far.

smenis 8y ago
Thanks for posting this, I really enjoyed browsing Wikiclaim and it feels to me like I've never seen anywhere on the Internet that centralises this kind of content (commonly heard claims with info for/against)... Having said that, the usual strategy when I hear a new claim I'm not sure about is to google it; so is the Google search engine itself the competition for this site? It does try to give an "answer" at the top of search results these days. Then there is Quora. The main advantage Wikiclaim has is that, like Wikipedia, the info can be moderated and updated, and hopefully can become an improving source of truth. Godspeed, devinplatt!
- devinplatt 8y ago
  Thanks Smenis! In time I hope that Google will surface Wikiclaim higher in its search results (whether URL or even answer box), so in my mind the competitor is also the ally :) I don't think the strong suit of Quora (information based personal experiences) overlaps with what I'm trying to do with Wikiclaim (information from numerous reliable sources, editable by anyone), so no issues there either.

uberdru 8y ago

Coz that's what we hire journalists for. . . .

volt4ire 8y ago

Sadly this "platform to open source factchecking" is not itself open source, as the site's source code is not available and the user content license restricts commercial use (see point 3 https://sourcedfact.com/terms_of_service https://sourcedfact.com/terms_of_service and https://blog.okfn.org/2010/06/24/why-share-alike-licenses-are-open-but-non-commercial-ones-arent/ https://blog.okfn.org/2010/06/24/why-share-alike-licenses-ar...)

vortico 8y ago
That's fine with me. The two concepts of "open-source" at hand are drastically different. The lack of ability to verify the website's source code doesn't prevent me in any way from seeing its content which helps me verify journalist claims. I hope the author is able to grow their service through marketing, licensing, and contracts with media agencies. Open-sourcing the service's software would be a distraction from that goal and would create fragmentation of its database when clones of the service are created. What's important for a service like this is open data, not necessarily an open-source stack.
- nautilus12 8y ago
  Except we are talking about journalism which is prone to things like censorship, shadow banning and the like which would defeat the purpose of it being open. The source code would reveal if there were such a functionality.
  - YazIAm 8y ago
    I intend to move user contributed fact verifications to some sort of distributed ledger model to ensure transparency. Still in alpha though, so lots of work ahead.

iandanforth 8y ago

This is reminiscent of how some scientific papers are written. You might be able to pick up some tips from those processes as well. For example: - Write each sentence on a different line. This allows for better change tracking, commentary etc in a number of tools including github. - Inline sources and compile into a references section. I want to see that a claim is backed by more than one piece of supporting work whenever possible. - Have a compiled / formatted version for holistic evaluation. After you've done the hard work of writing factual statements the piece also needs to be readable. This is an orthogonal view into the same information however and should be presented as such.

rfeather 8y ago

I like the idea and genuinely hope that fact checking becomes more mainstream. Seriously, kudos to the author for building something. But what I'm not getting is why a news platform would use this, unless this is intending on becoming a news platform itself? I think there is an underlying assumption that the general world cares about things like "primary sources" and "logical fallacies" enough to bother hovering over the text or viewing the content, and that there isn't an intentional manipulation of these things by media organizations to fit a narrative. Maybe if this was a browser plugin that came with <major browser> by default and automatically highlighted fact checked statements in exiting articles. That way folks wouldn't have to opt in, but as it is, whats the incentive for either the populace or the media to participate? Think of it this way- how did the fact checking in the examples get done? Someone spent 2 minutes on their favorite search engine. People willing to do that will do it if they have the time, and people not willing probably aren't interested in information that would contradict their opinions anyway - unless it comes from a source they already trust, like someone in their bubble. My own experience with this is that the folks who will care exist but are rare. Others, you can literally watch their eyes glaze over the moment you introduce a little cognitive dissonance.

Zoo3y 8y ago

Where are you pulling the PDFs of FCC documents from?

indymike 8y ago

First, love the project. Hope it works and makes journalism better. Even if it doesn't it will be interesting to see how much the news industry sees value in crowd-sourced fact-checking. Good luck!

anigbrowl 8y ago

Promising!

abathur 8y ago

Interesting work, YazIAm. Conceptually this reminds me a little bit of the idea behind Kialo. Down the road, there might be some interesting synergy potential between fact attestation and debate. I've had a few related thoughts, on the off chance you find them useful: 1. It would be interesting, variously for journalists and the reading public, if it was possible for journalists to generate, disclose, and research cryptographic sourcing identifiers that enable them to figure out when they share a source. This could, critically, help journalists identify sources with a record of feeding other journalists bad information. It'd be nice if the same work could help the rest of us unravel citation chains based on a small number of unique sources, but I'm not sure there's a way to achieve that without it being fairly simple to re-identify sources via brute-force. 2. If we can drill enough to find some bedrock, I think it might be useful to have virtuous-cycle user tools, like browser extensions that warn users when visiting an article by writers/publications (and potentially even sources, per above) with repeated sourcing/attestation problems. I don't think of this as purely a journalistic thing. In the sciences it could apply to methodology, data-collection/statistical integrity. With some bedrock, tooling in place, and established trust/community practice in place here, it might also be possible to expand the scope a little and address things like headlines that aren't at all moored to evidence.

mikedilger 8y ago

I appreciate the recognition that there is a problem in journalism. But I'm afraid that the fundamental problems are problems in the nature of humans and aren't ameniable to this sort of solution. Those problems include: 1) Many people choose to believe what feels right to them, rather than dispassionately seeing things for what they are. Unfortunately there are a great many of people that do this regularly, and they have practiced this their entire lives. Richard Feynman cautioned us in saying that "The first principle is that you must not fool yourself - and you are the easiest person to fool." That caution was aimed at scientists who are already among the most objective of us, yet they can still suffer from this ailment. 2) When people talk about the facts, they rarely just lay out the facts dispassionately and without judgement. Humans need a motivation to tell a story, and most people's motivations include convincing other people to support their ideology, or else berating those who dont. Facts are organized within a mental framework of the ideology of the teller, wrapped up in beliefs, desires, and biases andcarry additional information about which ideological position you ought to hold -- sometimes they are even intended to mislead and manipulate the listener. The modern news media is composed of businesses in search of profits. That goal is often not aligned with a straight reporting of facts. By pandering to an audiences ideology they increase ratings. By reporting on salacious and exciting news, even if incorrect, they get ratings. Buyer beware.

Razengan 8y ago
3) Whenever there is a chance to gain wealth and/or influence without immediate disadvantages, humans will usually take it.
ARandomerDude 8y ago
Indeed. The heart is deceitful above all things, and desperately sick; who can understand it?
- Jenz 8y ago
  > desperately sick How so?
  - 8y ago

jammygit 8y ago

What would you say if a well known, analytical journalist approached you to be able to perform fact checking, but you knew they were openly biased on many topics?

aleister_777 8y ago

The main fault I see is how do we augment your "facts" with other "facts". For instance, I clicked on Net Neutrality. It's tough to truly appreciate the "net neutrality" dynamic of the past few years without understanding the backstory of Reed Hastings battling the ILECs (https://www.dslreports.com/shownews/Netflix-CEO-Comcast-Wants-Whole-Internet-to-Pay-Them-129154 https://www.dslreports.com/shownews/Netflix-CEO-Comcast-Want...), buddying up to Obama (https://www.businessinsider.com/house-of-cards-obama-2013-12 https://www.businessinsider.com/house-of-cards-obama-2013-12), getting the law changed in his favor (https://www.nytimes.com/2015/03/13/technology/fcc-releases-net-neutrality-rules.html https://www.nytimes.com/2015/03/13/technology/fcc-releases-n...), and now rewarding Obama in kind: https://www.nytimes.com/2018/05/21/us/politics/barack-obama-netflix-show.html https://www.nytimes.com/2018/05/21/us/politics/barack-obama-.... It's almost as if it had nothing to with actual "net neutrality" at all since none of the horrible things have come to pass that were predicted. Broadband isn't as good as it could or should be, but it isn't getting worse.

yantra_ml 8y ago

This is really cool!!

enriquto 8y ago

But journalism is not only stating true facts, but choosing a limited set of facts to present. You can pretty well do fake news by stating a list of true facts (and omitting a few other, fundamental, true facts). Example: Published news : person A killed person B by pushing a knife on his stomach, at a time when person B was unarmed. Non-published statements : just before being killed, person B and a group of three friends armed with knifes attacked person A, which was unarmed at the time, and started cutting him. During the struggle, person A managed to get hold of B's knife and fought back, harming B. The friends of B were scared and ran away. Person A finally survived his injuries, but person B didn't. I suppose everybody agrees that publishing the first statement alone is a clear example of fake news.

GistNoesis 8y ago

Little feedback : Nice. Please add a vertical splitter in the verify page so that the verification is not constrained to a small margin on the right, which make it not easily readable. It would also be nice to have some color coding when reading articles, so you can know what's the majority opinion on the validity of the piece. Hopefully once you have gathered enough data, you can get one of these NLP bots to check the fact in an even more neutral way.

vegangaijin 8y ago

I <3 this.

alvalentini 8y ago

I think that the primary value of this platform would reside on making the fact-checking available and shareable to non-journalists.

JoelMcCracken 8y ago

This reminds me a bit of an application I wanted to build many years ago: A platform for collections of arguments. For example, the question "are humans contributing to global warming?" Allow crowd-submitted answers. Collect information about facts cited (similar to how this works). Mostly-rational people can be swayed by evidence and arguments, yet often disagree with each other because they doubt the truth of the other side's statements (fact checking), they believe the other side has missed important context in their argument, the other side has implicit assumptions that they disagree with (this is something that should be explored as a sub issue, then). I'm very happy to see this. I'd love to see more of this kind of thing in the future.

devinplatt 8y ago
Hi Joel. In case you didn't see my top level comment (it's buried quite low), I'm working on a similar project to document evidence for and against claims: https://www.wikiclaim.org https://www.wikiclaim.org The tech is dumb (just MediaWiki on Heroku), but it's been sufficient to allow me to spend several hundred hours creating about 200 wiki pages so far.

cphoover 8y ago

Love it... Good job.

Siira 8y ago

The site doesn’t work in iOS safari? I can’t get sources for the claims.

VvR-Ox 8y ago

This is a very good idea but I'm afraid it won't play along the human condition. People construct reality with things they want to believe. Most people aren't interested and/or skilled enough to be reasonable and think logically. How could someone like that orange ape become POTUS???? This is just one major indicator for the fall of reason. The enlightenment has failed and instead of a mass of mature citizens who are able to govern the world via consensus like the democratic system needs it to be we are driven by emotions. Keep on doing this and I hope it may still have some impact.