3 ms·

Reinventing the wheel, or in this case, reinventing the square wheel. Wikitribune solves a problem, just not the problem they have defined as the target. They

Reinventing the wheel, or in this case, reinventing the square wheel. Wikitribune solves a problem, just not the problem they have defined as the target. They've used a naive view of the problem; the model under this ignores the existence of antagonists and too much faith in crowd sourcing difficult problems. Antagonists will prey on services like this, and off the top of my head, here's 2 ways in which such a service can be made biased. 1) baseless accusations, oft repeated. Find the facts inimical to your (the antagonists) position. Ignore them. Find facts which are borderline, and have dog whistle properties - highlight these facts ad nauseum. Say that "Wikitribune is biased". Repeat till it sticks. Then target the facts inimical to you. 2) flood the service with facts that serve your cause- humans have only so much working memory. ---- The particular structure wikitribune has chosen, will result in issues. There's a reason print news papers had an editor and a whole staff dedicated to working together. With volunteers there's no structure, and that causes failures, just consider the Boston bomber case. Of course with a journalist in the mix the assumption is that they will push back. But the structure is supposedly egalitarian, which just means that this is going to end up causing the same politicking, and admin arguing that plagues Wikipedia. Recruit everyone, don't get volunteers. Get the whole team. > Articles are authored, fact-checked, and verified by professional journalists and community members working side by side as equals, and supported not primarily by advertisers, but by readers who care about good journalism enough to become monthly supporters The wisdom of the crowd fails all too often. As another article recently discussed, it's 5% of the people that take up most of your time. How will this structure deal with truly divisive news articles? Or people who have conflicts (and conflicts of interest) within the group? How will you deal with the fact that one day someone can say "volunteer X was a pedophile from <country>!" Kudos for trying it. This looks like a propaganda machine which will use the wiki brand about to be born.

NovaS1X 10y ago
I'd rather give this a shot than just shut it down with the cynical negativity that we oft see on HN. There are obvious faults like you have pointed out, but we have to try something to get us out of the the situation we're in. This is the first honest effort I've seen that has some credibility.
- intended 10y ago
  There's many efforts which have credibility, and just because it has jimmy wales name on it doesn't make it credible. Heres a previous discussion on the "fake news" problem with a far more credible start, because theyve actually got a better idea of the problem, and so are targetting an achievable goal - http://www.fakenewschallenge.org/ http://www.fakenewschallenge.org/ discussed here: https://news.ycombinator.com/item?id=13542428 https://news.ycombinator.com/item?id=13542428, in February. This isn't cynicism. I'm long long past that. This current media scenario has been cooking for more than 2 decades. Creating a less strict media structure will not fix it.
  - backpropaganda 10y ago
    Fake news is not a tech problem. It's better solved by incentive structures than nonexistant AI.
    - intended 10y ago
      Which is a conclusion shared to an extent by the fake challenge, as stated on the front page I linked >Assessing the veracity of a news story is a complex and cumbersome task, even for trained experts [3]. >Fortunately, the process can be broken down into steps or stages. A helpful first step towards identifying fake news is to understand what other news organizations are saying about the topic. We believe automating this process, called Stance Detection, could serve as a useful building block in an AI-assisted fact-checking pipeline. So stage #1 of the Fake News Challenge (FNC-1) focuses on the task of Stance Detection. In a nutshell, this interaction is the problem with the news cycle, and its the audience as much as its the reporting. Most people don't click on the link. --- From the FAQ, lower down the page - > WHY DID YOU CHOOSE THE STANCE DETECTION TASK RATHER THAN THE TASK OF LABELING A CLAIM, HEADLINE OR STORY TRUE/FALSE, WHICH SEEMS TO BE WHAT THE FAKE NEWS PROBLEM IS ALL ABOUT? ANSWER: There are several reasons Stance Detection makes for a good first task for the Fake News Challenge: Our extensive discussions with journalists and fact checkers made it clear both how difficult “truth labeling” of claims really is, and how they’d rather have reliable semi-automated tool to help them in do their job better rather than fully-automated system whose performance will inevitably fall far short of 100% accuracy. Truth labeling also poses several large technical / logistical challenge for a contest like the FNC: There exists very little labeled training data of fake vs. real news stories. The data that does exist (e.g. fact checker website archives) is almost all copyright protected. The data that does exist is extremely diverse and unstructured, making hard to train on. Any dataset containing claims with associated “truth” labels is going to be contested as biased. Together these make the truth labeling task virtually impossible with existing AI / NLP. In fact, even people have trouble distinguishing fake news from real news. The dataset we are using to support the Stance Detection task for FNC-1 was created by accredited journalists, making it both high quality and credible. It is also in the public domain. Variants of the FNC-1 Stance Detection task have already been explored and proven feasible but far from trivial by Andreas Vlachos & his students from U. of Sheffield. Cite: Ferreira & Vlachos (2016) & Augenstein et al. (2016). We considered targeting the truth labeling task for the FNC-1, but without giving teams any labeled training data. We decided against it both because we thought a competition with a more traditionally structured Machine learning tasks would appeal to more teams, and because such an open-ended truth labeling competition was recently completed, called the Fast & Furious Fact Check Challenge. Our discussions with human fact checkers lead us to believe that a solution to the stance detection problem could form the basis of a useful tool for real-life human fact checkers. Also see, next question/answer. thats a substantially more thought out approach to the problem than Wikitribune.
tmalsburg2 10y ago
It's a bit disingenuous to assume that they are ignoring the problem of adversarial actors. Jimmy Wales, through his experience with Wikipedia, is probably more aware of these issues – and potential solutions – than pretty much anyone else on this planet. Also, your arguments could equally be used to argue that something like Wikipedia cannot work but we know that this is not the case. Is this going to be easy? No. Is it worth trying? Most definitely.
- intended 10y ago
  But thats the key difference - wikipedia is not the news, and the news is not an encyclopedia. This is essentially the same sales pitch that gave us bloggers as journalists, which we already know did not create an alternative to the system (well depending on your perspective I guess) There's many things worth trying, but as I said - this set up solves a problem, just not the problem at the heart of their mission statement
  - tmalsburg2 10y ago
    I think Wales recognized the key difference between Wikipedia and news and that's precisely why he's is proposing a new format that is distinctly different from Wikipedia's format and from blogs. Their mission statement is the following: > We want to make sure that you read fact-based articles that have a > real impact in both local and global events. And that stories can be > easily verified and improved. You earlier said that adversaries will just repeat falsehoods until they are perceived as being true but even endless repetition will not make sources and evidence magically appear. Could you please elaborate on why you think they will fail to achieve their goals?
    - intended 10y ago
      1) I have provided examples of how adversaries can manipulate the naive implementation described above. Countering one example shouldn't be seen as countering the class. 2) to answer your specific question - self referential information loops already exist. News paper makes a claim, claim is put onto Wikipedia. That claim is repeated by other websites, which in then use wikipedia as the source. How will wiki tribune solve this? By doing longform research? Going out in the field and finding the facts? That's not what this setup is good for. It assumes that fact checkers can check facts, at the speed of the current news cycle. Fact is that this verification can't be done fast enough and broad enough to cover ground to be useful, unless it gets a multi billion dollar investment.

NovaS1X 10y ago

I'd rather give this a shot than just shut it down with the cynical negativity that we oft see on HN. There are obvious faults like you have pointed out, but we have to try something to get us out of the the situation we're in. This is the first honest effort I've seen that has some credibility.

intended 10y ago
There's many efforts which have credibility, and just because it has jimmy wales name on it doesn't make it credible. Heres a previous discussion on the "fake news" problem with a far more credible start, because theyve actually got a better idea of the problem, and so are targetting an achievable goal - http://www.fakenewschallenge.org/ http://www.fakenewschallenge.org/ discussed here: https://news.ycombinator.com/item?id=13542428 https://news.ycombinator.com/item?id=13542428, in February. This isn't cynicism. I'm long long past that. This current media scenario has been cooking for more than 2 decades. Creating a less strict media structure will not fix it.
- backpropaganda 10y ago
  Fake news is not a tech problem. It's better solved by incentive structures than nonexistant AI.
  - intended 10y ago
    Which is a conclusion shared to an extent by the fake challenge, as stated on the front page I linked >Assessing the veracity of a news story is a complex and cumbersome task, even for trained experts [3]. >Fortunately, the process can be broken down into steps or stages. A helpful first step towards identifying fake news is to understand what other news organizations are saying about the topic. We believe automating this process, called Stance Detection, could serve as a useful building block in an AI-assisted fact-checking pipeline. So stage #1 of the Fake News Challenge (FNC-1) focuses on the task of Stance Detection. In a nutshell, this interaction is the problem with the news cycle, and its the audience as much as its the reporting. Most people don't click on the link. --- From the FAQ, lower down the page - > WHY DID YOU CHOOSE THE STANCE DETECTION TASK RATHER THAN THE TASK OF LABELING A CLAIM, HEADLINE OR STORY TRUE/FALSE, WHICH SEEMS TO BE WHAT THE FAKE NEWS PROBLEM IS ALL ABOUT? ANSWER: There are several reasons Stance Detection makes for a good first task for the Fake News Challenge: Our extensive discussions with journalists and fact checkers made it clear both how difficult “truth labeling” of claims really is, and how they’d rather have reliable semi-automated tool to help them in do their job better rather than fully-automated system whose performance will inevitably fall far short of 100% accuracy. Truth labeling also poses several large technical / logistical challenge for a contest like the FNC: There exists very little labeled training data of fake vs. real news stories. The data that does exist (e.g. fact checker website archives) is almost all copyright protected. The data that does exist is extremely diverse and unstructured, making hard to train on. Any dataset containing claims with associated “truth” labels is going to be contested as biased. Together these make the truth labeling task virtually impossible with existing AI / NLP. In fact, even people have trouble distinguishing fake news from real news. The dataset we are using to support the Stance Detection task for FNC-1 was created by accredited journalists, making it both high quality and credible. It is also in the public domain. Variants of the FNC-1 Stance Detection task have already been explored and proven feasible but far from trivial by Andreas Vlachos & his students from U. of Sheffield. Cite: Ferreira & Vlachos (2016) & Augenstein et al. (2016). We considered targeting the truth labeling task for the FNC-1, but without giving teams any labeled training data. We decided against it both because we thought a competition with a more traditionally structured Machine learning tasks would appeal to more teams, and because such an open-ended truth labeling competition was recently completed, called the Fast & Furious Fact Check Challenge. Our discussions with human fact checkers lead us to believe that a solution to the stance detection problem could form the basis of a useful tool for real-life human fact checkers. Also see, next question/answer. thats a substantially more thought out approach to the problem than Wikitribune.

tmalsburg2 10y ago

It's a bit disingenuous to assume that they are ignoring the problem of adversarial actors. Jimmy Wales, through his experience with Wikipedia, is probably more aware of these issues – and potential solutions – than pretty much anyone else on this planet. Also, your arguments could equally be used to argue that something like Wikipedia cannot work but we know that this is not the case. Is this going to be easy? No. Is it worth trying? Most definitely.

intended 10y ago
But thats the key difference - wikipedia is not the news, and the news is not an encyclopedia. This is essentially the same sales pitch that gave us bloggers as journalists, which we already know did not create an alternative to the system (well depending on your perspective I guess) There's many things worth trying, but as I said - this set up solves a problem, just not the problem at the heart of their mission statement
- tmalsburg2 10y ago
  I think Wales recognized the key difference between Wikipedia and news and that's precisely why he's is proposing a new format that is distinctly different from Wikipedia's format and from blogs. Their mission statement is the following: > We want to make sure that you read fact-based articles that have a > real impact in both local and global events. And that stories can be > easily verified and improved. You earlier said that adversaries will just repeat falsehoods until they are perceived as being true but even endless repetition will not make sources and evidence magically appear. Could you please elaborate on why you think they will fail to achieve their goals?
  - intended 10y ago
    1) I have provided examples of how adversaries can manipulate the naive implementation described above. Countering one example shouldn't be seen as countering the class. 2) to answer your specific question - self referential information loops already exist. News paper makes a claim, claim is put onto Wikipedia. That claim is repeated by other websites, which in then use wikipedia as the source. How will wiki tribune solve this? By doing longform research? Going out in the field and finding the facts? That's not what this setup is good for. It assumes that fact checkers can check facts, at the speed of the current news cycle. Fact is that this verification can't be done fast enough and broad enough to cover ground to be useful, unless it gets a multi billion dollar investment.