4 ms·

> What can we do? Upvotes/downvotes? Won't work at scale, because likes/shares have a connotation of legitimacy. News verification seems like the perfect appli

by sedachv 10y ago

> What can we do? Upvotes/downvotes? Won't work at scale, because likes/shares have a connotation of legitimacy. News verification seems like the perfect application of NLP, AI, and data mining techniques. Can it make any money for anyone? Right now it is much more profitable to make fake news and use data mining to manipulate people: http://www.bloomberg.com/politics/features/2015-11-12/is-the-republican-party-s-killer-data-app-for-real- http://www.bloomberg.com/politics/features/2015-11-12/is-the...

yosito 10y ago
I don't think those technologies are well suited to this problem at all. I was just talking to some Swiss journalists last night. You know what Swiss news organizations are using those technologies for? To make their writing more "engaging", i.e. more sensationalist, not for fact-checking.
- sedachv 10y ago
  > I don't think those technologies are well suited to this problem at all. How so? Deciding whether news is fake or not seems like a textbook application of natural language classification and search.
  - OJFord 10y ago
    I haven't read any textbooks on the subject (I don't mean that facetiously) - what would this look like? Because it seems to me that to such an algorithm virality ~ legitimacy.
  - inimino 10y ago
    The only difference between a fake news story and a real news story that you can rely on is that the events in the real news story actually took place. Obviously there is no way to determine that by looking at the text itself. Yes, at present the fake news may be written more colorfully or poorly. Obviously if you filter on that difference, it will go away.
    - sedachv 10y ago
      How do you personally tell if a news story is suspicious? The publication it originated in, the places where it gets syndicated, shared, and re-posted, the style of the writing, and how sensational the content is. Authors that specialize in writing fake news stories will also have an identifiable individual writing style. There are a lot of signals you can use to classify news as fake or real, and that is before going to databases like Snopes. This is IMO a much easier problem than a lot of the NLP work for sentiment analysis and other marketing stuff that is working or being funded today. The problem is not technical, the problem is that fake news is profitable.
      - inimino 10y ago
        Suspicious, sure, but that's moving the goalposts. And all of these signals exist at present, but as soon as Facebook starts suppressing posts based on machine classification then obviously fake news publishers will adapt and it won't work anymore. The only reliable identifying factor of fake news is that it reports facts that aren't. This obviously cannot be detected by text analysis. People who are suggesting technical solutions just haven't thought past the very first step in the dance. To speculate, perhaps part of it a lack of respect for the hard work of real journalism, education, and geniune political dialogue, so the fantasy of surfacing all the crap from social media and then extracting the ground truth with a magical algorithm appeals to some people despite being a completely hopeless idea. What you're actually arguing for is the negative twin of today's filter bubble effect. Algorithms already prioritize things you'll like, but now that the SV elite is shocked (shocked!) by how that worked out, there is a crisis of fake conscience. Obviously the solution is to also filter out the things that you'll think are fake. Or the things that "look fake" to Facebook's algorithm. It's concerning to me that the relatively intelligent folks here could be taking this at all seriously.
        sedachv 10y ago
        > And all of these signals exist at present, but as soon as Facebook starts suppressing posts based on machine classification then obviously fake news publishers will adapt and it won't work anymore. The only reliable identifying factor of fake news is that it reports facts that aren't. Replace "news" with "email" and your argument remains the same. But spam filtering is working and working well right now with less signals to classify on. > To speculate, perhaps part of it a lack of respect for the hard work of real journalism, education, and geniune political dialogue, so the fantasy of surfacing all the crap from social media and then extracting the ground truth with a magical algorithm appeals to some people despite being a completely hopeless idea. Why are you ranting about synthesizing real news from social media? The problem I specifically mentioned was identifying fake news articles. > What you're actually arguing for is the negative twin of today's filter bubble effect. If you do not understand the difference between recommendation and classification you should not be arguing right now. It is a subtle and important distinction. Your argument is about as subtle as "computers can't solve it!" > It's concerning to me that the relatively intelligent folks here could be taking this at all seriously. In what substantive way is "fake news trying to get you to vote for a particular politician" different from "fake email trying to get you to buy fake pills"?
  - yosito 10y ago
    AI has not advanced to the point where it can go out into the real world and verify whether or not events actually took place.
    - sedachv 10y ago
      It is not hard to see some statement about or purportedly by an organization, government office, or celebrity and then call the organization, government office, or celebrity media contact to verify that information. Anonymous sources are protected by journalists and in that case you will have to have a trust metric for the journalist writing the story.

by sedachv 10y ago

yosito 10y ago

I don't think those technologies are well suited to this problem at all. I was just talking to some Swiss journalists last night. You know what Swiss news organizations are using those technologies for? To make their writing more "engaging", i.e. more sensationalist, not for fact-checking.

sedachv 10y ago
> I don't think those technologies are well suited to this problem at all. How so? Deciding whether news is fake or not seems like a textbook application of natural language classification and search.
- OJFord 10y ago
  I haven't read any textbooks on the subject (I don't mean that facetiously) - what would this look like? Because it seems to me that to such an algorithm virality ~ legitimacy.
- inimino 10y ago
  The only difference between a fake news story and a real news story that you can rely on is that the events in the real news story actually took place. Obviously there is no way to determine that by looking at the text itself. Yes, at present the fake news may be written more colorfully or poorly. Obviously if you filter on that difference, it will go away.
  - sedachv 10y ago
    How do you personally tell if a news story is suspicious? The publication it originated in, the places where it gets syndicated, shared, and re-posted, the style of the writing, and how sensational the content is. Authors that specialize in writing fake news stories will also have an identifiable individual writing style. There are a lot of signals you can use to classify news as fake or real, and that is before going to databases like Snopes. This is IMO a much easier problem than a lot of the NLP work for sentiment analysis and other marketing stuff that is working or being funded today. The problem is not technical, the problem is that fake news is profitable.
    - inimino 10y ago
      Suspicious, sure, but that's moving the goalposts. And all of these signals exist at present, but as soon as Facebook starts suppressing posts based on machine classification then obviously fake news publishers will adapt and it won't work anymore. The only reliable identifying factor of fake news is that it reports facts that aren't. This obviously cannot be detected by text analysis. People who are suggesting technical solutions just haven't thought past the very first step in the dance. To speculate, perhaps part of it a lack of respect for the hard work of real journalism, education, and geniune political dialogue, so the fantasy of surfacing all the crap from social media and then extracting the ground truth with a magical algorithm appeals to some people despite being a completely hopeless idea. What you're actually arguing for is the negative twin of today's filter bubble effect. Algorithms already prioritize things you'll like, but now that the SV elite is shocked (shocked!) by how that worked out, there is a crisis of fake conscience. Obviously the solution is to also filter out the things that you'll think are fake. Or the things that "look fake" to Facebook's algorithm. It's concerning to me that the relatively intelligent folks here could be taking this at all seriously.
      - sedachv 10y ago
        > And all of these signals exist at present, but as soon as Facebook starts suppressing posts based on machine classification then obviously fake news publishers will adapt and it won't work anymore. The only reliable identifying factor of fake news is that it reports facts that aren't. Replace "news" with "email" and your argument remains the same. But spam filtering is working and working well right now with less signals to classify on. > To speculate, perhaps part of it a lack of respect for the hard work of real journalism, education, and geniune political dialogue, so the fantasy of surfacing all the crap from social media and then extracting the ground truth with a magical algorithm appeals to some people despite being a completely hopeless idea. Why are you ranting about synthesizing real news from social media? The problem I specifically mentioned was identifying fake news articles. > What you're actually arguing for is the negative twin of today's filter bubble effect. If you do not understand the difference between recommendation and classification you should not be arguing right now. It is a subtle and important distinction. Your argument is about as subtle as "computers can't solve it!" > It's concerning to me that the relatively intelligent folks here could be taking this at all seriously. In what substantive way is "fake news trying to get you to vote for a particular politician" different from "fake email trying to get you to buy fake pills"?
- yosito 10y ago
  AI has not advanced to the point where it can go out into the real world and verify whether or not events actually took place.
  - sedachv 10y ago
    It is not hard to see some statement about or purportedly by an organization, government office, or celebrity and then call the organization, government office, or celebrity media contact to verify that information. Anonymous sources are protected by journalists and in that case you will have to have a trust metric for the journalist writing the story.