4 ms·

What is the error rate of fake news over verified (real) news? A 5% margin of error could be acceptable. A 50% error rate would indicate a serious problem. > A

What is the error rate of fake news over verified (real) news? A 5% margin of error could be acceptable. A 50% error rate would indicate a serious problem. > As part of a larger audit of Facebook’s Trending topics, the Intersect logged every news story that trended across four accounts during the workdays from Aug. 31 to Sept. 22. During that time, we uncovered five trending stories that were indisputably fake and three that were profoundly inaccurate. Yes, but out of how many Trending stories in total? Eight fake stories over a three week period seems like an acceptable rate of error. Trending stories are popular stories. A fake story has an element of satire. These stories could be popular because of the satire (and the subsequent humor). Trending does not imply veracity.

deleted 10y ago
[deleted]
dredmorbius 10y ago
This raises the really interesting point that news doesn't have a post mortem process. It would be a very interesting experiment to, say, revisit a sampling of top stories at some interval (say 3-6 months) and compare the reported facts (and omissions) with the actual reality. From such elements as "did names and places get accurately reported?" to "was the underlying narrative correct?". Quotes and attributions might be a particular element. Predictions (particularly in financial reporting) as well. This needn't be every article, but some weighted sample based on a story's placement, impact, subsequent references, etc.
- internaut 10y ago
  That would be wonderful! I hope you're fireproof though, because that would make a lot of people very angry. Not to disabuse you, because I do think it is a good idea, but many people are simply not interested in asking the right questions. I used to belong to a millennial cult which had formerly been proven wrong about 'The End of the World' half a dozen times in the 20th century, and each time they decide the past prophesies were metaphorical while next time it is literally the 'Big One'. Each time they do this they do lose a lot of support, so disseminating information about what happened in the past is not completely meaningless. I wouldn't have left the Truman Show without the Net I think, that and books. > was the underlying narrative correct? This is the one I want to have done most of all. The others you mentioned are easier to do but this is the crux of it. The cult believers treat every news articles as further evidence of their faith, and every discrepancy as a test from the Devil/oppositional lies. There is some deep stuff here. I remember reading something from a person who lived under Communism, and like some kind of conspiracy theorist they had every detail about their 'Matrix' wrong but they did know instinctively that something was very wrong because of the mismatch between reports and anecdotal observations.
  - dredmorbius 10y ago
    Pity you're downvoted here, as you're raising some good points. Yes, the activity would be quite incendiary, though that's somewhat the case for the news as it is. There is Project Censored, at Sonoma State University in California, which looks at under-reported stories. Cults and their own belief systems is a whole 'nother fascinating area. Have you written on your own experiences on this? On questions and answers I also agree -- I think we're pursuing a lot of the wrong questions, and the answers to the wrong questions are simply meaningless. I'm looking for a few good questions: https://redd.it/1v052g https://redd.it/1v052g Assessing narrative means some way of scoring a narrative. That's an interesting problem in itself. On your "Matrix" example: I'm convinced that one of the key fundamentals of an accurate model of the universe (e.g., a true or useful narrative) is that your different perceptions all tend to correspond. It's when you're getting different messages in different places, or from different sources, that you've got a problem. Mind, there are circumstances in which there is no one true narrative -- optical illusions such as the Necker cube, "dancing lady" illusion, or the rabbit-duck picture come to mind. But there's a huge realm in which a fairly accurate understanding is possible.
    - internaut 10y ago
      > Pity you're downvoted here, as you're raising some good points. I'm beginning to believe the slightest hint of negativity encourages downvotes irrespective of whether the context is appropriate or not. > Cults and their own belief systems is a whole 'nother fascinating area. Have you written on your own experiences on this? A little. Getting out of a cult you're born into is an experience like no other. Many people become zealots of another sort, a different religion or a secular kind. The nearest experience most people will have is finding that somebody they loved has betrayed them. It's a funny thing you know, I had great anxiety about leaving, but when I left I felt a peculiar joy I cannot adequately describe here. > I'm looking for a few good questions: https://redd.it/1v052g https://redd.it/1v052g I think I was reading your subreddit before last year. How is it going? Have you defined a terminal goal for the question-making or is it a philosophical work in progress? > I think we're pursuing a lot of the wrong questions, and the answers to the wrong questions are simply meaningless. I don't think there's a faster way to get diminishing returns than to ask the wrong question. I have suspected for a long time that many things we ask governments to do are mutually contradictory. The effective policy then, is not a government policy of direction and action, but to allow us to join the dots to see that our efforts are in vain. I envisage a real time system available in each home (optional!) which makes requests and takes requests from householders. Think of a StackExchange, only for the government and citizens to be able to interrelate frictionlessly. A newsfeed of items that actually effect you personally would be terrific by itself. Simple questions, like "How do I buy land and what regulations do I need to know about" and we already appear to be in the domain of experts. That isn't right. You shouldn't have to wade through a library of content on some obscure website hardly anybody visits. > Assessing narrative means some way of scoring a narrative. That's an interesting problem in itself. I think a narrative assessment looking for internal consistency would be an enormous improvement. However; there is a troll/prankster on reddit called vargas who posts these internally consistent but increasingly weird stories. It's the same thing that fools Facebook's trending feature into posting fake stories. That would mean to get somewhere you'd need to store a compare and contrast model of some sort. > On your "Matrix" example: I'm convinced that one of the key fundamentals of an accurate model of the universe (e.g., a true or useful narrative) is that your different perceptions all tend to correspond. It's when you're getting different messages in different places, or from different sources, that you've got a problem. That would make a good heuristic for increasing our confidence. You could achieve scaling by invoking human computation such as a kind of capcha that give users little 'stories', and ask them whether they are consistent. I am confident that will work because if your proposition can be shorn of context (events happening to mythical people in mythical lands) people think logically for the most part. > But there's a huge realm in which a fairly accurate understanding is possible. Take the Podesta Emails recently. Somebody was asking whether they were real or not. You should be able to verify whether they are real or a work of fiction with such a system. Since lies get harder to cover for the more they multiply, my guess is that this 'verifier' would work.
- ptaipale 10y ago
  >This raises the really interesting point that news doesn't have a post mortem process. Great idea, and what higher education should be doing for research. Generally there's no process. But personal experience is that whenever I have been involved in a situation about which news has been made, the outcome has always been that the reporting is somewhat sloppy, inaccurate, and agenda-driven. This is with so-called reputable journalism; "fake news" is then even worse. So, even if I do a non-professional post-mortem on my own, I see that the news was not completely healthy. And pointing this out to journalists generally makes them just angry. Even if you are completely polite, they block you on Twitter, etc. If you comment a news article in reputable newspapers' web sites to correct errors with good sources, it's not unusual that the comment is moderated away.
  - dredmorbius 10y ago
    I need to think myself on just what metrics ought be used. I find, for example, looking at fact-checking efforts, that the focus is often at too fine a level of detail. That by focusing minutely on facts, the big-picture accuracy (narrative) is often lost. The question of how to deal with missing narrative or omitted detail is also huge. There are efforts underway in that regard, usually in the legal field. The Innocence Project would be an example. I can think of a great deal of follow-up to the 2003 Iraq invasion which have turned up since, and in particular numerous instances of people caught blatently lying. The public though seems mostly to move on. I feel that's a mistake.

deleted 10y ago

[deleted]

dredmorbius 10y ago

This raises the really interesting point that news doesn't have a post mortem process. It would be a very interesting experiment to, say, revisit a sampling of top stories at some interval (say 3-6 months) and compare the reported facts (and omissions) with the actual reality. From such elements as "did names and places get accurately reported?" to "was the underlying narrative correct?". Quotes and attributions might be a particular element. Predictions (particularly in financial reporting) as well. This needn't be every article, but some weighted sample based on a story's placement, impact, subsequent references, etc.

internaut 10y ago
That would be wonderful! I hope you're fireproof though, because that would make a lot of people very angry. Not to disabuse you, because I do think it is a good idea, but many people are simply not interested in asking the right questions. I used to belong to a millennial cult which had formerly been proven wrong about 'The End of the World' half a dozen times in the 20th century, and each time they decide the past prophesies were metaphorical while next time it is literally the 'Big One'. Each time they do this they do lose a lot of support, so disseminating information about what happened in the past is not completely meaningless. I wouldn't have left the Truman Show without the Net I think, that and books. > was the underlying narrative correct? This is the one I want to have done most of all. The others you mentioned are easier to do but this is the crux of it. The cult believers treat every news articles as further evidence of their faith, and every discrepancy as a test from the Devil/oppositional lies. There is some deep stuff here. I remember reading something from a person who lived under Communism, and like some kind of conspiracy theorist they had every detail about their 'Matrix' wrong but they did know instinctively that something was very wrong because of the mismatch between reports and anecdotal observations.
- dredmorbius 10y ago
  Pity you're downvoted here, as you're raising some good points. Yes, the activity would be quite incendiary, though that's somewhat the case for the news as it is. There is Project Censored, at Sonoma State University in California, which looks at under-reported stories. Cults and their own belief systems is a whole 'nother fascinating area. Have you written on your own experiences on this? On questions and answers I also agree -- I think we're pursuing a lot of the wrong questions, and the answers to the wrong questions are simply meaningless. I'm looking for a few good questions: https://redd.it/1v052g https://redd.it/1v052g Assessing narrative means some way of scoring a narrative. That's an interesting problem in itself. On your "Matrix" example: I'm convinced that one of the key fundamentals of an accurate model of the universe (e.g., a true or useful narrative) is that your different perceptions all tend to correspond. It's when you're getting different messages in different places, or from different sources, that you've got a problem. Mind, there are circumstances in which there is no one true narrative -- optical illusions such as the Necker cube, "dancing lady" illusion, or the rabbit-duck picture come to mind. But there's a huge realm in which a fairly accurate understanding is possible.
  - internaut 10y ago
    > Pity you're downvoted here, as you're raising some good points. I'm beginning to believe the slightest hint of negativity encourages downvotes irrespective of whether the context is appropriate or not. > Cults and their own belief systems is a whole 'nother fascinating area. Have you written on your own experiences on this? A little. Getting out of a cult you're born into is an experience like no other. Many people become zealots of another sort, a different religion or a secular kind. The nearest experience most people will have is finding that somebody they loved has betrayed them. It's a funny thing you know, I had great anxiety about leaving, but when I left I felt a peculiar joy I cannot adequately describe here. > I'm looking for a few good questions: https://redd.it/1v052g https://redd.it/1v052g I think I was reading your subreddit before last year. How is it going? Have you defined a terminal goal for the question-making or is it a philosophical work in progress? > I think we're pursuing a lot of the wrong questions, and the answers to the wrong questions are simply meaningless. I don't think there's a faster way to get diminishing returns than to ask the wrong question. I have suspected for a long time that many things we ask governments to do are mutually contradictory. The effective policy then, is not a government policy of direction and action, but to allow us to join the dots to see that our efforts are in vain. I envisage a real time system available in each home (optional!) which makes requests and takes requests from householders. Think of a StackExchange, only for the government and citizens to be able to interrelate frictionlessly. A newsfeed of items that actually effect you personally would be terrific by itself. Simple questions, like "How do I buy land and what regulations do I need to know about" and we already appear to be in the domain of experts. That isn't right. You shouldn't have to wade through a library of content on some obscure website hardly anybody visits. > Assessing narrative means some way of scoring a narrative. That's an interesting problem in itself. I think a narrative assessment looking for internal consistency would be an enormous improvement. However; there is a troll/prankster on reddit called vargas who posts these internally consistent but increasingly weird stories. It's the same thing that fools Facebook's trending feature into posting fake stories. That would mean to get somewhere you'd need to store a compare and contrast model of some sort. > On your "Matrix" example: I'm convinced that one of the key fundamentals of an accurate model of the universe (e.g., a true or useful narrative) is that your different perceptions all tend to correspond. It's when you're getting different messages in different places, or from different sources, that you've got a problem. That would make a good heuristic for increasing our confidence. You could achieve scaling by invoking human computation such as a kind of capcha that give users little 'stories', and ask them whether they are consistent. I am confident that will work because if your proposition can be shorn of context (events happening to mythical people in mythical lands) people think logically for the most part. > But there's a huge realm in which a fairly accurate understanding is possible. Take the Podesta Emails recently. Somebody was asking whether they were real or not. You should be able to verify whether they are real or a work of fiction with such a system. Since lies get harder to cover for the more they multiply, my guess is that this 'verifier' would work.
ptaipale 10y ago
>This raises the really interesting point that news doesn't have a post mortem process. Great idea, and what higher education should be doing for research. Generally there's no process. But personal experience is that whenever I have been involved in a situation about which news has been made, the outcome has always been that the reporting is somewhat sloppy, inaccurate, and agenda-driven. This is with so-called reputable journalism; "fake news" is then even worse. So, even if I do a non-professional post-mortem on my own, I see that the news was not completely healthy. And pointing this out to journalists generally makes them just angry. Even if you are completely polite, they block you on Twitter, etc. If you comment a news article in reputable newspapers' web sites to correct errors with good sources, it's not unusual that the comment is moderated away.
- dredmorbius 10y ago
  I need to think myself on just what metrics ought be used. I find, for example, looking at fact-checking efforts, that the focus is often at too fine a level of detail. That by focusing minutely on facts, the big-picture accuracy (narrative) is often lost. The question of how to deal with missing narrative or omitted detail is also huge. There are efforts underway in that regard, usually in the legal field. The Innocence Project would be an example. I can think of a great deal of follow-up to the 2003 Iraq invasion which have turned up since, and in particular numerous instances of people caught blatently lying. The public though seems mostly to move on. I feel that's a mistake.