3 ms·

I’ve often thought maybe AI will end up helping here. It might be possible to build an AI that can take in all these biased stories and digest them into somethi

by gfodor 4y ago

I’ve often thought maybe AI will end up helping here. It might be possible to build an AI that can take in all these biased stories and digest them into something that actually has more signal than noise. Lots of possibilities: try to write a de-biased version, annotate the stories to point out the bias and rhetorical tricks used, find the set of facts that are the same across all sources and distill them, etc. not a silver bullet but seems like a rich space for exploring how to turn the garbage produced by the media into signal by leveraging the fact that these are adversarial organizations that do have some predictable constraints on what they will say.

asow92 4y ago
Why would the public trust an AI over an editorial team? Couldn't it be just as biased?
kevingadd 4y ago
How is an AI automatically going to turn bad data into good data? When in history has this been the case? So far we haven't managed to magically create an AI that can turn huge piles of bad data into good data, you see all sorts of stories about machine learning problems caused by misclassified training data, etc. Bad news stories and opinion pieces feel close to a worst case here since so many of them are subtly wrong in ways that would make it hard to even tag them clearly and accurately at scale for use as a training set. And at the end what's the AI going to do? Tag up every news story so that 60% of the text is yellow or red indicating that all of it is misleading or a lie? If an article quotes "sources" how is AI going to determine whether or not the sources are informed and whether the author is selectively quoting them?
- WalterSear 4y ago
  You train it on last year's media, on articles and news where the actual situation is now clearer.
  - kevingadd 4y ago
    Knowing the fact a year later doesn't tell you whether a given piece was good or bad journalism, just whether it was correct. That's very different. Are you suggesting that if you train a network on last year's media plus the state of the world a year after, we'll be able to teach an AI to predict the future? The problems are around the reporting process and the reporter's motives/influences, not whether everything they say turns out to be 100% accurate. Often the facts on the ground change or the information everyone has is inaccurate, and you want reporting that clearly communicates this ambiguity and isn't actively misleading you.
- gfodor 4y ago
  The bad vs good data dichotomy is a false frame. The problem with the media isn't that it's zero signal, its that it's low signal. Fortunately there's also signal in the differences between the reporting from the various outlets, and there's also signal in the repitition across reporting from the same outlet. For example, the more media outlets of varying political slants agree on a specific facet of a story, all other things being equal, the more likely it's true. And from there, if you can put together a firm set of things that seem highly likely to be true, you can have a scaffold to hang everything else on in a way that might be useful for understanding where the uncertainty is and its magnitude. It's this integration process that I think can possibly result in useful artifacts. It can be (and is) done by humans, of course, but at a coarse grain and is fundamentally resource constrained. I suspect if this kind of process could be systematized and cover all outputs by all media outlets, that could be something materially better. A lot of the resistence to what I'm saying stems from jumping to conclusions around what, exactly, these artifacts would be. The point would be to try many different kinds of artifacts. The ones that are obviously a bad idea are obviously bad ideas. But there are many that are not obviously good or obviously bad, such as having AI introduce additional cross-references, applying labels to various forms of rhetoric that are used to trick people, etc. Also, I said "AI" not "machine learning" and not "classifiers" and not "neural networks." I'm talking in this case very literally about AI in the broad sense.

by gfodor 4y ago

asow92 4y ago

Why would the public trust an AI over an editorial team? Couldn't it be just as biased?

kevingadd 4y ago

How is an AI automatically going to turn bad data into good data? When in history has this been the case? So far we haven't managed to magically create an AI that can turn huge piles of bad data into good data, you see all sorts of stories about machine learning problems caused by misclassified training data, etc. Bad news stories and opinion pieces feel close to a worst case here since so many of them are subtly wrong in ways that would make it hard to even tag them clearly and accurately at scale for use as a training set. And at the end what's the AI going to do? Tag up every news story so that 60% of the text is yellow or red indicating that all of it is misleading or a lie? If an article quotes "sources" how is AI going to determine whether or not the sources are informed and whether the author is selectively quoting them?

WalterSear 4y ago
You train it on last year's media, on articles and news where the actual situation is now clearer.
- kevingadd 4y ago
  Knowing the fact a year later doesn't tell you whether a given piece was good or bad journalism, just whether it was correct. That's very different. Are you suggesting that if you train a network on last year's media plus the state of the world a year after, we'll be able to teach an AI to predict the future? The problems are around the reporting process and the reporter's motives/influences, not whether everything they say turns out to be 100% accurate. Often the facts on the ground change or the information everyone has is inaccurate, and you want reporting that clearly communicates this ambiguity and isn't actively misleading you.
gfodor 4y ago
The bad vs good data dichotomy is a false frame. The problem with the media isn't that it's zero signal, its that it's low signal. Fortunately there's also signal in the differences between the reporting from the various outlets, and there's also signal in the repitition across reporting from the same outlet. For example, the more media outlets of varying political slants agree on a specific facet of a story, all other things being equal, the more likely it's true. And from there, if you can put together a firm set of things that seem highly likely to be true, you can have a scaffold to hang everything else on in a way that might be useful for understanding where the uncertainty is and its magnitude. It's this integration process that I think can possibly result in useful artifacts. It can be (and is) done by humans, of course, but at a coarse grain and is fundamentally resource constrained. I suspect if this kind of process could be systematized and cover all outputs by all media outlets, that could be something materially better. A lot of the resistence to what I'm saying stems from jumping to conclusions around what, exactly, these artifacts would be. The point would be to try many different kinds of artifacts. The ones that are obviously a bad idea are obviously bad ideas. But there are many that are not obviously good or obviously bad, such as having AI introduce additional cross-references, applying labels to various forms of rhetoric that are used to trick people, etc. Also, I said "AI" not "machine learning" and not "classifiers" and not "neural networks." I'm talking in this case very literally about AI in the broad sense.