4 ms·

Now let's run this experiment against the editorial boards in newsrooms. Obviously, AI isn't an improvement, but people who blindly trust the news have always

Now let's run this experiment against the editorial boards in newsrooms. Obviously, AI isn't an improvement, but people who blindly trust the news have always been credulous rubes. It's just that the alternative is being completely ignorant of the worldviews of everyone around you. Peer-reviewed science is as close as we can get to good consensus and there's a lot of reasons this doesn't work for reporting.

n4r9 1y ago
I guess the claim is not that rubes did not used to exist, but rather that technology is increasingly encouraging and streamlining rubism.
- MangoToupe 1y ago
  I agree with that assessment, or at least that this is indeed the claim. But, technology also gave us the internet, and social media. Yes, both are used to propagate misinformation, but it also laid bare how bad traditional media was at both a) representing the world competently and b) representing the opinions and views of our neighbors. Manufacturing consent has never been so difficult (or, I suppose, so irrelevant to the actions of the states that claim to represent us).
  - intended 1y ago
    Technology has been used to absolutely decimate the news media. Organizations like Fox have blazed the path forward for how news organizations succeed in the cable and later internet worlds. You just give up on uneconomical efforts at accuracy and you sell narratives that work for one political party or the other. It is a model that has been taken up world over. It just works. “The world is too complex to explain, so why bother?” And what will you or me do about it? Subscribe to the NYT? Most of us would rather spend that money on a GenAI subscription because that is bucketed differently in our heads.
- walkabout 1y ago
  I decided about a decade ago that McLuhan was a prophet, and that the “message” of the Internet may not include compatibility with democracy, as it turns out.
raincole 1y ago
Yep. How could a candidate who yelling "Fake News" like an idiot get elected? Because of the state of journalism. How could people turn to AI slop? Because of the state of human slop.
falcor84 1y ago
> Peer-reviewed science is as close as we can get to good consensus I think we're on the same side of this, but I just want to say that we can do a lot better. As per studies around the Replication Crisis over the last decade [0], and particularly this 2016 survey conducted by Monya Baker from Nature [1]: > 1,576 researchers who took a brief online questionnaire on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers, 67% of medical researchers, 64% of earth and environmental scientists, and 62% of all others), and more than half have failed to reproduce their own experiments. We need to expect better, needing both better incentives and better evaluation, and I think that AI can help with this. [0] https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis [1] https://www.nature.com/articles/533452a https://www.nature.com/articles/533452a
vidarh 1y ago
> Now let's run this experiment against the editorial boards in newsrooms. Or against people in general. It's a pet peeve of mine that we get these kinds of articles without a baseline established of how people do on the same measure. Is misrepresenting news content 45% of the time better or worse than the average person? I don't know. By extension: Would a person using an AI assistant misrepresent news more or less after having read a summary of the news provided by an AI assistant? I don't know that either. When they have a "Why this distortion matters" section, those things matter. They've not established if this will make things better or worse. (the cynic in me want another question answered too: How often does reporters misrepresent the news? Would it be better or worse if AI reviewed the facts and presented them vs. letting reporters do it? again: no idea)
- JumpCrisscross 1y ago
  > It's a pet peeve of mine that we get these kinds of articles without a baseline established of how people do on the same measure I don’t have a personal human news summarizer? The comparison is between a human reading the primary source against the same human reading an LLM hallucination mixed with an LLM referring the primary source. > cynic in me want another question answered too: How often does reporters misrepresent the news? The fact that you mark as cynical a question answered pretty reliably for most countries sort of tanks the point.
  - vidarh 1y ago
    > I don’t have a personal human news summarizer? Not a personal one. You do however have reporters sitting between you and the source material a lot of the time, and sometimes multiple levels of reporters playing games of telephone with the source material. > The comparison is between a human reading the primary source against the same human reading an LLM hallucination mixed with an LLM referring the primary source. In modern news reporting, a fairly substantial proportion of what we digest is not primary sources. It's not at all clear whether an LLM summarising primary sources would be better or worse than reading a reporter passing on primary sources. And in fact, in many cases the news is not even secondary sources - e.g. a wire service report on primary sources getting rewritten by a reporter is not uncommon. > The fact that you mark as cynical a question answered pretty reliably for most countries sort of tanks the point. It's a cynical point within the context of this article to point out that it is meaningless to report on the accuracy of AI in isolation because it's not clear that human reporting is better for us. I find it kinda funny that you dismiss this here, after having downplayed the games of telephone that news reporting often is earlier in your reply, thereby making it quite clear I am in fact being a lot more cynical than you about it.

n4r9 1y ago

I guess the claim is not that rubes did not used to exist, but rather that technology is increasingly encouraging and streamlining rubism.

MangoToupe 1y ago
I agree with that assessment, or at least that this is indeed the claim. But, technology also gave us the internet, and social media. Yes, both are used to propagate misinformation, but it also laid bare how bad traditional media was at both a) representing the world competently and b) representing the opinions and views of our neighbors. Manufacturing consent has never been so difficult (or, I suppose, so irrelevant to the actions of the states that claim to represent us).
- intended 1y ago
  Technology has been used to absolutely decimate the news media. Organizations like Fox have blazed the path forward for how news organizations succeed in the cable and later internet worlds. You just give up on uneconomical efforts at accuracy and you sell narratives that work for one political party or the other. It is a model that has been taken up world over. It just works. “The world is too complex to explain, so why bother?” And what will you or me do about it? Subscribe to the NYT? Most of us would rather spend that money on a GenAI subscription because that is bucketed differently in our heads.
walkabout 1y ago
I decided about a decade ago that McLuhan was a prophet, and that the “message” of the Internet may not include compatibility with democracy, as it turns out.

raincole 1y ago

Yep. How could a candidate who yelling "Fake News" like an idiot get elected? Because of the state of journalism. How could people turn to AI slop? Because of the state of human slop.

falcor84 1y ago

> Peer-reviewed science is as close as we can get to good consensus I think we're on the same side of this, but I just want to say that we can do a lot better. As per studies around the Replication Crisis over the last decade [0], and particularly this 2016 survey conducted by Monya Baker from Nature [1]: > 1,576 researchers who took a brief online questionnaire on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers, 67% of medical researchers, 64% of earth and environmental scientists, and 62% of all others), and more than half have failed to reproduce their own experiments. We need to expect better, needing both better incentives and better evaluation, and I think that AI can help with this. [0] https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis [1] https://www.nature.com/articles/533452a https://www.nature.com/articles/533452a

vidarh 1y ago

> Now let's run this experiment against the editorial boards in newsrooms. Or against people in general. It's a pet peeve of mine that we get these kinds of articles without a baseline established of how people do on the same measure. Is misrepresenting news content 45% of the time better or worse than the average person? I don't know. By extension: Would a person using an AI assistant misrepresent news more or less after having read a summary of the news provided by an AI assistant? I don't know that either. When they have a "Why this distortion matters" section, those things matter. They've not established if this will make things better or worse. (the cynic in me want another question answered too: How often does reporters misrepresent the news? Would it be better or worse if AI reviewed the facts and presented them vs. letting reporters do it? again: no idea)

JumpCrisscross 1y ago
> It's a pet peeve of mine that we get these kinds of articles without a baseline established of how people do on the same measure I don’t have a personal human news summarizer? The comparison is between a human reading the primary source against the same human reading an LLM hallucination mixed with an LLM referring the primary source. > cynic in me want another question answered too: How often does reporters misrepresent the news? The fact that you mark as cynical a question answered pretty reliably for most countries sort of tanks the point.
- vidarh 1y ago
  > I don’t have a personal human news summarizer? Not a personal one. You do however have reporters sitting between you and the source material a lot of the time, and sometimes multiple levels of reporters playing games of telephone with the source material. > The comparison is between a human reading the primary source against the same human reading an LLM hallucination mixed with an LLM referring the primary source. In modern news reporting, a fairly substantial proportion of what we digest is not primary sources. It's not at all clear whether an LLM summarising primary sources would be better or worse than reading a reporter passing on primary sources. And in fact, in many cases the news is not even secondary sources - e.g. a wire service report on primary sources getting rewritten by a reporter is not uncommon. > The fact that you mark as cynical a question answered pretty reliably for most countries sort of tanks the point. It's a cynical point within the context of this article to point out that it is meaningless to report on the accuracy of AI in isolation because it's not clear that human reporting is better for us. I find it kinda funny that you dismiss this here, after having downplayed the games of telephone that news reporting often is earlier in your reply, thereby making it quite clear I am in fact being a lot more cynical than you about it.