10 ms·

CNET's AI journalist appears to have committed extensive plagiarism

132 points by iamflimflam1 4y ago

noworld 4y ago
AI gets more human-like every day.
reset-password 4y ago
The examples provided are not that convincing. Especially the two headlines. Yes they are similar because there are a limited number of ways to ask "Can You Buy a Gift Card With a Credit Card?". Should the topic be off limit now that it's been written about? Take the code equivalent of asking 100 people to write a function in secret that takes 2 ints and returns a sum. I suspect a number of those code blocks will be "plagiarized" from each other too.
- kris_wayton 4y ago
  Also, typically, the "journalist" doesn't pick the headline...an editor does. Not sure if their AI section follows that tradition.
  - dwringer 4y ago
    I assume most major media outlet editors have used AI to help them come up with headlines for many years.
    - ghaff 4y ago
      Coming up with interesting headlines that entice readers without misrepresenting content is actually pretty difficult--especially at publications that value clever headlines.
    - rchaud 4y ago
      It's just A/B testing, not "AI". It's done the same way as digital ads. You come up with 3 different titles for the same link, and serve them to 3 random audiences, and figure out which one got more engagement. Google and Facebook of course do offer "ML-based conversion optimization", but it's widely regarded as hoopla by ad buyers.
      - dwringer 4y ago
        If those frameworks for A/B testing do any statistical analysis or automation I think they are easily classifiable as AI. I wouldn't be surprised if some publications at the top are using much more sophisticated software than that. I could only speculate to what level or with what success ML has been integrated into those pipelines.
        rchaud 4y ago
        Yes, A/B tests are evaluated using T-Test or the Mann-Whitney U statistic. They have existed for a long time and never been classified as "AI". Although with that buzzword currently creating a gold rush, I have no doubt that every half-assed platform will be using that terminology going forward.
- iamflimflam1 4y ago
  That was my thought when I read the article. Also can't help wondering what the same analysis on human generated articles would produce...
- theptip 4y ago
  I had the same thought. I think if you took random human-generated articles on the same subjects you’d find similar levels of similarity. The clickbait article is a formulaic format.
mavu 4y ago
We live in the dumbest possible timeline. What people call AI today is in fact not AI. It is a model that has been trained on input data, and generates prompted output based on that input. In other words THEY ARE AUTOMATED PLAGIARISM MACHINES. THAT IS HOW ALL "AI" today work. Truely dumb.
- broast 4y ago
  Hey that's how I work too!
- FeepingCreature 4y ago
  We will never build AI, because we can only build what we understand, and if we understand it we won't call it AI. https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
- CuriouslyC 4y ago
  Human exceptionalists hold up the 0.0001% of human content that isn't plagarized cliche garbage and act like somehow we're different. If you get an AI to produce 100,000 works and take only the few best they would be pearls just the same.
Someone1234 4y ago
That's literally how this "AI" (autoregressive language model) works. The logic is essentially: A little plagiarism is plagiarism, a lot of plagiarism is original works. So if it is detected, then they need to increase their level of plagiarism until it isn't. While what I just wrote may come across like a joke, there's a lot of truth in it. These language models aren't actually smart, they're just good at parroting smart sounding things they've been taught (although one could argue humans often do the same thing); the problem here is they need more sources to intersplice to hide the true origins. For other "AI" like Computer Vision this is less of a problem since even if you're inputting proprietary visual content, they aren't typically outputting parts of that content, but rather the tags that have been assigned to it (often by humans). This is becoming a problem as the outputs are directly made up using the inputs, and those inputs are proprietary.
- qingdao99 4y ago
  "These language models aren't actually smart, they're just good at parroting smart sounding things they've been taught" I still don't understand the purported difference between "sounds smart" and "is smart". How else can you verify an entity's intelligence other than its "output"? Every doctor, pilot, president has obtained that position of power and responsibility literally only by being tested on their outputs. We can't see inside a human's brain just as we can't see inside an AI's brain. What is the difference? How else do you propose we verify intelligence?
  - radicalbyte 4y ago
    By the output. Those professionals will not make up a random answer.
  - Someone1234 4y ago
    > I still don't understand the purported difference between "sounds smart" and "is smart". One understands the problem space and used logical reasoning to produce a solution, the other put the problem into Google, hit "I'm feeling lucky" and then parroted the first result right or wrong (or an aggregate of that). > How else can you verify an entity's intelligence other than its "output"? I'd argue that's actually a good test. I'd also point out that that is exactly where these solutions let themselves down, because they make [to us] obvious errors and don't understand that they did. That's why you know they are smart sounding rather than actually smart. > we can't see inside an AI's brain. We absolutely can see inside a deep learning model's structure.
kmeisthax 4y ago
> And especially once AIs are drawing from the writing of other AIs, which themselves are quoting AI (dark, I know) it might become quite difficult to detect, No, it will be easier to detect, because the new AIs will learn to mimic earlier generations of their AI - and their faults - rather than mimicking human writing. Because that will score higher on a loss function. I'm going to assume Dead Internet theory - aka most text online is spambots. It's not strictly true, as humans are obstinately continuing to use the Internet and pouring content into them. But the Internet puts those humans on a level playing field with bots. And AI is basically perfect spambot material - it looks superficially different, meaning you can't remove it with simple patterns. So the training set, which is just scraped off the Internet, will be primarily composed of AI-generated material by less-sophisticated systems. Human-written content will be in the minority of the training set, meaning that no matter how good the optimizer statistics or model architecture is, the system will be spending most of its time remembering how to sound like a bad AI.
- williamcotton 4y ago
  There’s still a fitness function based on human interactions with reality: the cost of running these language models needs to keep up with the revenue generated. The revenue ultimately comes from the human economy concerned with housing, food, warmth, water, etc, so the language models need to cater to the concerns of the source of such economic activity by creating content useful enough to real people in order to cover the costs of production.
- CuriouslyC 4y ago
  The real problem isn't AI content generation, it's a lack of content filtering. There has been a mountain of human garbage pushed out by people trying to make money which is already too big to navigate, AI will make the mountain much larger but the problem won't change. We need AI discriminators that analyze content and filter out stuff that is poorly written, cliche, trite and derivative.
ajb 4y ago
At the start of the 18th century, most work had to be done with the muscles of humans or animals. The development of the steam engine, and later engines, allowed these resources to be applied to more productive pursuits. Near the end of the 19th century, all data processing was by human clerks. Tabulating machines, and later computers, allowed these resources to be applied to more productive pursuits. What's the biggest drain of resources today? At least in the developed nations, a good candidate is lies and lack of integrity in decision-making, taken in its most general definition. Whether you are personally most irritated by its presence in government or private industry, it's obvious that this is widespread. Does AI look like it is going to reduce this waste? No, it looks like it's going to add to it.
UncleEntity 4y ago
Just watching these people who’s job is in peril write article after article about how AI is the worst thing ever invented is amusing. Journalism will survive. Journalism will always survive. Shitty content mill “journalists” should probably get the AI to pad out their resumes for them.
- rchaud 4y ago
  I wish we could have AI writing these comments, it might do a better job of masking the 'lurn 2 code' sentiment that every journalist on the internet gets told when job cuts are on the horizon.
honeybadger1 4y ago
Everything ever done is derivative of something done before it.
thrill 4y ago
If AIs can't hold copyrights then it seems to me that they can't be committing plagiarism.
trinsic2 4y ago
"CNET, a titan of tech journalism..." LOL, give me a break. No respectable technology professional considers CNET the bastion of tech journalism. Even before I knew that their articles are written by AI I never would rely on CNET as any kind of authority on matters technical.
- aardvarkr 4y ago
  Titan in this context could mean a large entity with a wide reach. No one is mistaking it as being a cutting edge source of news but it’s hard to deny that the brand name has a significant amount of awareness and namr recognition that has been fostered over two decades. Their audience isn’t tech workers. It’s people like your mom and dad looking for tech tips.
- college_physics 4y ago
  real techies(tm) might have other sources but there is an amplification chamber from such "techy-like websites" to the "techy" parts of newspapers, to the "techy" trends projectors like Gartner and their ilk, to the "modern, cutting edge" politicians that influence policy agendas about techology objective and balanced information about what happens in tech was always a problem as those sites and entities "never bite the hand that feeds them" but it seems it gets harder by the day as we drown in an ocean of fakery
- sct202 4y ago
  So sad to see how they've fallen into being a content farm, like there's no purpose except to jam up search engines to bother even putting an article out on "Can You Buy a Gift Card With a Credit Card?"
  - rchaud 4y ago
    How else would they earn their 5 cent commissions on the thousand affiliate links in every single story?
- rchaud 4y ago
  CNET is the only one still standing. That it looks indistinguishable from a Gizmodo or 9to5Mac style blogsite is a testament to how difficult it is to maintain an independent existence in the fickle world of digital media. Most 'pure' tech journalism does not last long as the market for it is too niche. BYTE magazine was dead before the Internet. The UK-based "Net" magazine (webdev-based) lasted until 2016. PC World is still kicking but runs on a skeleton crew and is online-only. CNET has changed formats many times, and the well-known editors who used to make it worthwhile moved on years ago. That it's trundling on with computer-generated articles is I guess a kind of ironic curtain call.
giaour 4y ago
I'm shocked, shocked I tell you, to hear that this automated plagiarism device committed plagiarism.
hwestiii 4y ago
There are ethics in tech journalism?
rchaud 4y ago
I miss the old CNET of the 2000s. It had a message board back then, true halcyon days when websites had the genius idea of providing something that users could come back to, instead of clickbait articles with auto-playing video ads. But CNET is just the canary in the coal mine; I haven't gone there for tech coverage in over a decade because of the bland writing and the lack of clear editorial voice. The problem for me will be when the entire internet reads like this.
- lax0 4y ago
  Whenever I see CNET it takes me back. And that shareware and freeware download section. So many gems were stuffed on there.
  - rchaud 4y ago
    download.com news.com They really had two of the most highly coveted URLs on the internet for a long time.
- ghaff 4y ago
  There are still some good writers there. But it's become mostly gadget/tech tips. [ADDED: But, really, mainstream consumer tech news in general.] It was a lot broader in the 2000s and had much more personality with a fairly large stable of affiliated blog network writers (which I did for a number of years). After CBS bought them, they started becoming a lot more narrowly focused and homogenized--and click-optimized.
  - rchaud 4y ago
    What was the timing of the CBS acquisition? I used to visit 5 different tech blogs back in 2012 or so, and it was clear that they were all shifting towards "general tech culture" instead of gadgets. I partially blame the iPhone for this, it killed the idea of single-purpose gadgets like MP3 players and handheld consoles which usually had their own quirks. With those products gone (and CES looking much more barren), the articles became more about stuff like Tesla and Uber, or what app updates Google and Apple were pushing out that week.
deleted 4y ago
[deleted]
6gvONxR4sf7o 4y ago
This kind of plagiarism detects near-verbatim mashups of small snippets. Has anyone seen similar ideas investigated in image generation? I’m used to seeing papers show pairs as evidence that their models aren’t copying: generated images and the closest images in the training set. But that wouldn’t catch this kind of rephrase-and-mashup pattern’s visual analog. Has anybody looked at that closely?
pdntspa 4y ago
I think it is stupid that a rephrased sentence is "plagiarism". There are only so many ways to express a given idea. All these morons hunting with a bone to pick are going to be the reason why language is going to be so obtuse for so many.
peter303 4y ago
Managers are figuring out how much money they can save replacing human journalists with AI. Computer generated articles are routine for boilerplate like financial summaries, sports results and police reports. But now creeping into more substantive articles.
voytec 4y ago
Related: "Inside CNET's AI-powered SEO money machine - Fake bylines. Content farming. Affiliate fees. What happens when private equity takes over a storied news site and milks it for clicks?"[1] [1] https://www.theverge.com/2023/1/19/23562966/cnet-ai-written-stories-red-ventures-seo-marketing https://www.theverge.com/2023/1/19/23562966/cnet-ai-written-...
gowld 4y ago
This is silly. The "plagiarism" is a bunch extremeley common phrases describing basic facts, used by everyone all over the SEO spam web. Fortune, one of the "victims" of this "plagiarism", makes its money by subscribing to the New York Times et. al., and paraphrasing articles and placing ads next to the paraphrases.

132 points by iamflimflam1 4y ago

AI gets more human-like every day.

The examples provided are not that convincing. Especially the two headlines. Yes they are similar because there are a limited number of ways to ask "Can You Buy a Gift Card With a Credit Card?". Should the topic be off limit now that it's been written about? Take the code equivalent of asking 100 people to write a function in secret that takes 2 ints and returns a sum. I suspect a number of those code blocks will be "plagiarized" from each other too.

kris_wayton 4y ago
Also, typically, the "journalist" doesn't pick the headline...an editor does. Not sure if their AI section follows that tradition.
- dwringer 4y ago
  I assume most major media outlet editors have used AI to help them come up with headlines for many years.
  - ghaff 4y ago
    Coming up with interesting headlines that entice readers without misrepresenting content is actually pretty difficult--especially at publications that value clever headlines.
  - rchaud 4y ago
    It's just A/B testing, not "AI". It's done the same way as digital ads. You come up with 3 different titles for the same link, and serve them to 3 random audiences, and figure out which one got more engagement. Google and Facebook of course do offer "ML-based conversion optimization", but it's widely regarded as hoopla by ad buyers.
    - dwringer 4y ago
      If those frameworks for A/B testing do any statistical analysis or automation I think they are easily classifiable as AI. I wouldn't be surprised if some publications at the top are using much more sophisticated software than that. I could only speculate to what level or with what success ML has been integrated into those pipelines.
      - rchaud 4y ago
        Yes, A/B tests are evaluated using T-Test or the Mann-Whitney U statistic. They have existed for a long time and never been classified as "AI". Although with that buzzword currently creating a gold rush, I have no doubt that every half-assed platform will be using that terminology going forward.
iamflimflam1 4y ago
That was my thought when I read the article. Also can't help wondering what the same analysis on human generated articles would produce...
theptip 4y ago
I had the same thought. I think if you took random human-generated articles on the same subjects you’d find similar levels of similarity. The clickbait article is a formulaic format.

mavu 4y ago

We live in the dumbest possible timeline. What people call AI today is in fact not AI. It is a model that has been trained on input data, and generates prompted output based on that input. In other words THEY ARE AUTOMATED PLAGIARISM MACHINES. THAT IS HOW ALL "AI" today work. Truely dumb.

broast 4y ago
Hey that's how I work too!
FeepingCreature 4y ago
We will never build AI, because we can only build what we understand, and if we understand it we won't call it AI. https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
CuriouslyC 4y ago
Human exceptionalists hold up the 0.0001% of human content that isn't plagarized cliche garbage and act like somehow we're different. If you get an AI to produce 100,000 works and take only the few best they would be pearls just the same.

Someone1234 4y ago

That's literally how this "AI" (autoregressive language model) works. The logic is essentially: A little plagiarism is plagiarism, a lot of plagiarism is original works. So if it is detected, then they need to increase their level of plagiarism until it isn't. While what I just wrote may come across like a joke, there's a lot of truth in it. These language models aren't actually smart, they're just good at parroting smart sounding things they've been taught (although one could argue humans often do the same thing); the problem here is they need more sources to intersplice to hide the true origins. For other "AI" like Computer Vision this is less of a problem since even if you're inputting proprietary visual content, they aren't typically outputting parts of that content, but rather the tags that have been assigned to it (often by humans). This is becoming a problem as the outputs are directly made up using the inputs, and those inputs are proprietary.

qingdao99 4y ago
"These language models aren't actually smart, they're just good at parroting smart sounding things they've been taught" I still don't understand the purported difference between "sounds smart" and "is smart". How else can you verify an entity's intelligence other than its "output"? Every doctor, pilot, president has obtained that position of power and responsibility literally only by being tested on their outputs. We can't see inside a human's brain just as we can't see inside an AI's brain. What is the difference? How else do you propose we verify intelligence?
- radicalbyte 4y ago
  By the output. Those professionals will not make up a random answer.
- Someone1234 4y ago
  > I still don't understand the purported difference between "sounds smart" and "is smart". One understands the problem space and used logical reasoning to produce a solution, the other put the problem into Google, hit "I'm feeling lucky" and then parroted the first result right or wrong (or an aggregate of that). > How else can you verify an entity's intelligence other than its "output"? I'd argue that's actually a good test. I'd also point out that that is exactly where these solutions let themselves down, because they make [to us] obvious errors and don't understand that they did. That's why you know they are smart sounding rather than actually smart. > we can't see inside an AI's brain. We absolutely can see inside a deep learning model's structure.

kmeisthax 4y ago

> And especially once AIs are drawing from the writing of other AIs, which themselves are quoting AI (dark, I know) it might become quite difficult to detect, No, it will be easier to detect, because the new AIs will learn to mimic earlier generations of their AI - and their faults - rather than mimicking human writing. Because that will score higher on a loss function. I'm going to assume Dead Internet theory - aka most text online is spambots. It's not strictly true, as humans are obstinately continuing to use the Internet and pouring content into them. But the Internet puts those humans on a level playing field with bots. And AI is basically perfect spambot material - it looks superficially different, meaning you can't remove it with simple patterns. So the training set, which is just scraped off the Internet, will be primarily composed of AI-generated material by less-sophisticated systems. Human-written content will be in the minority of the training set, meaning that no matter how good the optimizer statistics or model architecture is, the system will be spending most of its time remembering how to sound like a bad AI.

williamcotton 4y ago
There’s still a fitness function based on human interactions with reality: the cost of running these language models needs to keep up with the revenue generated. The revenue ultimately comes from the human economy concerned with housing, food, warmth, water, etc, so the language models need to cater to the concerns of the source of such economic activity by creating content useful enough to real people in order to cover the costs of production.
CuriouslyC 4y ago
The real problem isn't AI content generation, it's a lack of content filtering. There has been a mountain of human garbage pushed out by people trying to make money which is already too big to navigate, AI will make the mountain much larger but the problem won't change. We need AI discriminators that analyze content and filter out stuff that is poorly written, cliche, trite and derivative.

ajb 4y ago

At the start of the 18th century, most work had to be done with the muscles of humans or animals. The development of the steam engine, and later engines, allowed these resources to be applied to more productive pursuits. Near the end of the 19th century, all data processing was by human clerks. Tabulating machines, and later computers, allowed these resources to be applied to more productive pursuits. What's the biggest drain of resources today? At least in the developed nations, a good candidate is lies and lack of integrity in decision-making, taken in its most general definition. Whether you are personally most irritated by its presence in government or private industry, it's obvious that this is widespread. Does AI look like it is going to reduce this waste? No, it looks like it's going to add to it.

UncleEntity 4y ago

Just watching these people who’s job is in peril write article after article about how AI is the worst thing ever invented is amusing. Journalism will survive. Journalism will always survive. Shitty content mill “journalists” should probably get the AI to pad out their resumes for them.

rchaud 4y ago
I wish we could have AI writing these comments, it might do a better job of masking the 'lurn 2 code' sentiment that every journalist on the internet gets told when job cuts are on the horizon.

honeybadger1 4y ago

Everything ever done is derivative of something done before it.

thrill 4y ago

If AIs can't hold copyrights then it seems to me that they can't be committing plagiarism.

trinsic2 4y ago

"CNET, a titan of tech journalism..." LOL, give me a break. No respectable technology professional considers CNET the bastion of tech journalism. Even before I knew that their articles are written by AI I never would rely on CNET as any kind of authority on matters technical.

aardvarkr 4y ago
Titan in this context could mean a large entity with a wide reach. No one is mistaking it as being a cutting edge source of news but it’s hard to deny that the brand name has a significant amount of awareness and namr recognition that has been fostered over two decades. Their audience isn’t tech workers. It’s people like your mom and dad looking for tech tips.
college_physics 4y ago
real techies(tm) might have other sources but there is an amplification chamber from such "techy-like websites" to the "techy" parts of newspapers, to the "techy" trends projectors like Gartner and their ilk, to the "modern, cutting edge" politicians that influence policy agendas about techology objective and balanced information about what happens in tech was always a problem as those sites and entities "never bite the hand that feeds them" but it seems it gets harder by the day as we drown in an ocean of fakery
sct202 4y ago
So sad to see how they've fallen into being a content farm, like there's no purpose except to jam up search engines to bother even putting an article out on "Can You Buy a Gift Card With a Credit Card?"
- rchaud 4y ago
  How else would they earn their 5 cent commissions on the thousand affiliate links in every single story?
rchaud 4y ago
CNET is the only one still standing. That it looks indistinguishable from a Gizmodo or 9to5Mac style blogsite is a testament to how difficult it is to maintain an independent existence in the fickle world of digital media. Most 'pure' tech journalism does not last long as the market for it is too niche. BYTE magazine was dead before the Internet. The UK-based "Net" magazine (webdev-based) lasted until 2016. PC World is still kicking but runs on a skeleton crew and is online-only. CNET has changed formats many times, and the well-known editors who used to make it worthwhile moved on years ago. That it's trundling on with computer-generated articles is I guess a kind of ironic curtain call.

giaour 4y ago

I'm shocked, shocked I tell you, to hear that this automated plagiarism device committed plagiarism.

hwestiii 4y ago

There are ethics in tech journalism?

rchaud 4y ago

I miss the old CNET of the 2000s. It had a message board back then, true halcyon days when websites had the genius idea of providing something that users could come back to, instead of clickbait articles with auto-playing video ads. But CNET is just the canary in the coal mine; I haven't gone there for tech coverage in over a decade because of the bland writing and the lack of clear editorial voice. The problem for me will be when the entire internet reads like this.

lax0 4y ago
Whenever I see CNET it takes me back. And that shareware and freeware download section. So many gems were stuffed on there.
- rchaud 4y ago
  download.com news.com They really had two of the most highly coveted URLs on the internet for a long time.
ghaff 4y ago
There are still some good writers there. But it's become mostly gadget/tech tips. [ADDED: But, really, mainstream consumer tech news in general.] It was a lot broader in the 2000s and had much more personality with a fairly large stable of affiliated blog network writers (which I did for a number of years). After CBS bought them, they started becoming a lot more narrowly focused and homogenized--and click-optimized.
- rchaud 4y ago
  What was the timing of the CBS acquisition? I used to visit 5 different tech blogs back in 2012 or so, and it was clear that they were all shifting towards "general tech culture" instead of gadgets. I partially blame the iPhone for this, it killed the idea of single-purpose gadgets like MP3 players and handheld consoles which usually had their own quirks. With those products gone (and CES looking much more barren), the articles became more about stuff like Tesla and Uber, or what app updates Google and Apple were pushing out that week.

deleted 4y ago

[deleted]

6gvONxR4sf7o 4y ago

This kind of plagiarism detects near-verbatim mashups of small snippets. Has anyone seen similar ideas investigated in image generation? I’m used to seeing papers show pairs as evidence that their models aren’t copying: generated images and the closest images in the training set. But that wouldn’t catch this kind of rephrase-and-mashup pattern’s visual analog. Has anybody looked at that closely?

pdntspa 4y ago

I think it is stupid that a rephrased sentence is "plagiarism". There are only so many ways to express a given idea. All these morons hunting with a bone to pick are going to be the reason why language is going to be so obtuse for so many.

peter303 4y ago

Managers are figuring out how much money they can save replacing human journalists with AI. Computer generated articles are routine for boilerplate like financial summaries, sports results and police reports. But now creeping into more substantive articles.

voytec 4y ago

Related: "Inside CNET's AI-powered SEO money machine - Fake bylines. Content farming. Affiliate fees. What happens when private equity takes over a storied news site and milks it for clicks?"[1] [1] https://www.theverge.com/2023/1/19/23562966/cnet-ai-written-stories-red-ventures-seo-marketing https://www.theverge.com/2023/1/19/23562966/cnet-ai-written-...

gowld 4y ago

This is silly. The "plagiarism" is a bunch extremeley common phrases describing basic facts, used by everyone all over the SEO spam web. Fortune, one of the "victims" of this "plagiarism", makes its money by subscribing to the New York Times et. al., and paraphrasing articles and placing ads next to the paraphrases.