7 ms·

USA Today Owner Pauses AI Articles After Butchering Sports Coverage

50 points by nxten 3y ago

LapsangGuzzler 3y ago
People think journalism is draped in ulterior motives now, just wait until these publishers are just front ends for AI content created by private companies. What’s the point in having sponsored content when your supposedly unpaid content is churned out by some kids who raised a couple million bucks off of this AI wave? The ultimate goal of advertisement is to be invisible, to be influencing without having to admit as much, and AI is the perfect vehicle to do just that.
- Martinussen 3y ago
  To be fair, just because we can generate text well doesn't mean we're any closer to actually knowing how to influence people subtly like that. Anything that makes content can be used for vague general propaganda/lobbying, but impacting product-decision in a specific way? Seems as difficult as ever.
  - LapsangGuzzler 3y ago
    > Anything that makes content can be used for vague general propaganda/lobbying, but impacting product-decision in a specific way? A/B testing and tracking metrics on user behavior is a very mature practice in the industry. Some types of cases and behaviors might be hard to track, but then really obvious ones (like Jan 6) also stand out. High profile universities like Stanford built behavioral design labs backed by millions in funding to advance our understanding of how to influence behavior through technology. This is a massive commercial research space with huge money on the line. We can’t ignore that.
soligern 3y ago
We’re not at a point where this can be a human free pipeline, but I bet you could make this work with 1-2 humans in the pipeline that review instead of the 100 you would traditionally need.
- warning26 3y ago
  In theory, but I think the problem you'd find in practice is that reviewing 100s of nearly-identical AI-written articles gets very boring very quickly, and a lot of errors would slip through.
  - guiambros 3y ago
    Ok, so 2 is not enough. What if you put 4 humans? Or 8? Or maybe even 20 (instead of 100, per OP)? These experiments fail because they greedily try to outsource all the work to AI. Another recent example: the lawyer who submitted ChapGPT hallucinations directly to a court case. You don't need to eliminate humans, and certainly not at first. You just need to be much more efficient than status quo in order for AI to be deployed at scale.
  - soligern 3y ago
    Why? Lots of people do a lot of reading for a living. Reading a new article is way more interesting than thousands of jobs I can think of. Data analysts and accountants literally pour though millions of featureless numbers over their career, why would this be any different?
    - maxbond 3y ago
      It's like trying to find typos in your own writing. It's very difficult to stay focused when reading a long series of nearly identical documents. The people you're talking about are reading a lot of novel documents.
      - soligern 3y ago
        Why would these be identical? New stories everyday, new games played etc.
        maxbond 3y ago
        Because they're populating a template. > One article even failed to populate properly, with the text instead featuring a bracketed glimpse at how its opening sentence was supposed to read. > "The Worthington Christian [[WINNING_TEAM_MASCOT]] defeated the Westerville North [[LOSING_TEAM_MASCOT]] 2-1 in an Ohio boys soccer game on Saturday," reads the butchered intro.
vel0city 3y ago
I'm surprised it failed this hard. AI writing has been used for a while in fantasy sports platforms for a while to put in little quips about player performances based on their recent stats and automated projections and read pretty well.
- htrp 3y ago
  Probably a lazy implementation that they farmed out to the lowest bidder
catchnear4321 3y ago
the owner is gannett. the butchered coverage was in local newspapers, also owned by gannett. which begs the question, why doesn’t the headline just, you know, name gannett? edit: anyone downvoting care to indicate why? it seems rather pertinent to name the owner, given the scale, and the potential impact of using such technology at scale.
- maxbond 3y ago
  I generally downvote complaints about the title of articles. (I don't downvote requests to change the title on HN though.)
  - catchnear4321 3y ago
    there was a bit more to my comment than a “this title is bad.” do you know of gannett? legitimately curious.
    - maxbond 3y ago
      The thesis of your comment seems to be that Gannet should have been in the title, with your other two points being supporting evidence. I've never heard of Gannet before today. I don't really mean much by the downvote. It's a personal pet peeve of mine when people comment on the headline, and I think it's against the spirit of this part of the guidelines: > Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting. My downvote definitely didn't mean that I didn't think there was something substantive to say about Gannet's involvement, or that I disagreed. (The guidelines also tell us not to talk about voting, which I'm breaking of course, but I do try to respond when people genuinely want to know why I voted a certain way. I understand why that part of the guidelines exist, but I have mixed feelings about it, I don't see how else people are meant to know why they were downvoted.)
      - catchnear4321 3y ago
        > I've never heard of Gannet before today. this is my very point, and it is anything but tangential. thank you.
adocomplete 3y ago
AI for content creation is still in its infancy. It will get better, smarter, and more effective. Even humans make mistakes sometimes.
Gigablah 3y ago
I don't know about you, but "a close encounter of the athletic kind" is a hit for me.
idontwantthis 3y ago
Where does the data on the sports event come from? There must be other articles already written about it for the AI to attempt to reproduce, right? Why wouldn’t they just use those original articles? Or are these made up whole cloth and a CEO thought AI just magically knows what happens on high school football fields?
- jfengel 3y ago
  They start with raw data collected at the event itself. A lot of games have statisticians who collect not just the scores but gather data on every play/at-bat/etc. Even high school games have those; it's often the official job of some student. That does have enough information to make an article. In the early days of baseball, a radio announcer could receive the stats continuously and narrate the game, with a bit of imagination. Some of the examples look as if they got nothing more than the box scores, and used AI hallucination on other sports articles to generate a bunch of generic verbiage out of that. (To me, as a non sports fan, that's what most sports writing looks like anyway. I have no idea why people want to read that, but there is a market for it.)

50 points by nxten 3y ago

People think journalism is draped in ulterior motives now, just wait until these publishers are just front ends for AI content created by private companies. What’s the point in having sponsored content when your supposedly unpaid content is churned out by some kids who raised a couple million bucks off of this AI wave? The ultimate goal of advertisement is to be invisible, to be influencing without having to admit as much, and AI is the perfect vehicle to do just that.

Martinussen 3y ago
To be fair, just because we can generate text well doesn't mean we're any closer to actually knowing how to influence people subtly like that. Anything that makes content can be used for vague general propaganda/lobbying, but impacting product-decision in a specific way? Seems as difficult as ever.
- LapsangGuzzler 3y ago
  > Anything that makes content can be used for vague general propaganda/lobbying, but impacting product-decision in a specific way? A/B testing and tracking metrics on user behavior is a very mature practice in the industry. Some types of cases and behaviors might be hard to track, but then really obvious ones (like Jan 6) also stand out. High profile universities like Stanford built behavioral design labs backed by millions in funding to advance our understanding of how to influence behavior through technology. This is a massive commercial research space with huge money on the line. We can’t ignore that.

soligern 3y ago

We’re not at a point where this can be a human free pipeline, but I bet you could make this work with 1-2 humans in the pipeline that review instead of the 100 you would traditionally need.

warning26 3y ago
In theory, but I think the problem you'd find in practice is that reviewing 100s of nearly-identical AI-written articles gets very boring very quickly, and a lot of errors would slip through.
- guiambros 3y ago
  Ok, so 2 is not enough. What if you put 4 humans? Or 8? Or maybe even 20 (instead of 100, per OP)? These experiments fail because they greedily try to outsource all the work to AI. Another recent example: the lawyer who submitted ChapGPT hallucinations directly to a court case. You don't need to eliminate humans, and certainly not at first. You just need to be much more efficient than status quo in order for AI to be deployed at scale.
- soligern 3y ago
  Why? Lots of people do a lot of reading for a living. Reading a new article is way more interesting than thousands of jobs I can think of. Data analysts and accountants literally pour though millions of featureless numbers over their career, why would this be any different?
  - maxbond 3y ago
    It's like trying to find typos in your own writing. It's very difficult to stay focused when reading a long series of nearly identical documents. The people you're talking about are reading a lot of novel documents.
    - soligern 3y ago
      Why would these be identical? New stories everyday, new games played etc.
      - maxbond 3y ago
        Because they're populating a template. > One article even failed to populate properly, with the text instead featuring a bracketed glimpse at how its opening sentence was supposed to read. > "The Worthington Christian [[WINNING_TEAM_MASCOT]] defeated the Westerville North [[LOSING_TEAM_MASCOT]] 2-1 in an Ohio boys soccer game on Saturday," reads the butchered intro.

vel0city 3y ago

I'm surprised it failed this hard. AI writing has been used for a while in fantasy sports platforms for a while to put in little quips about player performances based on their recent stats and automated projections and read pretty well.

htrp 3y ago
Probably a lazy implementation that they farmed out to the lowest bidder

catchnear4321 3y ago

the owner is gannett. the butchered coverage was in local newspapers, also owned by gannett. which begs the question, why doesn’t the headline just, you know, name gannett? edit: anyone downvoting care to indicate why? it seems rather pertinent to name the owner, given the scale, and the potential impact of using such technology at scale.

maxbond 3y ago
I generally downvote complaints about the title of articles. (I don't downvote requests to change the title on HN though.)
- catchnear4321 3y ago
  there was a bit more to my comment than a “this title is bad.” do you know of gannett? legitimately curious.
  - maxbond 3y ago
    The thesis of your comment seems to be that Gannet should have been in the title, with your other two points being supporting evidence. I've never heard of Gannet before today. I don't really mean much by the downvote. It's a personal pet peeve of mine when people comment on the headline, and I think it's against the spirit of this part of the guidelines: > Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting. My downvote definitely didn't mean that I didn't think there was something substantive to say about Gannet's involvement, or that I disagreed. (The guidelines also tell us not to talk about voting, which I'm breaking of course, but I do try to respond when people genuinely want to know why I voted a certain way. I understand why that part of the guidelines exist, but I have mixed feelings about it, I don't see how else people are meant to know why they were downvoted.)
    - catchnear4321 3y ago
      > I've never heard of Gannet before today. this is my very point, and it is anything but tangential. thank you.

adocomplete 3y ago

AI for content creation is still in its infancy. It will get better, smarter, and more effective. Even humans make mistakes sometimes.

Gigablah 3y ago

I don't know about you, but "a close encounter of the athletic kind" is a hit for me.

idontwantthis 3y ago

Where does the data on the sports event come from? There must be other articles already written about it for the AI to attempt to reproduce, right? Why wouldn’t they just use those original articles? Or are these made up whole cloth and a CEO thought AI just magically knows what happens on high school football fields?

jfengel 3y ago
They start with raw data collected at the event itself. A lot of games have statisticians who collect not just the scores but gather data on every play/at-bat/etc. Even high school games have those; it's often the official job of some student. That does have enough information to make an article. In the early days of baseball, a radio announcer could receive the stats continuously and narrate the game, with a bit of imagination. Some of the examples look as if they got nothing more than the box scores, and used AI hallucination on other sports articles to generate a bunch of generic verbiage out of that. (To me, as a non sports fan, that's what most sports writing looks like anyway. I have no idea why people want to read that, but there is a market for it.)