6 ms·

If "this is the last game for Alex Morgan" wasn't included as any part of the input/prompt, how on earth could the AI summary have come up with this for inclusi

by sigmar 2y ago

If "this is the last game for Alex Morgan" wasn't included as any part of the input/prompt, how on earth could the AI summary have come up with this for inclusion in the game recap? If some teenage intern was given a table with the goals scored (player and minute mark), they would have written a similar article... but that's definitely not a good excuse for news orgs and sports sites to just use generative AI for everything, so I can see why people are annoyed with ESPN.

HelloMcFly 2y ago
I don't think the critique is "how stupid of the AI" but rather "this is one example of how AI content falls short" even when it is supposedly reviewed by a human as ESPN claims
- deleted 2y ago
  [deleted]
tivert 2y ago
> If "this is the last game for Alex Morgan" wasn't included as any part of the input/prompt, how on earth could the AI summary have come up with this for inclusion in the game recap? It couldn't, which is the problem. One of the big selling points of generative AI is to cut people out of the process of writing. If someone actually has to watch the game and describe what happened in the prompt, what's the point of the technology at all? This is what an application of generative AI looks like: using low-quality input to generate something that looks like an article. This is our glorious future, brought to you by OpenAI.
- kmoser 2y ago
  I'm sure the fact that this would be Morgan's last game was somewhere out there on the Internet, and certainly should have been part of the training data, in which case I would guess that a more nuanced prompt to the AI could have elicited a more robust output.
  - e_y_ 2y ago
    LLM's general knowledge is trained on data that's months old (or longer). Alex Morgan only announced her retirement a few days ago, so in order for the AI to know about it, someone would need to pick out recent, relevant articles about her and feed it to the generator. And even then, it's unclear that the AI would be smart enough to identify "retirement" as something that should be called out in the new article without specific prompting.
  - starkparker 2y ago
    Not only was it out there on the internet, but the human writer paid by ESPN to cover the league had already written about it days prior. https://www.espn.ph/football/story/_/id/41136440/alex-morgan-retirement-leaves-towering-uswnt-nwsl-legacy https://www.espn.ph/football/story/_/id/41136440/alex-morgan...
gs17 2y ago
Really, they need to set up some kind of RAG system to include this kind of information in the prompt. You could definitely make this semi-automated, but it requires putting more effort in to the system.
crazygringo 2y ago
Indeed. This doesn't really have anything to do with the limitations of AI inherently, but more about using it better. It seems like it highlights a clear avenue for improvment: rather than just feeding the events of a game to the LLM and asking it to summarize them, it seems important for the prompt to include summaries of e.g. the last 10 news stories involving participants in the game (players, coaches, etc.) and maybe the 10 top all-time news stories as well. Then the prompt can be asked to summarize the game (not the other information), but to draw from the other information where it might make the article better. Seems like exactly the kind of things LLM's can do, right?
qq66 2y ago
It was well-known to people who cared about this (i.e., anyone who would read this recap) that this was Alex Morgan's last game. If ESPN can't capture this and incorporate it into their piece, given their wide reach in sports media, why should a publisher use the ESPN recap service instead of random Y Combinator startup?

by sigmar 2y ago

HelloMcFly 2y ago

I don't think the critique is "how stupid of the AI" but rather "this is one example of how AI content falls short" even when it is supposedly reviewed by a human as ESPN claims

deleted 2y ago
[deleted]

tivert 2y ago

> If "this is the last game for Alex Morgan" wasn't included as any part of the input/prompt, how on earth could the AI summary have come up with this for inclusion in the game recap? It couldn't, which is the problem. One of the big selling points of generative AI is to cut people out of the process of writing. If someone actually has to watch the game and describe what happened in the prompt, what's the point of the technology at all? This is what an application of generative AI looks like: using low-quality input to generate something that looks like an article. This is our glorious future, brought to you by OpenAI.

kmoser 2y ago
I'm sure the fact that this would be Morgan's last game was somewhere out there on the Internet, and certainly should have been part of the training data, in which case I would guess that a more nuanced prompt to the AI could have elicited a more robust output.
- e_y_ 2y ago
  LLM's general knowledge is trained on data that's months old (or longer). Alex Morgan only announced her retirement a few days ago, so in order for the AI to know about it, someone would need to pick out recent, relevant articles about her and feed it to the generator. And even then, it's unclear that the AI would be smart enough to identify "retirement" as something that should be called out in the new article without specific prompting.
- starkparker 2y ago
  Not only was it out there on the internet, but the human writer paid by ESPN to cover the league had already written about it days prior. https://www.espn.ph/football/story/_/id/41136440/alex-morgan-retirement-leaves-towering-uswnt-nwsl-legacy https://www.espn.ph/football/story/_/id/41136440/alex-morgan...

gs17 2y ago

Really, they need to set up some kind of RAG system to include this kind of information in the prompt. You could definitely make this semi-automated, but it requires putting more effort in to the system.

crazygringo 2y ago

Indeed. This doesn't really have anything to do with the limitations of AI inherently, but more about using it better. It seems like it highlights a clear avenue for improvment: rather than just feeding the events of a game to the LLM and asking it to summarize them, it seems important for the prompt to include summaries of e.g. the last 10 news stories involving participants in the game (players, coaches, etc.) and maybe the 10 top all-time news stories as well. Then the prompt can be asked to summarize the game (not the other information), but to draw from the other information where it might make the article better. Seems like exactly the kind of things LLM's can do, right?

qq66 2y ago

It was well-known to people who cared about this (i.e., anyone who would read this recap) that this was Alex Morgan's last game. If ESPN can't capture this and incorporate it into their piece, given their wide reach in sports media, why should a publisher use the ESPN recap service instead of random Y Combinator startup?