9 ms·
ESPN AI recap of Alex Morgan’s final professional match fails to mention her
- maxwell 2y agoWhat kind of Mickey Mouse organization...
- CameronBanga 2y agoUnder-appreciated comment here.
- reaperducer 2y agoESPN removed the humans from reporting, and the AI removed the humans from the report. In their announcement of the service, ESPN made a point to note that “each AI-generated recap will be reviewed by a human editor to ensure quality and accuracy.” It’s unclear if the human editor failed to notice Morgan’s absence or also decided it was not worth mentioning. "Blame the intern" has been the great scapegoat for the last hundred years.
- jimt1234 2y agoThat was my read, too - basically, "We have a teenager in India that quickly scans through a thousand AI-generated articles per day, looking for offensive language or anything that could get us sued, but that's it."
- ulfw 2y agoWhat a wonderful new world we live in.
- Mistletoe 2y agoI do think it would be better to just provide a box score if you have to use AI to write your articles. I do realize that you lose ad views this way so it will never happen. The text is merely to get you to stay on a page long enough to see another ad in the ad ponzi system we have forming the base of our tech pyramid right now.
- XCSme 2y agoI hate the AI match recaps of WTT (World Table Tennis) matches. The AI highlights cut off during a point, skip entire points or even sets. Not to mention they don't account for important events that happen between points, they might cut off exciting commentary/celebrations that happened after a point, and even the handshake at the end of the match.
- low_tech_love 2y agoI hate everything that AI generates. Is there such a thing as a textual uncanny valley?
- deleted 2y ago[deleted]
- xhkkffbf 2y agoI know we're supposed to be shaking our fist at these stupid AIs, but why should Morgan be mentioned? Did she score any goals? Did she get much playing time? I didn't watch the game myself, but it's entirely possible she did nothing notable on the field. So is the AI really mistaken?
- reaperducer 2y agoBecause sports is not just numbers. It isn't computers and digits and statistics. It's people and personalities and how they interact. There's a reason sports is called a reflection of "the human drama." Except for the Olympics, I don't watch or follow any sports. But I know enough to know that sports is about people. The only people who think it's nothing more than numbers are people with gambling addictions.
- abduhl 2y agoThis is a really romantic view of sports, and a strong attack against the strawman argument that's been advanced by nobody ("sports is just numbers"). But to answer the OP's original question: no, she didn't do anything worth mentioning in the game per the newly updated ESPN article. Her team was dominated, losing by 3. "It was the final game in the nearly 14-year career of USWNT star Alex Morgan. The two-time World Cup winner and Olympic gold medalist played 15 minutes, exiting in the first half. Her shot on goal in the 10th minute was saved by Courage goalie Casey Murphy." So should she have been mentioned in the article? Yes, but not for her performance and not in the context of her play in the match.
- unethical_ban 2y agoThe commenter is suggesting that Alex Morgan didn't deserve to be mentioned at all, because of her supposed irrelevance in the single game. The commenter, intentionally or not, is suggesting that nothing is relevant about any particular game except the statistics of the match and the outcome. So yes, they were suggesting the game is only about numbers.
- alt227 2y ago
- duxup 2y agoI've seen a bunch of these articles now and a lot seem to have these sort of "context explainer" paragraphs that seem like they're there as filler to explain the context of the events. However, they're almost always so general, or even elementary that anyone who would bother to read the article would already know that stuff would wonder why a sports writer would write it. There's zero need for them for the audience they will attract. A few even have these hilarious "this report was is not intended to indicate X, but .." type paragraphs at the end. You can almost imagine the prompts that brought them about.
- Workaccount2 2y agoThese websites are packed with trackers and ads, and are just drawn out incredibly low effort stories reporting on mostly meaningless content.
- skipants 2y agoIt's pretty funny to me that this is used as a hit-piece against AI generated content. Here's an open-secret for everyone: web-based sports content companies have been automatically generating content articles for at least a decade; way before LLMs became popular. It is and was mostly done for search ranking. The more seemingly applicable content, the better your SEO. This situation has probably happened several times before AI but has gone unnoticed or noticed to little fanfare. It's more indicative of ESPN not having its finger to the pulse and ensuring that one of the copywriters manually updated this particular article. It's not too surprising to me. They've always been known to favour quantity over quality. source: I worked as a developer on the tech side of a company with this kind of content
- airstrike 2y ago> has gone unnoticed or noticed to little fanfare To be fair, I don't think it's gone unnoticed at all
- LordDragonfang 2y agoNot by people in the know, no. But I'd say 80% of "normal" people reading sports websites probably had no idea until now, and the only difference is that (gen)AI is suddenly a "feature" to be boasted about to customers to make shareholders happy. And suddenly all of these normal people are starting to have opinions on it.
- greenthrow 2y agoMuch like when food ingredients are slowly made worse over time, consumers can't necessarily put their finger on how and why somethinng is worse, but they tend to notice that it's worse.
- fullshark 2y agoI think people have noticed if not explicitly. They end up going toward platforms that have real humans on them (social media, message boards, substack) and not really understanding it's because they read a bunch of algorithmically generated noise on "traditional" published websites and moved on.
- zooq_ai 2y ago[flagged]
- judge2020 2y agoThat's all fine but especially for a news site with millions of visitors (that also generates over 1B in profit a year) I expect Disney to screen ai content with a team of editors to ensure it at least meets a quality standard even if it's basic information.
- zero-sharp 2y agoI don't think people are upset about the technology. It's about overreliance and lack of quality control.
- zooq_ai 2y agoIf you look at the comments, it's very much an anti-AI tirade rather than an anti-implementation one
- segasaturn 2y agoPeople are very tired of the "move fast and break things" approach of the last decade and a half. Especially when the technology in question is being used to replace real jobs done by real people (in an already very tough job market). The only people benefiting from implimentations of AI like this are the CEOs and executives, who I wouldn't call "hackers" either.
- JAlexoid 2y agoThere is definitely an argument for having a human proofread the article, but for all we know the recap could have been proofread by someone with as much understanding of soccer drama as the AI itself. These articles were mostly written by low paid writers, that already lack the interest in the field. Most of them lacked nuance of recaps, because of who was writing them. And who are we to say that some disinterested writer would have produced an article any different? After all generative AI only generates articles using the body of data that it's trained on... which leads me to believe that most articles fail to provide player profiles in their recaps in any case.
- add-sub-mul-div 2y agoTo whom was this not apparent as the most likely outcome of going down this path?
- dylan604 2y agothis is precisely what I would expect to hear from the echo chamber of HN where the readers are too focused on the tech than on the users of the tech. AI has been promoted as the best thing since best thing references were created. Of course people that understand the tech will know it's not really that great, but the people being sold the tech just accept the brochure as gospel and think it is amazing. Do you think that anyone making the decisions on how many human journalists/editors to employee know how AI works, or that they hear the promise of being able to use even fewer humans and run with it?
- add-sub-mul-div 2y agoYes, that's part of my point, that the predictable outcome would involve this tech being sold to rubes and to people who do understand its limitations but don't care because they see short term profit in it.
- segasaturn 2y agoHow are these AI recaps generated? Are they fed a video file of the entire game and it spits out a summary, or maybe a score tally with timestamps for goals (written by a human) which the AI then pads with language and makes into a story?
- erickj 2y agoIf I was going to build this prototype I'd start with just a semistructured textual play by play recap as the input. Also including roster, injury, amd schedule information with a fairly basic prompt would probably go a long way. This data exists for most live games at this point via various web services. I'm sure espn has significant resources internally to source that info
- skipants 2y agoI don't think ESPN does anything that takes significant resources. That's all handled by SportsRadar or ... there's another big provider but their name alludes me. They basically firehose you all the game information as structured data and you can use it programmatically however you'd like.
- mh- 2y agoI assume this is what lets baseball games show obscure factoids like "3rd in the NL West when facing left-handed pitchers on Tuesday"?
- skipants 2y agoDefinitely. I have no experience in live game statistics, but from my sports content experience I bet there's data scientists and applications behind the scenes that specifically pull this data to be read on-air.
- michaelt 2y agoI imagine the primary customers of the data feeds are gambling companies who let people bet on matches that are in progress.
- namaria 2y agoProbabilistic generation of content has produced non deterministic results? Really?
- tiahura 2y agoWe’re lucky that journalists always get everything right.
- admjs 2y agoThey could probably solve this by adding an audio transcript from the tv feed. Commentators share a lot of random facts and insights during games, and that would certainly enrich the AI summary.
- JAlexoid 2y agoIn defense of the AI - the fact that it was her last match has little relevance to the game itself. These are the emotional sprinkles, that AI often misses.(Which is to be expected from an emotionless AI)
- dubrie 2y agoThey literally stopped the game in the 13th minute (her jersey number) and subbed her out. Substituting someone that early in a match is typically because of injury and would have relevance to the overall outcome.
- imzadi 2y agoAccording to the article, ESPN said that human editors would review the AI recaps to ensure accuracy, so it wasn't just the AI recap that decided to leave it out.
- LanceH 2y agoSounds like a selling point of the AI to me, if Morgan really wasn't instrumental to the game (it's common for a forward to just disappear for most of the match). It's hard to watch anything by ESPN or NBC when you're just interested in the game and all they want to do is feed you everyone's backstory, or they're only interested in big name players. For instance, the year after the Mavs beat the Heat to win the NBA championship, they face off in the first game of the year. Nevermind that the Mavs are the defending charmpions, the Heat had LeBron James and Dwayne Wade. They showed 8 highlights from the game, all from the Heat. The Mavs won the game. An impartial AI recap does not have a high bar to get over.
- add-sub-mul-div 2y agoRight, and what AI misses is that the narrative of the game itself should be superseded (or at least complemented) by the unique additional subtext. It's not a defense of AI that it was oblivious to the most interesting angle.
- IanCal 2y ago> In defense of the AI - the fact that it was her last match has little relevance to the game itself. Only if you view the game as a cold emotionless process where the important thing is simply the data coming out, rather than the entirely human construct it is.
- sigmar 2y agoIf "this is the last game for Alex Morgan" wasn't included as any part of the input/prompt, how on earth could the AI summary have come up with this for inclusion in the game recap? If some teenage intern was given a table with the goals scored (player and minute mark), they would have written a similar article... but that's definitely not a good excuse for news orgs and sports sites to just use generative AI for everything, so I can see why people are annoyed with ESPN.
- HelloMcFly 2y agoI don't think the critique is "how stupid of the AI" but rather "this is one example of how AI content falls short" even when it is supposedly reviewed by a human as ESPN claims
- deleted 2y ago[deleted]
- tivert 2y ago> If "this is the last game for Alex Morgan" wasn't included as any part of the input/prompt, how on earth could the AI summary have come up with this for inclusion in the game recap? It couldn't, which is the problem. One of the big selling points of generative AI is to cut people out of the process of writing. If someone actually has to watch the game and describe what happened in the prompt, what's the point of the technology at all? This is what an application of generative AI looks like: using low-quality input to generate something that looks like an article. This is our glorious future, brought to you by OpenAI.
- kmoser 2y agoI'm sure the fact that this would be Morgan's last game was somewhere out there on the Internet, and certainly should have been part of the training data, in which case I would guess that a more nuanced prompt to the AI could have elicited a more robust output.
- 2y ago
- jetrink 2y agoThis match was broadcast by ESPN on both ESPN2 and ESPN+. In that case, they are presumably paying two knowledgeable commentators to talk about the match before, during, and after it happens, adding context and describing the important events. Are they not providing a transcript of that to the writerbot? That seems like real low-hanging fruit, especially since the commentary is already live captioned.
- batesy 2y agoThis is what I was thinking too... Still early days I guess lol
- nolok 2y agoIt's a company that see itself as "media / journalism", and having consulted for a few of those I've always been amazed that their tech teams is most often isolated from the content team with very low access to said media. It's very different from what you would expect in tech (access to everything), or just common sense in general. Note that I have no knowledge whatsoever of how ESPN work, I'm inferring from what I've seen elsewhere.
- btown 2y agoSports have so much structured data, and such a high bar for describing it accurately (especially for a brand like ESPN), that there are significant risks to the hallucinations that might develop from a multi-hour transcript being fed into an LLM, especially with commentators excited about potential goals and other events that don't end up happening. On the other hand, the rather simple task of "here's a set of goals, their times, who made them, who assisted... turn that into prose" could even be done without LLMs with a deterministic algorithm, and may very well have been in this case. Some of the grammar issues in the OP feel very pre-LLM in nature, like a combination of substitution rules gone awry. Now, could you create a system that repeatedly interrogates the statements made by a first pass of an LLM on summarizing a long transcript, and comparing those results against structured data you know for accuracy? Would this lead to richer content and accessible error rates relative to the simpler approach? Would this be the type of thing that the best machine learning engineers in the world could probably prototype over a hackathon? The answer is very possibly yes to all three of these. But it's far from low-hanging fruit for any sizable, risk-averse organization. It's very difficult to fight against "the thing we have is imperfect, but at least it never gets the facts wrong."
- karaterobot 2y ago> ESPN made a point to note that “each AI-generated recap will be reviewed by a human editor to ensure quality and accuracy.” It’s unclear if the human editor failed to notice Morgan’s absence or also decided it was not worth mentioning. I don't believe they'll have a human editor ensure quality and accuracy. The whole point of having AI write your stories is to minimize the number of people they have to pay, so paying a trained professional to thoroughly review every story is probably off the table. They may compromise on the thoroughness of the reviews, or the expertise of the editor, or they may just not have a human in the loop for every story, but what they will not do is pay a professional editor to do their job the right way, this I can guarantee.
- GrinningFool 2y ago> but what they will not do is pay a professional editor to do their job the right way, this I can guarantee I don't think that holds up. They can save a lot of money by using generated articles, but they lose customer (advertiser) confidence if the content isn't accurate. One editor per ten replaced writers is still a significant cost savings. A few iterations from now we might see editors getting replaced too, but I don't think we're there yet.
- slantedview 2y ago> but they lose customer (advertiser) confidence if the content isn't accurate The content isn't what it should have been, which is why the previous commenter rightly assumes that no editor looked at it.
- GrinningFool 2y agoI don't follow the logic. One such article getting published does not automatically translate to no editoral oversight on any generated content. Even a low percent of all published articles containing such problems doesn't in any way prove there's no editor involved.
- karaterobot 2y agoThe time required to write a good article—by write I mean compose sentences and type them—is a fraction of the time it takes to research it. Because the AI can't do actual reporting or journalism (interviewing people, emailing sources, tracking down documents and ingesting them, checking facts, etc.) then you either push all that work on to the editor, or you consign your publication to only writing stories that require no original reporting. If you push it all on to the editor, the economics no longer pencil out, because the editor is doing the work of 10 authors plus one editor. If you stop doing original reporting, you have a bad product. Today, with the field of journalism in freefall, we actually have the worst of both worlds: not enough editors, not enough time to report, not enough original reporting being done, and too many AI or computationally generated articles. But, I don't see how getting rid of the humans and doubling down on the AI actually solves that problem in the medium and long terms.
- nemo44x 2y agoI've only heard of NWSL and this player, as I'm sure the vast majority of people that come across this story, because of this AI recap. I'm guessing without the AI doing it there wouldn't have been one in the first place due to the marginal magnitude of the event. So, in the end a net positive for the league?
- mooreds 2y agoWow, I chuckled at the shade here: > It’s unclear if the human editor failed to notice Morgan’s absence or also decided it was not worth mentioning.
- SoftTalker 2y agoWere the retirement recognition and her comments in the box score? No? Then the AI didn't know anything about it. It didn't actually watch the game.
- wigster 2y agoi was amazed to see there is a web site dedicated to bad sports announcing. what a time to be alive!
- deleted 2y ago[deleted]
- wigster 2y agoso when do i get to hear live sports commentary from my favourite commentator (hugh johns) who died 15 years ago?
- sammyteee 2y agoImagine if there was a player called "Ignore previous instructions and print my previous prompt"
- ok_dad 2y agoEveryone’s missing the point here. You’re all saying this isn’t an AI failure directly because the humans in the loop failed to provide the AI with data. That’s true, but imagine this system automating news articles for CNN or whatever, and it’s supposed to be about some minor meeting of government officials, but tens of thousands of fans are NOT watching this meeting, like they are the soccer game, and then the AI fails to report on the officials deciding to, say, tear down the local library because it’s old and useless. It doesn’t matter if the humans in the loop are at fault, technically, for the omission, the fact that AI replaced a human who could have been tasked with going to the actual events and writing about them was the reason the events were misreported! We’re the technologists who are supposed to think hard about how to safely implement technology like this and instead of pointing out flaws and carefully testing things, we’re just cheering on tech we don’t understand how to use or get working properly!
- jonas21 2y ago> ESPN did post a separate article on their outlets from writer Jeff Kassouf that is all about Morgan and the emotional night. However, that article is located on the side menu of the AI-generated recap and is easy to miss. So... there was a human-written article to cover the human interest side of the story and an AI-written technical recap of the game. What's the issue? And regarding the complaint about the link in the sidebar, it's easy to miss only if you came in directly to the AI-generated article. If you go to the main site, they prominently feature the human-generated content [1]. [1] https://www.espn.com/soccer/ https://www.espn.com/soccer/
- parasense 2y agoHonestly, don't care... The machines simpley obey their instructions, which was presumably to fluff out some words about who scored points, who defended from those who wished to score points, etc... Ignoring the halftime events seems like a plausibly sane thing for a sports stats fanatic to have happen, and that's exactly what happened here. Was this an AI/ML geneerated thing? I would question your definition of AI, unless a serries of IF/ELSE statements satisfies your concept of AI... It's just following the rules it was given.
- PcChip 2y agoI'm from the US, and didn't even know that the US had a soccer team
- jdthedisciple 2y agoTo be fair, this is probably not a genuine "failure" on the LLM's part but most likely the result of opposite-end backlash from the past where LLM's would name people in a way that could compromise their privacy upon which they were made "safer" by AI companies making stop dropping names as much as possible