8 ms·
Generative AI and Wikipedia editing: What we learned in 2025
- genie3io 8mo ago[dead]
- ks2048 9mo ago[flagged]
- ragesoss 9mo agolol. would have written something shorter for HN, but the main expected audience for it was Wikipedians.
- vibeprofessor 9mo ago[flagged]
- coffeebeqn 8mo agoAh yes Elon Musk the man with no agenda
- vibeprofessor 8mo agohere is just one example, wikipedia turned into antisemitic propaganda outlet https://x.com/just_whatever/status/2017948781895880726?s=20 https://x.com/just_whatever/status/2017948781895880726?s=20
- kmeisthax 8mo agoGrokipedia is a pile of propaganda written by an AI that moonlights as a CSAM generator, built to serve as a weapon in a culture war being waged by a bunch of billionaires trying to normalize pedophilia by selling it to neo-Nazis. For now, I think I'll take the Wikipedia edit gangs.
- deleted 8mo ago[deleted]
- ColinWright 9mo agoThe title I've chosen here is carefully selected to highlight one of the main points. It comes (lightly edited for length) from this paragraph: Far more insidious, however, was something else we discovered: More than two-thirds of these articles failed verification. That means the article contained a plausible-sounding sentence, cited to a real, relevant-sounding source. But when you read the source it’s cited to, the information on Wikipedia does not exist in that specific source. When a claim fails verification, it’s impossible to tell whether the information is true or not. For most of the articles Pangram flagged as written by GenAI, nearly every cited sentence in the article failed verification.
- dang 9mo agoSubmitted title was "For most flagged articles, nearly every cited sentence failed verification". I agree, that's interesting, and you've aptly expressed it in your comment here.
- the_fall 9mo agoFWIW, this is a fairly common problem on Wikipedia in political articles, predating AI. I encourage you to give it a try and verify some citations. A lot of them turn out to be more or less bogus. I'm not saying that AI isn't making it worse, but bad-faith editing is commonplace when it comes to hot-button topics.
- mjburgess 9mo agoAny articles where newspapers are the main source are basically just propaganda. An encyclopaedia should not be in the business of laundering yellow journalism into what is supposed to be a tertiary resource. If they banned this practice, that would immediately deal with this issue.
- mmooss 9mo agoA blanket dimsissal is a simple way to avoid dealing with complexity, here both in understanding the problem and forming solutions. Obviously not all newspapers are propaganda and at the same time not all can be trusted; not everything in the same newspaper or any other news source is of the same accuracy; nothing is completely trustworthy or completely untrustworthy. I think accepting that gets us to the starting line. Then we need to apply a lot of critical thought to sometimes difficult judgments. IMHO quality newspapers do an excellent job - generally better than any other category of source on current affairs, but far from perfect. I remember a recent article for which they intervied over 100 people, got ahold of secret documents, read thousands of pages, consulted experts .... That's not a blog post or Twitter take, or even a HN comment :), but we still need to examine it critically to find the value and the flaws.
- chrisjj 9mo agoSo, a small proportion of articles were detected as bot-written, and a large proportion of those failed validation. What if in fact a large proportion of articles were bot-written, but only the unverifiable ones were bad enough to be detected?
- EdwardDiego 9mo agoHuman editors, I suspect, would pick up the "tells" of generated text, although as we know, there's a lot of false positives in that space. But it looks like Pangram is a text classifying NN trained using a technique where they get a human to write a body of text on a subject, and then get various LLMs to write a body of text on the same subject, which strikes me as a good way to approach the problem. Not that I'm in anyway qualified to properly understand ML. More details here: https://arxiv.org/pdf/2402.14873 https://arxiv.org/pdf/2402.14873
- aionwikipedia 8mo agoIf you know what you're looking for -- and no, that does not mean em-dashes, they're a low-signal indicator and people get precious about them -- your false positive rate can get pretty low. See Russell et al, "People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text." Anecdotally speaking, I'm heavily involved in AI cleanup work on Wikipedia. At this point, I can read article text typical of AI, guess within 6-12 months when that edit was made based on differences in LLM output over the years, and usually be right. (e.g., "delve" was common through early 2024 until GPT-4o killed it; I've seen theories that it's not a GPT-4o thing but rather the result of people avoiding the word when it became a meme, but its frequency dropped off hard even in edits that clearly were not reviewed or edited at all.) Given that Wikipedia has accumulated billions of edits over 25+ years, that says a lot.
- candiddevmike 9mo agoI feel like this is such a tragedy of the commons for the LLM providers. Wikipedia probably makes up a huge bulk of their dataset, why taint it? Would be interesting if there was some kind of "you shall not use our platform on Wikipedia" stance adopted.
- ohyoutravel 9mo agoI don’t think it’s the providers doing this, it’s the awful users. They’re doing the same thing on GitHub. It’s maddening.
- asadotzler 8mo agoI don't think Lockheed Martin or Raytheon are doing this, it's the awful pilots and intercept operators launching missiles into Palestinian homes. I don't think Rostec Corporation is doing this. It's only the grunts on the ground pressing the button sending heavy munitions into crowds of Ukranian civilians. These mega corporations are entirely free from blame and you're gonna see to it none of us question their role, right?
- ohyoutravel 8mo agoIt’s a bad analogy. In this case Lockheed isn’t building a killer drone and then finding a market for it, nation states are sending requirements to Lockheed based on what they want to do. Hence the label “defense contractor.” I think your analogy would hold if slop creators were creating requirements and contracting OpenAI to build the thing that lets them slop edit Wikipedia and GitHub issues. But since they aren’t, this is breaking the analogy. You are still within the edit window to change up your analogy (but unfortunately not to completely delete your post), so you have a little time to make it coherent.
- tehjoker 8mo agoI suspect that this is a very simplistic view of how R&D and the revolving door work. For example, previous Secretary of War Lloyd Austin was on the board of Ratheon https://www.opensecrets.org/revolving-door/lloyd-austin/summary?id=82688 https://www.opensecrets.org/revolving-door/lloyd-austin/summ...
- simianwords 9mo agoI find it very interesting that the main competitor to Wikipedia which is Grokipedia is taking a 180 degree approach being AI first.
- ktzar 9mo agoDidn't know about Grokipedia, I've just opened an article in it about Spain, scrolled to a random paragraph, and the information in it is plain wrong: From https://grokipedia.com/page/Spain#terrain-and-landforms https://grokipedia.com/page/Spain#terrain-and-landforms > Spain's peninsular terrain is dominated by the Meseta Central, a vast interior plateau covering about two-thirds of the country's land area, with elevations ranging from 610 to 760 meters and averaging around 660 meters Segovia is at 1.000 meters, and so is most of the top half of the "Meseta". https://en-gb.topographic-map.com/map-763q/Spain/?center=41.60371%2C-4.24218&zoom=8 https://en-gb.topographic-map.com/map-763q/Spain/?center=41.... I still stand on not trusting any of what AI spits out, be it code or text. And it takes me usually longer to check that everything is ok than doing it myself, but my brain is enticed by the "effort shortcut" that AI promised.
- charcircuit 9mo agoGrok does cite that claim as being from https://countrystudies.us/spain/30.htm https://countrystudies.us/spain/30.htm a page in Eric Solsten and Sandra W. Meditz, editors. Spain: A Country Study. Washington: GPO for the Library of Congress, 1988. The nice thing about grokipedia is that if you have counter examples like that you can provide it as evidence to change it and it will rewrite the article to be more clear.
- malfist 9mo agoYou know what other site you can provide evidence to and change to be more correct?
- homebrewer 9mo agoI don't ever edit English wikipedia because my English is not nearly up to the standard, and suggestions for improvement (worthwhile IMO) are usually ignored. Grok at least won't ignore you. (I tend to post suggestions to unpopular pages with sparse edit history, which is probably the reason for them going unnoticed.)
- crazygringo 9mo ago> That means the article contained a plausible-sounding sentence, cited to a real, relevant-sounding source. But when you read the source it’s cited to, the information on Wikipedia does not exist in that specific source. When a claim fails verification, it’s impossible to tell whether the information is true or not. This has been a rampant problem on Wikipedia always. I can't seem to find any indicator that this has increased recently? Because they're only even investigating articles flagged as potentially AI. So what's the control baseline rate here? Applying correct citations is actually really hard work, even when you know the material thoroughly. I just assume people write stuff they know from their field, then mostly look to add the minimum number of plausible citations after the fact, and then most people never check them, and everyone seems to just accept it's better than nothing. But I also suppose it depends on how niche the page is, and which field it's in.
- crabmusket 9mo agoThere was a fun example of this that happened live during a recent episode of the Changelog[1]. The hosts noted that they were incorrectly described as being "from GitHub" with a link to an episode of their podcast which didn't substantiate that claim. Their guest fixed the citation as they recorded[2]. [1]: https://changelog.com/podcast/668#transcript-265 https://changelog.com/podcast/668#transcript-265 [2]: https://en.wikipedia.org/w/index.php?title=Eugen_Rochko&diff=prev&oldid=1323127336 https://en.wikipedia.org/w/index.php?title=Eugen_Rochko&diff...
- chr15m 9mo agoHow did they know it was not LLM generated?
- michaelt 8mo agoThe false claim was added 7 Nov 2022 [1] while chatgpt wasn't released until 30 Nov 2022. [1] https://en.wikipedia.org/w/index.php?title=Eugen_Rochko&diff=next&oldid=1120454224 https://en.wikipedia.org/w/index.php?title=Eugen_Rochko&diff...
- wry_durian 9mo agoNote that this article is only about edits made through the Wiki Edu program, which partners with universities and academics to have students edit Wikipedia on course-related topics. It's not about Wikipedia writ large!
- tovej 8mo agoI've found Wiki Edu -edited pages with pages of creative writing exercises. When I have read their sources they were clumsily paraphrasing and misunderstanding the source. LLMs definitely fit the use-case of Wiki Edu students, who are just looking to pass a grade, not to look into a topic because of their interest.
- ketzu 8mo agoThat's interesting as my first thought reading the comments was "this problem seems very similar to many students writing papers just finding citations that sound correct". Sometimes it is really sad to read from (even PhD level) students on social media about their paper writing practices.
- Jepacor 8mo agoAh, so when you force students to edit Wikipedia for their courses, you get worse results than someone editing something voluntarily because they're passionate about it. That's... Hardly surprising. So it's more about how generative AI is a problem in college right now because lazy students are using it to do the work than about Wikipedia itself, I think.
- arjie 9mo ago> That means the article contained a plausible-sounding sentence, cited to a real, relevant-sounding source. But when you read the source it’s cited to, the information on Wikipedia does not exist in that specific source. This happens a lot on Wikipedia. I'm not sure why, but it does and you can see its traces through the Internet as people post the mistaken information around. One that took me a little work to fix was pointed out by someone on Twitter: https://x.com/Almost_Sure/status/1901112689138536903 https://x.com/Almost_Sure/status/1901112689138536903 When I found the source, the twitter poster was correct! Someone had decided to translate "A hundred years ago, people would have considered this an outrage. But now..." as "this function is an outrage" which honestly is ironically an outrageous translation. What the hell dude. But it takes a lot of work to clean up stuff like that! https://en.wikipedia.org/w/index.php?title=Weierstrass_function&diff=prev&oldid=1280828769 https://en.wikipedia.org/w/index.php?title=Weierstrass_funct... I had to go find the actual source (not the other 'sources' that repeated off Wikipedia or each other) and then make sure it was correct before dealing with it. A lie can travel halfway around the world...
- asyncadventure 9mo ago[dead]
- HPsquared 9mo agoThis goes much further than Wikipedia, it's just particularly visible there.
- gwern 9mo agoThanks for the LLM comment, but that's dumb. If the problem really was as bad with humans (it obviously is not), then OP wouldn't've happened: > For most of the articles Pangram flagged as written by GenAI, nearly every cited sentence in the article failed verification.
- chr15m 9mo agoAgree. I'm curious about the human contribution baseline.
- throwaway5465 9mo agoThere seems much defensiveness in the comments here along the lines of "not a new thing" and "not unique to LLM/AI". It seems to deflect, even gaslight TFA. > For most of the articles Pangram flagged as written by GenAI, nearly every cited sentence in the article failed verification. So why deflect that into convenient other pedantry (surely not under the guise tech forums often do so)? WSo why the discomfort for part of HN at an assertion AI is being used for nefarious purposes and creation of alternate 'truths'?
- emp17344 9mo agoAstroturfing or marketing, I’d guess. I’ve noticed you’re no longer allowed to say negative things about AI here without significant pushback, and I’d bet this isn’t an organic shift in perception.
- malfist 9mo agoI've found that generally people reserve down votes for posts that don't add to the conversation, in general, just like we're supposed to do. Its always been down vote city if you happen to criticize political positions that benefit libertarian technologists. But lately anything critical of AI tends to get a lot of down votes. Even on older posts that you can't find on the front page anymore... It feels inorganic
- oblio 8mo ago> Its always been down vote city if you happen to criticize political positions that benefit libertarian technologists. This varies wildly by timezone. Usually I get upvoted during European timezones and then brace for the Americans to wake up.
- shmeeed 8mo agoWell, I guess these days there's just a sizeable chunk of users on HN that earn their living with AI, one way or another. It's really only natural that some of them get thin-skinned if you shit on their lawn. I'm not a fan of certain trends either, but I wouldn't say it's inorganic. It's just a shift in the industry, and humans being human.
- theendisney 9mo agoWhat would be a truly epic application would be their own chat bot to ask about applying edit guidelines. After reading almost all of the guidelines the talkpage debates, even amoung experienced edditors, looked waaaay off. The pattern of revert first make up excuses later seems the worse newbie deterrent possible. This while it should be fine to make mistakes. Many such excuses would get debunked by a bot imediately. It simply wont do any favors. If established editors dont like it they can edit the guidelines.
- fernly 9mo agoSet aside the effect within Wikipedia and consider the larger picture, millions of people generating text with LLMs and at least some of that text being accepted as correct by millions of readers. The WikiEdu article clearly demonstrates what everyone should have known already: an LLM has no commitment to the truth. An LLM's only commitment is to correct syntax.
- Jepacor 8mo agoAn LLM's only commitment isn't to correct syntax either. It's only commitment is to popular syntax. It happens that what is popular is correct often enough for the whole thing to somewhat work but I think it's always gonna be bristle.
- thorum 8mo agoI’m honestly surprised LLMs are still screwing up citations. It does not feel like a harder task than building software or generating novel math proofs. In both those cases, of course, there is a verifier, but self-verification with “Does this text support this claim?” seems like it ought to be within the capabilities of a good reasoning model. But as I understand the situation, even the major Deep Research systems still have this issue.
- 12_throw_away 8mo ago> LLMs [...] reasoning model Found your problem right there
- Lapsa 8mo agowikipedia is great but I can't get over this - https://www.wikifunctions.org/view/en/Z16393 https://www.wikifunctions.org/view/en/Z16393
- alt227 8mo agoHave you tried going under it? /s
- huflungdung 8mo ago[dead]
- PlatoIsADisease 8mo agoITT: People saying what I got downvoted for on the last wikipedia HN thread I don't care if AI is used. I care about citations. I don't know what happened between that thread and this, maybe the narrative really changes how people respond.
- shevy-java 8mo agoSo, AI spam can degrade quality. But ... isn't this with regards to Wikipedia a much more general problem? Usually revisions are approved manually by real people. This already can be negative; takes a lot of time; no guarantee that new information is true but old information can be wrong too. To me it seems more as if the problem has much more to do with the quality control problems of wikipedia itself. Yes, AI spam fatigues here but if the quality control steps are bad then AI spam will only make this worse. But AI spam going away, does not mean the quality control steps have gotten any better. These two issues should be separate. Wikipedia needs to find better quality control mechanisms in general. And that also includes existing articles - some are written by people who are experts in the field. But they don't really explain anything at all. So, these articles appear good but are virtually useless for 98% of the people. I am not saying one should dumb down wikipedia, but you need to kind of focus primarily on the average person really - not stupid but not a godlike expert either. Explain it to, say, someone at age 18 or perhaps even a bit less than that.
- aionwikipedia 8mo ago"Usually revisions are approved manually by real people." almost never the case besides a select few articles that get heavily vandalized and require all edits to be approved. otherwise, anyone can edit Wikipedia at any time, which famously is the point of Wikipedia
- shevy-java 8mo agoAnother issue, somewhat indirectly, is Grokipedia. As we now have more and more information, the AI that is used here deliberately engineers Grokipedia to contain, shall we say it ... "alternative facts". If you look at Grokipedia, it actually looks visually better than Wikipedia, on a smartphone at the least. At the same time it tries to destroy an objective purpose, e. g. Wikipedia trying to show accurate information without any "spin". I don't believe that how AI is used by, e. g. Elon or mega-corporations, has purity and truth at heart though. We may have to look carefully at what happens to Wikipedia - it almost seems as if the attacks against Wikipedia by AI may not be merely "accidental". (Since it stores a lot of data, of course AI bots will leech off regularly, but I am talking here about purposes by organisations who may dislike democracy, for instance.)
- julienchastang 8mo ago"Never copy and paste the output from generative AI chatbots" is mentioned in the article three times. This has been my experience as well. Initial AI output can be stunning until you quickly realize that it is mostly BS, filler and pap. However, I do find LLMs to be really useful for brainstorming, ideation, sounding boards etc.