10 ms·
LLMs can get "brain rot"
- AznHisoka 1y agoCan someone explain this in laymen terms?
- PaulHoule 1y agoThey benchmark two different feeds of dangerous tweets: (1) a feed of the most popular tweets based on likes, retweets, and such (2) an algorithmic feed that looks for clickbait in the text and blend these in different proportions to a feed of random tweets that are not popular nor clickbait and find that feed (1) has more of damaging effect on the performance of chatbots. That is, they feed that blend of tweets into the model and then they ask the models to do things and get worse outcomes.
- ForHackernews 1y agoBlended in how? To the training set?
- PaulHoule 1y agoVery early training.
- sailingparrot 1y agotrain on bad data, get a bad model
- xpe 1y ago> train on bad data, get a bad model Right: in the context of supervised learning, this statement is a good starting point. After all, how can one build a good supervised model if you can't train it on good examples? But even in that context, it isn't an incisive framing of the problem. Lots of supervised models are resilient to some kinds of error. A better question, I think, is: what kinds of errors at what prevalence tend to degrade performance and why? Speaking of LLMs and their ingestion processing, there is a lot more going on than purely supervised learning, so it seems reasonable to me that researchers would want to try to tease the problem apart.
- rriley 1y agoThe study introduces the "LLM Brain Rot Hypothesis," asserting that large language models (LLMs) experience cognitive decline when continuously exposed to low-quality, engaging content, such as sensationalized social media posts. This decline, evident in diminished reasoning, long-context understanding, and ethical norms, highlights the critical need for careful data curation and quality control in LLM training. The findings suggest that standard mitigation strategies are insufficient, urging stakeholders to implement routine cognitive health assessments to maintain LLM effectiveness over time. TL;DR from https://unrav.io/#view/8f20da5f8205c54b5802c2b623702569 https://unrav.io/#view/8f20da5f8205c54b5802c2b623702569
- RobMurray 1y ago6 7
- pixelmelt 1y agoIsn't this just garbage in garbage out with an attention grabbing title?
- philipallstar 1y agoAttention is all you need.
- echelon 1y agoIn today's hyper saturated world, attention is everything: - consumer marketing - politics - venture fundraising When any system has a few power law winners, it makes sense to grab attention. Look at Trump and Musk and now Altman. They figured it out. MrBeast... Attention, even if negative, wedges you into the system and everyone's awareness. Your mousey quiet competitors aren't even seen or acknowledged. The attention grabbers suck all the oxygen out of the room and win. If you go back and look at any victory, was it really better solutions, or was it the fact that better solutions led to more attention? "Look here" -> build consensus and ignore naysayers -> keep building -> feedback loop -> win It might not just be a societal algorithm. It might be one of the universe's fundamental greedy optimization algorithms. It might underpin lots of systems, including how we ourselves as individuals think and learn. Our pain receptors. Our own intellectual interests and hobbies. Children learning on the playground. Ant colonies. Bee swarms. The world is full of signals, and there are mechanisms which focus us on the right stimuli.
- CaptainOfCoit 1y ago> continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs). TLDR: If your data set is junk, your trained model/weights will probably be junk too.
- b0gb 1y agoAIs need supervision, just like regular people... /s
- ath3nd 1y ago[dead]
- thelastgallon 1y agoIf most of the content produced by younger generations is about skibidi toilet[1] and 67[2], isn't that what LLMs are going to be trained on? [1] https://en.wikipedia.org/wiki/Skibidi_Toilet https://en.wikipedia.org/wiki/Skibidi_Toilet [2] https://en.wikipedia.org/wiki/6-7_(meme) https://en.wikipedia.org/wiki/6-7_(meme)
- micromacrofoot 1y agoonly if the trends last long enough (which they rarely do!), skibidi is already old news according to some kids I know
- ciaranmca 1y agoAgreed, “ Popularity as a better indicator”. Hypothetically you could look at popularity over time to filter out viral rot content and work out if people feel the content is useful.
- Isamu 1y agoAnother analogy to help us understand that LLMs are a useful part of what people do but are wildly misconstrued as the whole story
- draw_down 1y ago[dead]
- bbstats 1y agomaking a model worse is very easy.
- moffkalast 1y agoAh yes, something the local LLM fine tuning community figured out how to do in creative ways as soon as llama 1 released. I'm glad it has a name.
- killshotroxs 1y agoIf only I got money every time my LLM kept looping answers and telling stuff I didn't even need. Just recently, I was stuck with LLM answers, all while it wouldn't even detect simple syntax errors...
- rriley 1y agoThis paper makes me wonder the long lasting effects of the current media consumption patterns by the alpha-gen kids.
- AznHisoka 1y agowhy just kids?
- rriley 1y agoI am mostly concerned with the irreversibility part. More developed brains probably would not be affected as much.
- jama211 1y agoHave you opened facebook recently? Seems the older folk are plenty affected to me.
- FactolSarin 1y agoBut don't worry, us middle aged people are definitely immune.
- jama211 1y agoHahaha, I’m 35. Ut I feel myself getting crazier every day for sure… but at least I try and stay away from too much brain rot…
- rriley 1y agoGood point :-)
- vanderZwan 1y agoI recently saw an article about the history of Sesame Street that claimed that in the late 1960s American preschool kids watched around twenty-seven hours of television per week on average[0]. And most of that was not age-appropriate (education TV had yet to be invented). So maybe we should check in on the boomers too if we're sincere about these worries. [0] https://books.google.se/books?id=KOUCAAAAMBAJ&pg=PA48&vq=sesame+street&redir_esc=y#v=onepage&q=sesame%20street&f=true https://books.google.se/books?id=KOUCAAAAMBAJ&pg=PA48&vq=ses...
- zmailnmail786 1y ago[dead]
- avazhi 1y ago“Studying “Brain Rot” for LLMs isn’t just a catchy metaphor—it reframes data curation as cognitive hygiene for AI, guiding how we source, filter, and maintain training corpora so deployed systems stay sharp, reliable, and aligned over time.” An LLM-written line if I’ve ever seen one. Looks like the authors have their own brainrot to contend with.
- askafriend 1y agoIf it conveys the intended information then what's wrong with that? You're fighting a tsunami here. People are going to use LLMs to help their writing now and forever.
- binary132 1y agoThe brainrot apologists have arrived
- askafriend 1y agoWhy shouldn't the author use LLMs to assist their writing? The issue is how tools are used, not that they are used at all.
- grey-area 1y agoBecause they produce text like this.
- xanderlewis 1y agoIs it really so painful to just think for yourself? For one sentence? The answer to your question is that it rids the writer of their unique voice and replaces it with disingenuous slop. Also, it's not a 'tool' if it does the entire job. A spellchecker is a tool; a pencil is a tool. A machine that writes for you (which is what happened here) is not a tool. It's a substitute. There seem to be many falling for the fallacy of 'it's here to stay so you can't be unhappy about its use'.
- 1y ago
- gaogao 1y agoBrain rot texts seems reasonably harmful, but brain rot videos are often surreal and semantically dense in a way that probably improves performance (such as discussed on this German brain rot analysis https://www.youtube.com/watch?v=-mJENuEN_rs&t=37s https://www.youtube.com/watch?v=-mJENuEN_rs&t=37s). For example, Švankmajer is basically proto-brainrot, but is also the sort of thing you'd watch in a museum and think about. Basically, I think the brain rot aspect might be a bit of terminology distraction here, when it seems what they're measuring is whether it's a puff piece or dense.
- f_devd 1y agoI do not think this is the case, there has been some research into brainrot videos for children[0], and it doesn't seem to trend positively. I would argue anything 'constructed' enough will not classify as far on the brainrot spectrum. [0]: https://www.forbes.com/sites/traversmark/2024/05/17/why-kids-shows-like-cocomelon-hamper-critical-brain-development/ https://www.forbes.com/sites/traversmark/2024/05/17/why-kids...
- gaogao 1y agoYeah, I don't think surrealism or constructed is good in the early data mix, but as part of mid or post-training seems generally reasonable. But also, this is one of those cases where anthropomorphizing the model probably doesn't work, since a major negative effect of Cocomelon is kids only wanting to watch Cocomelon, while for large model training, it doesn't have much choice in the training data distribution.
- f_devd 1y agoI would a agree a careful and very small amount of above brainrot in post-training could improve certain metrics, if the main dataset didn't contain any. But given how much data current LLMs consume and how much is being produced and put back into the cycle I doubt it will miss be missed
- moritzwarhier 1y agoFor this reason, I believe thar the current surge we see in AI use for people manipulation (art is also a form of manipulation, even if unintended) is much more important than their hyped usage as a technical information processors. Brainrot created by LLMs is important to worry about, their design as "people pleasers". Their anthropomorphization can be scary too, no doubt.
- conception 1y agoThis is a potential moat for the big early players in a pre-atomic steal sort of way as any future players won’t have a non-AI-slop/dead internet to train new models on.
- andai 1y agoI encourage everyone with even a slight interest in the subject to download a random sample of Common Crawl (the chunks are ~100MB) and see for yourself what is being used for training data. https://data.commoncrawl.org/crawl-data/CC-MAIN-2025-38/segments/1757047532641.17/wet/CC-MAIN-20250905112101-20250905142101-00000.warc.wet.gz https://data.commoncrawl.org/crawl-data/CC-MAIN-2025-38/segm... I spotted here a large number of things that it would be unwise to repeat here. But I assume the data cleaning process removes such content before pretraining? ;) Although I have to wonder. I played with some of the base/text Llama models, and got very disturbing output from them. So there's not that much cleaning going on.
- deleted 1y ago[deleted]
- throwaway314155 1y ago> But I assume the data cleaning process removes such content before pretraining? ;) I didn't check what you're referring to but yes, the major providers likely have state of the art classifiers for censoring and filtering such content. And when that doesn't work, they can RLHF the behavior from occurring. You're trying to make some claim about garbage in/garbage out, but if there's even a tiny moat - it's in the filtering of these datasets and the purchasing of licenses to use other larger sources of data that (unlike Common Crawl) _aren't_ freely available for competition and open source movements to use.
- jedimastert 1y ago> purchasing of licenses to use other larger sources of data https://www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-settlement-pirated-chatbot-training-material https://www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-...
- dist-epoch 1y agoKarpathy made a point recently that the random Common Crawl sample is complete junk, and that something like an WSJ article is extremely rare in it, and it's a miracle the models can learn anything at all.
- commandlinefan 1y agoMy son just sent me an instagram reel that explained how cats work internally, but it was a joke, showing the "purr center" and "knocking things off tables" organ. It was presented completely seriously in a way that any human would realize was just supposed to be funny. My first thought was that some LLM is training on this video right now.
- Night_Thastus 1y agoI'm reminded of this 'repair' video: https://www.youtube.com/watch?v=3e6motL4QMc https://www.youtube.com/watch?v=3e6motL4QMc
- deleted 1y ago[deleted]
- koakuma-chan 1y agohttps://www.youtube.com/watch?v=sZkB11pO9R8 https://www.youtube.com/watch?v=sZkB11pO9R8
- chuckreynolds 1y agois that why chatGPT always tells me "6 7 lol"? ;)
- jdkee 1y ago" Studying “Brain Rot” for LLMs isn’t just a catchy metaphor—it reframes data curation as cognitive hygiene for AI, guiding how we source, filter, and maintain training corpora so deployed systems stay sharp, reliable, and aligned over time." Is this slop?
- Profan 1y ago... it sure reads like slop and you know what they say, if it walks like slop, quacks like slop and talks like slop, it's probably slop
- deleted 1y ago[deleted]
- earth2mars 1y agoduh! isn't that obvious. is this some students wanted a project with pretty graphs on writing experience?! I am not trying to be cynical or anything. just questioning the obvious thing here.
- antegamisou 1y agoMy Goodness, looks like Computer 'Science' is a complete euphemism now.
- guelo 1y agoIt's turning into a social science.
- nakamoto_damacy 1y agoOur metaphorical / analogical muscle is too well developed. Maybe there is a drug we can take to reduce how much we lean into it. If you look at two random patterns of characters and both contain 6s you could say they are similar (because you’re ignoring that the similarity is less than 0.01%). That’s how comparing LLMs to brains feels like. Like roller skates to a cruise ship. They both let you get around.
- buyucu 1y agoI don't understand why people have a hard time understanding 'garbage in, garbage out'. If you train your model on junk, then you will have a junk model.
- kayodelycaon 1y agoI suspect this is “the computer is always right”. A lot of people think computers have better answers than people. AI is just another type of computer. It knows a lot of things and sounds confident. Why wouldn’t it be right?
- nomel 1y ago"Brain rot" is just the new term for "slang that old people don't understand". "Cool" and "for real" are no different than "rizz" and "no cap". You spoke "brain rot" once, and "cringed" when your parents didn't understand. The cycle repeats.
- kcatskcolbdi 1y agoThis both has nothing to do with the linked article (beyond the use of brain rot in the title, but I'm certain you must have read the thing you're commenting on, surely) and is simply incorrect. Brain rot in this context is not a reference to slang.
- nomel 1y ago> Brain rot in this context is not a reference to slang. I suppose I should have replied to one of the many comments here where my response is the correct context, rather than a top level.
- Razengan 1y agoBased on my hobby of collecting old gaming magazines, I think 1980s UK had the best slang.
- buellerbueller 1y agoBy all means, let's make sure the LLMs have healthier media diets than the humans. We wouldn't want the humans to realize they are being dumbed down into cattle. /s
- themafia 1y ago> as cognitive hygiene LLMs are not cognizant. It's a terrible metaphor. It hides the source of the issue. The providers cheaped out on sourcing their data and now their LLMs are filled with false garbage and copyrighted material.
- donaldihunter 1y agoLikewise, cognitive decline isn't what's happening here since that would require cognition. At best it is a simulation of cognitive decline.
- numpad0 1y agoNot surprising that trending tweets as data is junk, not only from brainrots-be-brainrots perspective: trending tweets are contextual. They don't make sense without the rest of the timeline. And now I know why bots on Twitter don't even work, even with humans in it - they're shooting blind.
- dankai 1y agoNaive question: Whats new about the finding that data quality matters when training an LLM?
- donkeylazy456 1y agocan't wait LLM says "tung x9 sahur" without any context.
- snorbleck 1y agointeresting, since trivial or unchallenging online content rots actual brains too!
- skopje 1y ago"Trivial or Unchallenging Content" (points to Twitter). I love it.
- alexchantavy 1y agoOff topic completely but I really like the font used in this blog
- liqilin1567 1y agoI like the gradient font style in the overview :)
- f4uCL9dNSnQm 1y agoI hate it but I still found it interesting that something like it can be done with just CSS (-webkit-linear-gradient) Edit: I noticed that replacing it with "standard" "linear-gradient" reverses the direction of gradient.
- Version467 1y agoSo they trained LLM's on a bunch of junk and then notice that it got worse? I don't understand how that's a surprising, or even interesting result?
- Perz1val 1y agoI seen claims that you can train the models with anything, so it would be a research to check that
- nazgul17 1y agoThey also tried to heal the damage, to partial avail. Besides, it's science: you need to test your hypotheses empirically. Also, to draw attention to the issue among researchers, performing a study and sharing your results is possibly the best way.
- yieldcrv 1y agoI don’t understand, so this is just about training an LLM with bad data and just having a bad LLM? just use a different model? dont train it with bad data and just start a new session if your RAG muffins went off the rails? what am I missing here
- ramon156 1y agoDo you know the conceot of brain rot? The gist here is that if you train on bad data (if you fuel your brain with bad information) it becomes bad
- yieldcrv 1y agoI don’t understand why this is news or relevant information in October 2025 as opposed to October 2022
- chipsrafferty 1y agoThe idea of brain rot is that if you take a good brain and give it bad data it becomes bad. Obviously if you give a baby (blank brain) bad data it will become bad. This is about the rot, though.
- Rileyen 1y agoAfter reading this, I just felt like everyone already knows the data is a mess, but no one really cares. We feed the models a bunch of junk, then act surprised when they start getting dumber. Honestly, did we even need a study to figure that out?
- mikeiz404 1y ago> "brain rot", "Thought-skipping", "primary lesion", "Cognitive Declines", ... In general using these medical/biological metaphors doesn't seem like a good idea in things like computer science research papers and similar. Their use forces many inaccurate comparisons (when compared in detail) and they engender human qualities to what are already forgotten to be just computer models. I get this may be done with a slight tongue-in-cheek but with research papers there is also the risk that these terms start to be adopted. And undoing that would be a much taller order in either the research community or general media. Maybe I am just yelling at clouds.
- Animats 1y agoThe two big problems listed: * Thought-skipping as the primary lesion: models increasingly truncate or skip reasoning chains, explaining most of the error growth. * Popularity as a better indicator: the popularity, a non-semantic metric, of a tweet is a better indicator of the Brain Rot effect than the length in M1. That's what you'd expect. Popular culture content tends to jump from premise to conclusion without showing the work. Train on popular culture and you get that. Really, what's supposed to come from training on the Twitter firehose? (Can you still buy that feed? Probably not.) This is a surprise-free result. At least have a curated model (no social media) and a junk model to compare.
- deleted 1y ago[deleted]
- dee_s101 1y agoIs it just me or has GPT5 turned into a bit of a donkey?
- nathias 1y agospoken as a true LLM
- loloquwowndueo 1y agoTralalero tralala
- RLAIF 1y ago[dead]
- gamerrk 1y agoSo LLMs are indeed human like
- b800h 1y agoIt's like showing modern children's TV to kids.
- ederamen 1y agoLLMs are brain rot
- devin 1y agoSpeaking of, someone shared with me an AI financial challenge where they are pitting the LLMs against one another to make trades, manage risk, etc. and then track their performance against one another with starting capital of 10,000 USD each. Their starting portfolios are ludicrous. They are trading BTC, XRP, DOGE, etc. I thought the idea was somewhat interesting, but then I felt like the only reasonable takeaway I had was that these models have intense brainrot from consuming twitter, reddit, etc. and as such have a completely warped view of "finance".
- 3Sophons 1y agowhy they don't prompt ai to avoid using dashes. and bullet points etc