10 ms·
What can we take away from the ‘stochastic parrot’ saga?
- hulitu 2y ago> The Parrot Is Dead The page says "Something has gone terribly wrong :(". He's not dead, he's resting.
- treetalker 2y ago> The parrot is dead. Don’t be the shopkeeper. Continuing the metaphor, we never wanted to work in a pet shop in the first place. We wanted to be … lumberjacks! Floating down the mighty rivers of British Columbia! With our best girls by our side!
- int_19h 2y agoThe whole "debate" around LMs being stochastic parrots is strictly a philosophical one, because the argument hinges on a very specific definition of intelligence. Thought experiments such as Chinese room make this abundantly clear.
- jfengel 2y agoThat's about the only thing the Chinese room makes clear. The argument otherwise strikes me as valueless.
- chongli 2y agoNo, the Chinese Room is essentially the death knell for the Turing Test as a practical tool for evaluating whether an AI is actually intelligent.
- int_19h 2y agoOnly if you buy into the whole premise, which is dubious to say the least, and is a good example of begging the question.
- chongli 2y agoWhat exactly is dubious about faking an AI with a giant lookup table and fooling would-be Turing Test judges with it? Or did you mean the Turing Test is dubious? Because that’s what the Chinese Room showed (back in 1980).
- Kim_Bruning 2y agoIs it just me, or would a giant lookup table fails much weaker tests that you can throw against it. (for instance: just keep asking it to do sums until it runs out)
- chongli 2y agoWell presumably the lookup table can have steps you go through (produce this symbol, then go to row 3568), with state as well, so it’s more like a Turing machine than a single-shot table lookup.
- Kim_Bruning 2y agoThat's a motte-and-baily really. If one starts out with a LUT and then retreats to a turing machine when challenged, that is. If our friend in the Chinese room is in fact permitted to operate some sort of Turing machine, I'd make a very different set of inferences!
- int_19h 2y agoThe dubious part is claiming that a large enough lookup table is not intelligent. It's basically asserted on the grounds "well of course it isn't", but no meaningful arguments are presented to this effect.
- pixl97 2y agoThe Chinese Room is a sophisticated way for humans to say they don't understand systematic systems and processes.
- chongli 2y agoNo, I think the Chinese Room is widely misunderstood by non-philosophers. The goal of the argument is not to show that machines are incapable of intelligent behaviour. Even a thermostat can show intelligent behaviour. The issue for the thermostat is that all the intelligence has happened ahead of time.
- pixl97 2y agoI mean that is just talking about probabilistic systems where the probability is either zero or one. When you get in to probabilistic systems with a wider range of options than that, can you can feed back new data into the system you start getting systems that look adaptively intelligent.
- chongli 2y agoThere's nothing inherent to the Chinese Room thought experiment that prohibits the operator inside from using a random number source combined with an arbitrarily sophisticated sequence of lookup tables to produce "stochastic parrot" behaviour. Anyone who has played as a Dungeon Master in D&D has used dice and tables for this. Similarly for feedback. All the operator needs to do is log each input in a file marked for that user and then when new input arrives the old input is used as context in the lookup table. Ultimately, arbitrarily sophisticated intelligent behaviour can be produced without the operator ever having any understanding of it.
- red75prime 2y agoThe Chinese Room didn't show anything. It's a misleading intuition pump that for some reason is being brought up again and again. Just think about it. All the person in the room does are mechanical manipulations. The person's understanding or not understanding of Chinese language is causally disconnected from everything including functioning of the room. There's zero reasons to look at their understanding to make conclusions about the room. The second point is that it's somehow about syntactic manipulation specifically. But why? What would change if the person in the room is solving QM-equations of your brain quantum state? Would it mean that the perfect model of your brain doesn't understand English language?
- chongli 2y agoThe Chinese Room argument is silent on the question of the necessary and sufficient conditions for intelligence, thinking, and understanding. It’s an argument against philosophical functionalism in the theory of mind which states that it is sufficient to compare inputs and outputs of a system to infer intelligence. The Chinese Room is also an argument that mere symbolic manipulation is insufficient to model a human mind. As for the QM-equations, the many-body problem in QM is your enemy. You would need a computer far larger than the entire universe to simulate the quantum states of a single neuron, never mind a human brain.
- red75prime 2y agoAgain. It's not an argument. It's a misleading intuition pump. Or a failure of philosophy to filter away bullshit, if you will. Please, read again what I wrote. Regarding "larger than Universe": "the argument" places no restrictions on runtime or space complexity of the algorithm. It's just another intuitive notion: syntactic processing is manageable by a single person, other kinds of processing aren't. I'm sorry for the confrontational tone, but I really dismayed that this thing keeps floating and keeps being regarded as a foundational result.
- jltsiren 2y agoIt's important to understand the argument in context. Theoretical computer science established early that input-output behavior captures the essence of computation. The causal mechanism underlying the computation does not matter, because all plausible mechanisms seem to be fundamentally equivalent. The Chinese room argument showed that this does not extend to intelligence. That intelligence is fundamentally a causal rather than computational concept. That you can't use input-output behavior to tell the difference between an intelligent entity and a hard-coded lookup table. On one level, LLMs are literally hard-coded lookup tables. But they are also compressed in a way that leads to emergent structures. If you use the LLM through a chat interface, you are interacting with a Chinese room. But if the LLM has other inputs beyond your prompt, or if it has agency to act on its own instead of waiting for input, it's causally a different system. And if the system can update the model on its own instead of using a fixed lookup table to deal with the environment, this is also a meaningful causal difference.
- kentonv 2y ago> The Chinese room argument showed that this does not extend to intelligence. Searle's argument doesn't actually show anything. It just illustrates a complex system that appears intelligent. Searle then asserts, without any particular reasoning, that the system is not intelligent, simply because, well, how could it be, it's just a bunch of books and a mindless automaton following them? It's a cyclic argument: A non-human system can't be intelligent because, uhh, it's not human. This is wrong. The room as a whole is intelligent, and knows Chinese. People have, of course, made this argument, since it is obivous. Searle responds by saying "OK, well now imagine that the man in the room memorizes all the books and does the entire computation in his head. Now where's the intelligence???" Ummm, ok, now the man is emulating a system in his head, and the system is intelligent and knows Chinese, even though the man emulating it does not -- just like how a NES emulator can execute NES CPU instructions even though the PC it runs on doesn't implement them. Somehow Searle just doesn't comprehend this. I guess he's not a systems engineer. As to whether a lookup table can be intelligent: I assert that a lookup table that responds intelligently to every possible query is, in fact, intelligent. Of course, such a lookup table would be infinite, and thus physically impossible to construct.
- gwern 2y agoIt was not 'strictly philosophical' in the way some things like the Chinese Room argument are; in Chinese Room, it's stipulated that the Room is pragmatically capable of responding like a native Chinese speaker (it is somehow implemented super-fast and can chat in Mandarin with you and pass a Mandarin Turing Test). However, the stochastic parrot arguments (and Gary Marcus in many of his writings) made specific, unambiguous empirical predictions about how LLMs would never be able to do many things, such as 'add numbers'. For example, in the original Bender & Koller 2020 paper "Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data" (which lays out the core ideas which are the justification for the 2021 paper that actually introduced the rhetoric of 'stochastic parrot') made many clear, falsifiable statements about what LLMs would never be able to do; here's one of them: https://aclanthology.org/2020.acl-main.463.pdf#page=14 https://aclanthology.org/2020.acl-main.463.pdf#page=14 > To get a sense of how existing LMs might do at such a task, we let GPT-2 complete the simple arithmetic problem 'Three plus five equals'. The five responses below, created in the same way as above, show that this problem is beyond the current capability of GPT-2, and, we would argue, any pure LM. One could say many things about this claim, but not that it is a 'strictly philosophical one'.
- pyfon 2y agoDead parrot is a Monty Python reference. Also where the Python language get's its name.
- petermcneeley 2y agoAnd yet... https://uwaterloo.ca/news/media/qa-experts-why-chatgpt-struggles-math https://uwaterloo.ca/news/media/qa-experts-why-chatgpt-strug...
- Tossrock 2y agoMy favorite part of the "stochastic parrot" discourse was all the people repeating it without truly understanding what they were talking about.
- parpfish 2y agoMy take on stochastic parrots is the similar to the authors concluding section. This debate isn’t about the computations underlying cognition, it’s about wanting to feel special. The contention that “it’s a stochastic parrot” usually implied “it’s merely a stochastic parrot, and we know that we must be so much more than that, so obviously this thing falls short”. But… there never was any compelling proof that we’re anything more than stochastic parrots. Moreover, those folks would say that any explanation of cognition falls short because they can always move the goal posts to make sure that humans are special.
- MichaelZuo 2y agoWell it’s even more dismal in reality. Gather enough parrots on a stage and at least one can theoretically utter a series of seemingly meaningfuly word-like sounds that is legitimately novel, that has never been uttered before. But I doubt any randomly picked HN user will actually accomplish that in fact before say age 40. Most people just don’t ever get enough meaningful speaking opportunities to make that statistically likely. There’s just too many tens of billions of people that have already existed and uttered words.
- anon373839 2y agoThat not reality, it’s theory.
- MichaelZuo 2y agoCan you write down the actual argument? It seems to be plausible, to me, given enough parrots.
- thfuran 2y agoNovel utterances happen all the damn time. See https://venturebeat.com/business/15-of-all-google-searches-are-unique-which-is-also-a-problem-for-chatbots/ https://venturebeat.com/business/15-of-all-google-searches-a... for tangential evidence. Edit: actually that looks like it's just an offhand mention of Google's initial report, but I don't really feel like spending more time tracking down details to rebut so silly a claim.
- devmor 2y agoI am getting fairly tired of seeing articles about LLMs that claim “[insert criticism] was wrong” but offer nothing other than the opinion of the author’s interpretation of a collection of other people’s writings with limited veracity.
- lukasb 2y agoGiven LLMs' OOD performance the parrot metaphor still looks good to me
- deleted 2y ago[deleted]
- skybrian 2y agoThere’s still a lot to learn about how LLM’s do things. They could be doing it in either a deep or a shallow way (parroting information) depending on the task. It’s not something to be settled once and for all. So what’s “dead?” Overconfidently assuming you can know how an LLM does something without actually investigating it.
- alganet 2y ago"ESSE É UM ESPERTO", or, "this is a smart one", in portuguese. So far, LLM models have not demonstrated grasp on dual language phonetic jokes and false cognates. Humans learn a second language very quickly, and false cognates that work on phonetics are the first steps in doing so, doesn't require a genius to understand. I am yet to see an LLM that can demonstrate that. They can translate it, or repeat known false cognates, but can't come up with new ones on the spot. If they do acquire that, we will come up with another creative example of what humans can do that machines can't.
- NooneAtAll3 2y agodo deaf/mute people recognize phonetic bilingual jokes?
- alganet 2y agoSign language in Brazil (Libras) is different from ASL. I am sure there are false cognate signs among them, and dual users of both sign languages can appreciate them.
- _heimdall 2y agoI have a deaf friend who can read lips in two languages. As far as I am aware she can pick up humor of all kinds in both. She knows ASL as well, but I don't think she knows any other dialect of sign language (is dialect the right term? I'm not actually sure).
- nopinsight 2y agoFor the skeptics: Scoring just 10% or so in Math-Perturb-Hard below the original MATH Level 5 (hardest) dataset seems in line with or actually better than most people would do. Does that mean most people are merely parrots too? https://math-perturb.github.io/ https://math-perturb.github.io/ https://arxiv.org/abs/2502.06453 https://arxiv.org/abs/2502.06453 Leaderboard: https://math-perturb.github.io/#leaderboard https://math-perturb.github.io/#leaderboard Anyone who continues to use the parrot metaphor should support it with evidence at least as strong as the “On the Biology of a Large Language Model” research by Anthropic which the article refers to: https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...
- _heimdall 2y agoYou seem to be coming with the assumption that the difference between parrots and what many would consider intelligence is math, or that math is a reliable indicator of those different groups. What makes you believe that is the case?
- nopinsight 2y agoSolving hard math problems requires understanding the structure of complex mathematical reasoning. No animal is known to be capable of that. Most definitions and measurements of intelligence by most laypeople and psychologists include the ability to reason, with mathematical reasoning widely accepted as part of or a proxy for it. They are imperfect but “intelligence” does not have a universally accepted definition. Do you have a better measurement or definition?
- bluefirebrand 2y ago> Solving hard math problems requires understanding the structure of mathematical reasoning Not when you already know all of the answers and just have to draw a line between the questions and the answers!
- 2y ago
- NooneAtAll3 2y agomy personal anecdote about stochastic parrot arguments is that the argument itself became so repetitive that its defenders sound as parrots...
- zeofig 2y agoI'm so glad We have all Decided this Together and we can now Enjoy the Koolaid
- jrmg 2y agoFor a while, some people dismissed language models as “stochastic parrots”. They said models could just memorise statistical patterns, which they would regurgitate back to users. … The problem with this theory, is that, alas, it isn’t true. If a language model was just a stochastic parrot, when we looked inside to see what was going on, we’d basically find a lookup table. … But it doesn’t look like this. But does that matter? My understanding is that, if you don’t inject randomness (“heat”) into a model while it’s running, it will always produce the same output for the same input. In effect, a lookup table. The fancy stuff happening inside that the article describes is, in effect, [de]compression of the lookup table. Of course, maybe that’s all human intelligence is too (the whole ‘free will is an illusion in a deterministic universe’ argument is all about this) - but just because the internals are fancy and complicated doesn’t mean it’s not a lookup table.
- red75prime 2y agoEverything can be represented as a lookup table. Well, at least everything we can rigorously reason about. Because set theory can serve as a foundation of mathematics. And relations there are sets of pairs (essentially lookup tables). I guess it means that we can throw away the notion that "it can be represented as a lookup table" has some profound meaning. Without further clarifications, at least. Finite/infinite lookup table, can/can't be constructed in time polynomial in the number of entries. Things like that.
- derbOac 2y agoThis struck me as a strawman argument against the "stochastic parrot" interpretation. I really disagree with this premise in particular: "if a language model was just a stochastic parrot, when we looked inside to see what was going on, we’d basically find a lookup table." I'm not sure how the latter follows from the former at all. As someone else pointed out, I think there's deep philosophical issues about intelligence and consciousness underlying all this and I'm not sure it can be resolved this way. In some sense, we all might be stochastic parrots — or rather, I don't think the problem can be waved away without deeper and more sophisticated treatments on the topic.
- _heimdall 2y agoThis argument isn't particularly compelling in my opinion. I don't actually like the stochastic parrot argument either to be fair. I feel like the author is ignoring the various knobs (randomization factors may be a better term) applied to the models during inference that are tuned specifically to make the output more believable or appealing. Turn the knobs too far and the output is unintelligible garbage. Don't then them far enough and the output feels very robotic or mathematical, its obvious that the output isn't human. The other risk of not turning the knobs far enough would be copyright infringement, but I don't know if that happens often in practice. Claiming that LLMs aren't stochastic parrots without dealing with the fact that we forced randomization factors into the mix misses a huge potential argument that they are just cleverly disguised stochastic parrots.
- mbauman 2y agoYes, this really seems like an argument between two contrived straw people at the absolute extremes.
- chongli 2y agoThis seems like it was inevitable. Most people do not understand the meaning of the word "stochastic" and so they're likely to simply ignore it in favour of reading the term as "_____ parrot." What you have described, a probability distribution with carefully-tuned parameters, is perfectly captured by the word stochastic as it's commonly used by statisticians.
- amenhotep 2y agoHuman brains are similarly finely tuned and have similar knobs, it seems to me. People with no short term memory have the same conversations over and over again. Drunk people tend to be very predictable. There are circuits that give us an overwhelming sense of impending doom, or euphoria, or the conviction that our loved ones have been replaced by imposters. LLMs with very perturbed samplers bear, sometimes, a striking resemblance to people on certain mind-altering substances.
- _heimdall 2y ago
- kerkeslager 2y ago> This kind of circuitry—to plan forwards and back—was learned by the model without explicit instruction; it just emerged from trying to predict the next word in other poems. This author has no idea what's going on. The AI didn't just start trying to predict the next word in other poems, it was explicitly instructed to do so. It then sucked in a bunch of poems and parroted them out. And... the author drastically over-represents its success with a likely cherry-picked example. When I gave Claude lines to rhyme with, it gave me back "flicker" to rhyme with "killer" and "function" to rhyme with "destruction". Of the 10 rhymes I tried, only two actually matched two syllables ("later/creator" and "working"/"shirking")I'm not sure how many iterations the author had to run to find a truly unusual rhyme like "rabbit/grab it", but it pretty obviously is selection bias. And... I actually agree with the other poster who says that part of this stochastic parrot argument is about humans wanting to feel special. Exceptionalism runs deep: we want to believe our group (be it our nation, our species, etc.) are better than other groups. It's often wrong: I don't think we're particularly unique in a lot of aspects--it's sort of a combination of things that makes us special if we are at all. AI are obviously stochastic parrots if you know how they work. The research is largely public and unless there's something going on in non-public research, they're all just varieties of stochastic parroting. But, these systems were designed in part off of how the human brain works. I do no think it's in evidence at all that humans aren't stochastic parrots. The problem is that we don't have a clear definition of what it means to understand something that's clearly distinct from being a stochastic parrot. At a certain level of complexity of stochastic parroting, a stochastic parrot is likely indistinguishable from someone who truly understands concepts. I think ultimately, the big challenge for AI isn't that it is a stochastic parrot (and it is a stochastic parrot)--I think a sufficiently complex and sufficiently trained stochastic parrot can probably be just as intelligent as a human. I think the bigger challenge is simply that entire classes of data simply have not been made available to AI, and can't be made available with current technology. Sensory data. The kind of data a baby gets from doing something and seeing what happens. Real-time experimentation. I think a big part of why humans are still ahead of AI is that we have a lot of implicit training we haven't been able to articulate, let alone pass on to AI.
- anothernewdude 2y ago> If a language model was just a stochastic parrot, when we looked inside to see what was going on, we’d basically find a lookup table I disagree right away. There are more sophisticated probability models than lookup tables. > It'd be running a search for the most similar pattern in its training data and copying this. Also untrue. Sophisticated probability models combine probabilities based on combining all the bits of context, and by fuzzing similar tokens together via compressing (i.e. you don't care what particular token is used, just that a similar one is used.) They're parrots, just better parrots than this person can conceive of.
- cadamsdotcom 2y agoWhy the existential crisis? LLMs are stochastic parrots and so are humans - but humans still get to be special. Humans are more stochastic as we act on far more input than a several-thousand token prompt.
- agentultra 2y agoThe conclusion goes into that glassy-eyed realm of, “what if we’re no better than the algorithm?” Problem is, we don’t even know what makes us think. So you can jump to any conclusion and nobody could really tell if you’re wrong. We do know how transformers and layers work. They’re algorithms that crunch numbers. A great deal of numbers. And we can use the training set to generate plausible outputs given some input. Yes, stochastic parrot is a reduction of all the technical sophistication in LLMs. But it’s not entirely baseless. At the end of the day it is copying what’s in the training data. In a very clever way. However, resist the temptation to believe we understand human brains and human thought. And resist the temptation to anthropomorphize algorithms. It’s data and patterns.
- getnormality 2y ago"Stochastic parrot" is a deepity, an ambiguous phrase that blends a defensible but trivial meaning with a more profound but false meaning. It's true, and trivial, that all next word predictors are stochastic and are designed to generate output based on information from their training data. The claim that this generation merely "parrots" the training data is more significant, but obviously false if you interact with these models at all.