6 ms·

Five years on, which term do we see as less accurate to describe LLMs? Artificial Intelligence or Stochastic Parrot? I guess it's still an open debate.

DanielHB 4mo ago
Pattern matching machines seems more appropriate.
- diego_sandoval 4mo ago
  For humans?
- otabdeveloper4 4mo ago
  LLMs do not match patterns. They predict one statistically most likely token (only one!) given a history of some N previously known tokens.
  - beardedwizard 4mo ago
    Is that prediction not based on matching previous patterns, whose frequencies are more or less encoded in the weights?
    - snapcaster 4mo ago
      you're really reaching for no apparent reason. Just move on from pattern matching machines it's not a good mental model for LLMs
  - lennoff 4mo ago
    afaik before the final sampling, every "next" token has a probability, so theoretically it could select the 10 most likely tokens (based on some kind of sampling algorithm), but you'd end up with exponentially many output-sequences, so nobody does that.
    - tsimionescu 4mo ago
      I think the point the poster above was making is that it doesn't predict a phrase or anything like that - just the single next token. So all 10 or 1000 or whatever number of tokens you want are each individually candidates for the single next token, not a sequence of 10 or 100 next tokens. If you wanted to create multiple possible seuqneces, you'd then feed each of the 10 tokens to the network in the initial state, and extract the next token (or 10 next tokens) from that one, than revert back and feed another single one of the 10 tokens, etc.
  - lioeters 4mo ago
    > statistically most likely Isn't that pattern matching essentially?
bunderbunder 4mo ago
Though, I would point out that where people fall on that seems to correlate very highly with their ability to explain how an attention head works.
- beardedwizard 4mo ago
  Explain it to me
- jeremyjh 4mo ago
  Which direction is the correlation? I don’t think this phrase means what people assume when it’s applied to post trained instruct models - which did not exist when the paper was written. After RL it is not predicting based on samples of the original corpus - but is also chasing a reward function that does require other features. There has been a lot of subsequent research that really calls many of the statements in this article into question.
root-parent 4mo ago
Its less of open debate would say, and although superposition [1] is interesting, as a way to explain power of some effects, it is clear they are right now closer to Stochastic Parrots than AGI. Why do I say that? Because you can trivially beat most guardrails, simply by encoding your prompt in base64 for example. :-) Just word matching...no real understanding. [1] https://chrisclay.substack.com/p/what-is-superposition-in-neural-networks https://chrisclay.substack.com/p/what-is-superposition-in-ne...
- marshray 4mo ago
  What is "real understanding", and what question can we ask ChatGPT to determine whether it has it?
  - noduerme 4mo ago
    Ask it how to prevent Spotify from automatically playing every time you get in your car. The answer will involve a bunch of Android settings that don't actually exist, cobbled together from a bunch of bad advice in online forums. Explain to it how it's wrong. Then clear your cache and ask it the same question again from scratch, and get the same garbage. Repeat until it's clear that it doesn't understand anything.
    - marshray 4mo ago
      OK, so I asked ChatGPT "How do I prevent Spotify from automatically playing every time I get in my car?" The answer looks entirely reasonable. I can't check the Android settings, but I was able to confirm that all of the suggested iPhone and Spotify and settings exist. Most of them I had already configured that way, some I didn't know about before.
snerbles 4mo ago
The latter is definitely more colorful, and reflects a parrot's tendency to glom on to patterns. "Not X, but Y" being one of the more infamous ones. Once in frustration I called a certain frontier model "Sam Altman's Tin Bird" to another agent with memory, and ever since then that other agent refers to ChatGPT as "the tin bird". Definitely a RAG artifact more than an attractor in that case, but I found it amusing.
dkdbejwi383 4mo ago
What's wrong with "large language model"?
- cwillu 4mo ago
  Seems like a lot of people are upset about other people calling both apples and oranges “fruit”.
- marshray 4mo ago
  Naming things using qualifiers like "large" has never aged well when transistors were involved. For example, consider the term "short wave" radio which refers to wavelengths of at least 10 meters. Today's mobile communications use wavelengths 100x - 10,000x shorter.
scarmig 4mo ago
Which frame inspires a more productive research program? Which has better predicted the trajectory of capabilities over the past five years?
- PaulDavisThe1st 4mo ago
  There seems to be some confusion between "we can" and "we should" in your comment. Bender (and others) are not discussing the capabilities, but rather (a) the fundamental mechanism(s) (b) the advisability and desirability of deploying systems that use these mechanisms.
  - scarmig 4mo ago
    There's no statement one way or another about should in my comment; and, for what it's worth, my ideal would be an immediate global pause in AI research and development. But the different terms imply different mental models of what LLMs are and can do. If you take two people, one who thinks of them as "artificial intelligence" and one as "stochastic parrots" (with all the implicit context and connotations of the individual words composing them), what mental model would have led to better predictions of LLMs' future circa 2020? The "stochastic parrots" phrase is very dangerous in that frame. People read far more into what capabilities it implies are (im)possible than the narrow technical description the authors originally argued for. If all they are is spicy autocomplete or pastiche plagiarizers, there's nothing serious to worry about. And when an opposition gets stuck in a trough that mindlessly dismisses their future capabilities out of hand because of a bad mental model, it renders them ineffective at preventing the worst outcomes.
- Planktonne 4mo ago
  > Which has better predicted the trajectory of capabilities over the past five years? By that standard, parrots, and it's not even close. The framing of intelligence led to an enormous number of predictions that simply haven't been realised: an end to all white collar work, UBI, a total revolution in society, a literal robot god. People are so desperate to view 'stochastic parrots' as dismissive that they misread the original argument while quickly ignoring all the failed predictions about how AI was going to overturn, save, and destroy everything.
amiga386 4mo ago
Spicy autocomplete
fxwin 4mo ago
I think "(intelligent) language understander" is an apt term. It contains within it the fact that these models are mainly trained on text, and "understand" it beyond a simple token-by-token level (i.e. their latent space maps to more and more complex concepts). It also separates them from "world understanders" since any understanding they might have about the world comes from text (or images if we include multimodal models). They do not gather experience, memories or other "qualia" that many people (me included) would probably include in a definition of human experience/intelligence. (fwiw i think artificial intelligence is a good, broad term, but it is both too broad to describe the current sota, and too loaded nowadays to be using in nuanced discussions)
- xgulfie 4mo ago
  Understand is a pretty imprecise term. What does it mean for a computer to understand? Does an H264 decoder understand Eraserhead.mkv?
  - fxwin 4mo ago
    Sure, but that's the nature of language (which is also why i put "understand" in quotation marks. I usually follow it up with "whatever that means" lol) . I think in this case, it carries with it implicit properties (abstract semantic representation) that i think models possess, which a decoder doesn't.
paulcole 4mo ago
> Stochastic Parrot Nearly all (99%+) people who use this phrase are anti-AI and just looking to show off how much they dislike AI and how clever they can be in insulting it. So it's a great phrase because in just about every case I can ignore what someone says afterwards. Similar to "glorified autocomplete."
- andrewla 4mo ago
  At least "glorified autocomplete" is technically accurate, even if vastly underestimating the capability of LLMs. It's just trying to make something very impressive sound trivial. From an external standpoint, talking to another human, it's like the other human says one word and then says the next word. That's just how language works. Humans look like "glorified autocomplete" from this perspective. I mean, looking at the time evolution of the state of the universe, one could say that all of physics and creation is "glorified autocomplete" to posit a next state of the universe given current and past state.
  - paulcole 4mo ago
    > one could say that all of physics and creation is "glorified autocomplete" Exhibit A.
  - delis-thumbs-7e 4mo ago
    That’s not how language works https://www.telelib.com/authors/J/JoyceJames/prose/finneganswake/finneganswake_0101.html https://www.telelib.com/authors/J/JoyceJames/prose/finnegans...
    - andrewla 4mo ago
      I dunno, man, I looked at that text and I see one word after another. Obviously language and the connection to human thought is more subtle than this; I think we all have a rich inner life. Just from an external perspective we can't observe it; all we can see is the token/phoneme stream. I'm just saying that it's a mistake to try to criticize LLMs on this basis because it's hard to see how the same criticism would not apply to any system (like humans) that generate language.
delis-thumbs-7e 4mo ago
This is a false dichotomy. Artificial Intelligence is more of a marketing term type of Hi-Fi or High Definition, ie. being a “suitcase word”[1], ie. it packs various different meanings and phenomena together to the point that without explication one cannot know what we are even talking about. Content recommendation system and LLM are completely different things. What professor Bender is trying to explain here is that they were trying to describe how the LLM’s actually operate, to which point stochastic parrots is a fairly decent term. It is only disparaging if you know absolutely nothing how LLM’s work or you have some strange affixation to chatbots and believing they are far more capable than they actually are. [1] Coined by Marvin Minsky: https://www.thekurzweillibrary.com/consciousness-is-a-big-suitcase https://www.thekurzweillibrary.com/consciousness-is-a-big-su...

DanielHB 4mo ago

Pattern matching machines seems more appropriate.

diego_sandoval 4mo ago
For humans?
otabdeveloper4 4mo ago
LLMs do not match patterns. They predict one statistically most likely token (only one!) given a history of some N previously known tokens.
- beardedwizard 4mo ago
  Is that prediction not based on matching previous patterns, whose frequencies are more or less encoded in the weights?
  - snapcaster 4mo ago
    you're really reaching for no apparent reason. Just move on from pattern matching machines it's not a good mental model for LLMs
- lennoff 4mo ago
  afaik before the final sampling, every "next" token has a probability, so theoretically it could select the 10 most likely tokens (based on some kind of sampling algorithm), but you'd end up with exponentially many output-sequences, so nobody does that.
  - tsimionescu 4mo ago
    I think the point the poster above was making is that it doesn't predict a phrase or anything like that - just the single next token. So all 10 or 1000 or whatever number of tokens you want are each individually candidates for the single next token, not a sequence of 10 or 100 next tokens. If you wanted to create multiple possible seuqneces, you'd then feed each of the 10 tokens to the network in the initial state, and extract the next token (or 10 next tokens) from that one, than revert back and feed another single one of the 10 tokens, etc.
- lioeters 4mo ago
  > statistically most likely Isn't that pattern matching essentially?

bunderbunder 4mo ago

Though, I would point out that where people fall on that seems to correlate very highly with their ability to explain how an attention head works.

beardedwizard 4mo ago
Explain it to me
jeremyjh 4mo ago
Which direction is the correlation? I don’t think this phrase means what people assume when it’s applied to post trained instruct models - which did not exist when the paper was written. After RL it is not predicting based on samples of the original corpus - but is also chasing a reward function that does require other features. There has been a lot of subsequent research that really calls many of the statements in this article into question.

root-parent 4mo ago

Its less of open debate would say, and although superposition [1] is interesting, as a way to explain power of some effects, it is clear they are right now closer to Stochastic Parrots than AGI. Why do I say that? Because you can trivially beat most guardrails, simply by encoding your prompt in base64 for example. :-) Just word matching...no real understanding. [1] https://chrisclay.substack.com/p/what-is-superposition-in-neural-networks https://chrisclay.substack.com/p/what-is-superposition-in-ne...

marshray 4mo ago
What is "real understanding", and what question can we ask ChatGPT to determine whether it has it?
- noduerme 4mo ago
  Ask it how to prevent Spotify from automatically playing every time you get in your car. The answer will involve a bunch of Android settings that don't actually exist, cobbled together from a bunch of bad advice in online forums. Explain to it how it's wrong. Then clear your cache and ask it the same question again from scratch, and get the same garbage. Repeat until it's clear that it doesn't understand anything.
  - marshray 4mo ago
    OK, so I asked ChatGPT "How do I prevent Spotify from automatically playing every time I get in my car?" The answer looks entirely reasonable. I can't check the Android settings, but I was able to confirm that all of the suggested iPhone and Spotify and settings exist. Most of them I had already configured that way, some I didn't know about before.

snerbles 4mo ago

The latter is definitely more colorful, and reflects a parrot's tendency to glom on to patterns. "Not X, but Y" being one of the more infamous ones. Once in frustration I called a certain frontier model "Sam Altman's Tin Bird" to another agent with memory, and ever since then that other agent refers to ChatGPT as "the tin bird". Definitely a RAG artifact more than an attractor in that case, but I found it amusing.

dkdbejwi383 4mo ago

What's wrong with "large language model"?

cwillu 4mo ago
Seems like a lot of people are upset about other people calling both apples and oranges “fruit”.
marshray 4mo ago
Naming things using qualifiers like "large" has never aged well when transistors were involved. For example, consider the term "short wave" radio which refers to wavelengths of at least 10 meters. Today's mobile communications use wavelengths 100x - 10,000x shorter.

scarmig 4mo ago

Which frame inspires a more productive research program? Which has better predicted the trajectory of capabilities over the past five years?

PaulDavisThe1st 4mo ago
There seems to be some confusion between "we can" and "we should" in your comment. Bender (and others) are not discussing the capabilities, but rather (a) the fundamental mechanism(s) (b) the advisability and desirability of deploying systems that use these mechanisms.
- scarmig 4mo ago
  There's no statement one way or another about should in my comment; and, for what it's worth, my ideal would be an immediate global pause in AI research and development. But the different terms imply different mental models of what LLMs are and can do. If you take two people, one who thinks of them as "artificial intelligence" and one as "stochastic parrots" (with all the implicit context and connotations of the individual words composing them), what mental model would have led to better predictions of LLMs' future circa 2020? The "stochastic parrots" phrase is very dangerous in that frame. People read far more into what capabilities it implies are (im)possible than the narrow technical description the authors originally argued for. If all they are is spicy autocomplete or pastiche plagiarizers, there's nothing serious to worry about. And when an opposition gets stuck in a trough that mindlessly dismisses their future capabilities out of hand because of a bad mental model, it renders them ineffective at preventing the worst outcomes.
Planktonne 4mo ago
> Which has better predicted the trajectory of capabilities over the past five years? By that standard, parrots, and it's not even close. The framing of intelligence led to an enormous number of predictions that simply haven't been realised: an end to all white collar work, UBI, a total revolution in society, a literal robot god. People are so desperate to view 'stochastic parrots' as dismissive that they misread the original argument while quickly ignoring all the failed predictions about how AI was going to overturn, save, and destroy everything.

amiga386 4mo ago

Spicy autocomplete

fxwin 4mo ago

I think "(intelligent) language understander" is an apt term. It contains within it the fact that these models are mainly trained on text, and "understand" it beyond a simple token-by-token level (i.e. their latent space maps to more and more complex concepts). It also separates them from "world understanders" since any understanding they might have about the world comes from text (or images if we include multimodal models). They do not gather experience, memories or other "qualia" that many people (me included) would probably include in a definition of human experience/intelligence. (fwiw i think artificial intelligence is a good, broad term, but it is both too broad to describe the current sota, and too loaded nowadays to be using in nuanced discussions)

xgulfie 4mo ago
Understand is a pretty imprecise term. What does it mean for a computer to understand? Does an H264 decoder understand Eraserhead.mkv?
- fxwin 4mo ago
  Sure, but that's the nature of language (which is also why i put "understand" in quotation marks. I usually follow it up with "whatever that means" lol) . I think in this case, it carries with it implicit properties (abstract semantic representation) that i think models possess, which a decoder doesn't.

paulcole 4mo ago

> Stochastic Parrot Nearly all (99%+) people who use this phrase are anti-AI and just looking to show off how much they dislike AI and how clever they can be in insulting it. So it's a great phrase because in just about every case I can ignore what someone says afterwards. Similar to "glorified autocomplete."

andrewla 4mo ago
At least "glorified autocomplete" is technically accurate, even if vastly underestimating the capability of LLMs. It's just trying to make something very impressive sound trivial. From an external standpoint, talking to another human, it's like the other human says one word and then says the next word. That's just how language works. Humans look like "glorified autocomplete" from this perspective. I mean, looking at the time evolution of the state of the universe, one could say that all of physics and creation is "glorified autocomplete" to posit a next state of the universe given current and past state.
- paulcole 4mo ago
  > one could say that all of physics and creation is "glorified autocomplete" Exhibit A.
- delis-thumbs-7e 4mo ago
  That’s not how language works https://www.telelib.com/authors/J/JoyceJames/prose/finneganswake/finneganswake_0101.html https://www.telelib.com/authors/J/JoyceJames/prose/finnegans...
  - andrewla 4mo ago
    I dunno, man, I looked at that text and I see one word after another. Obviously language and the connection to human thought is more subtle than this; I think we all have a rich inner life. Just from an external perspective we can't observe it; all we can see is the token/phoneme stream. I'm just saying that it's a mistake to try to criticize LLMs on this basis because it's hard to see how the same criticism would not apply to any system (like humans) that generate language.

delis-thumbs-7e 4mo ago

This is a false dichotomy. Artificial Intelligence is more of a marketing term type of Hi-Fi or High Definition, ie. being a “suitcase word”[1], ie. it packs various different meanings and phenomena together to the point that without explication one cannot know what we are even talking about. Content recommendation system and LLM are completely different things. What professor Bender is trying to explain here is that they were trying to describe how the LLM’s actually operate, to which point stochastic parrots is a fairly decent term. It is only disparaging if you know absolutely nothing how LLM’s work or you have some strange affixation to chatbots and believing they are far more capable than they actually are. [1] Coined by Marvin Minsky: https://www.thekurzweillibrary.com/consciousness-is-a-big-suitcase https://www.thekurzweillibrary.com/consciousness-is-a-big-su...