5 ms·

I paid a bit of attention to this paper and the phrase 'stochastic parrots' when it came out and i thought this was worth saying and doing at that time. their s

by dwa3592 3mo ago

I paid a bit of attention to this paper and the phrase 'stochastic parrots' when it came out and i thought this was worth saying and doing at that time. their suggestions about financial and environmental costs are worth studying, their concern about carefully evaluating datasets to feed to the model rather than feeding the entire internet is fully justified. so - to everyone saying this was a bad paper; if you have actually read the paper then please list a few criticisms. all i have seen is "oh this wasn't that good of a paper" or "can't believe how bad this paper was".

CamperBob2 3mo ago
Those costs have to be compared to the way things are currently done without AI. They never are. Ever.
- supern0va 3mo ago
  >They never are. Ever. And even when they are: they sure seem to bet against Moore's Law or just the general tendency for things to get better/efficient over time. It's frankly remarkable how capable the models have become that we can run locally now on a decent laptop. The same thing happened with image generation. I've had arguments with people that image generators are killing the environment, but I can do it in 20-30 seconds on my GPU. No one bats an eyelash when I play 20-30 minutes or even hours of a video game on my GPU, but the images are burning down the planet. It's slightly maddening.
azakai 3mo ago
My main criticism of the paper is that it says LLMs work "haphazardly", using probabilistic information. That is a hypothesis, but it is stated as a known fact, a fundamental limitation. It is true that LLMs often behave haphazardly, and do rely on statistics. But plenty of research has shown them behaving in methodical ways too. There are findings going both ways! Granted, many of the strongest contradictory results appeared after the Stochastic Parrots paper, so it isn't like they were ignoring the literature at the time. But they did make a very strong claim, and in the half-decade since, a lot of evidence has come out against it.
- dwa3592 3mo ago
  not sure your criticism makes sense though - they did this pre chatgpt. they are talking about the language models of that time. they did not make predictions about the future.
  - mediaman 3mo ago
    They did use RLHF at the time, at which point it is not a pure probabilistic representation of the training corpora. Bizarrely, RLHF never came up in the paper.
  - azakai 3mo ago
    They made a claim about language models in general, not just ones that had been released so far. The point of the paper, in fact, is that language models are getting "too big", and another approach is needed to make progress, so they were certainly predicting things about later models. With that said, they talked about "pure" language models, so it is fair to say that they didn't talk about, say, LLMs that are multimodal or that have tool use, which are advances that happened after their paper.
- zmgsabst 3mo ago
  Statistical operation doesn’t preclude logical processing. We’ve know that since 1943 when McCulloch-Pitts came up with the first “artificial neuron” definition. And since LLMs are a descendant technology — our assumption should be they’re reasoning in some internal learned logic. This is what the evidence supports — eg, the “stochastic parrot” crowd never can explain transfer learning. Whereas for the internal reasoning crowd that is easy: removing your top level judgments from a theory still leaves you with useful terms for describing a new theory — eg, removing your judgments about “which animal is this?” but preserving the underlying structure for representing an image in your new judgments, “is this cancer?” There’s 80 years of reason to think DNNs reason and zero support other than “sTaTs R mAgIc!” to support the stochastic parrot interpretation. Ignorance isn’t argument.
matusp 3mo ago
It is a good blog, not a good paper.
daishi55 3mo ago
My criticism centers on the part of the paper they chose for their title, the “stochastic parrot” metaphor. And my criticism is that if you observe Claude code with opus 4.8 working through an entirely novel problem that nobody has ever worked on before and which certainly wasn’t in its training data, the choice to even metaphorically call them stochastic parrots turned out to be egregiously wrong. And secondarily, and maybe only partially the authors’ fault, is the enormous tidal wave of morons that this paper minted who plague us with their misunderstandings to this day.
- dwa3592 3mo ago
  oh boy! the lack of critical thinking here is staggering. >>And my criticism is that if you observe Claude code with opus 4.8 working through an entirely novel problem that nobody has ever worked on before and which certainly wasn’t in its training data, the choice to even metaphorically call them stochastic parrots turned out to be egregiously wrong. First of all, for your own benefit - Claude code stopped showing the real reasoning. It only shows a summarized version of it now. So don't ever ask someone to observe the model. Second, do you know when this paper came out? do you know when opus 4.8 came out??? how do you know what is novel? did opus 4.8 tell you it was novel? how do you know no one has worked on it before?
  - daishi55 3mo ago
    I strongly suspect my critical thinking skills are better than yours. For example, note my use of the phrase “turned out” and then consider whether your main point about the release date of plus 4.8 vs the paper makes any sense at all? > how do you know what is novel? Because I work on novel ASICs that were created by my company and have never been used or seen outside of my company?
ToValueFunfetti 3mo ago
The contention that there is no grounding because the training data is linguistic and thus can only reference a world model is disproven in "This sentence has five words"- there's real, grounded information about what "five" means within that sentence. While that's a trivial counterexample, I don't know that it's an obvious one (I didn't come up with it myself). It's not a criticism of the paper itself, but multimodal models came shortly after and provide grounding that is more of the sort the paper is getting at, and it didn't seem like anybody updated on that at all. If multimodal models were still stochastic parrots by the original argument, humans would have to be as well; we don't have any way to ground anything beneath sense data and evolution can't have programmed some innate grounding into us because it didn't either. But (and maybe this is my own misperception) nobody threw in the towel at that point. I confess I never read the original paper until now, opting to absorb by osmosis instead, and I was quite surprised that they don't really make a deeper case than that. After just a few paragraphs about how they can't be grounded because humans don't express their thoughts directly, it lurches into a page about how they can be biased by training. And they certainly can be, but that has little to say about their stochastic nature- humans are biased as a rule with no exception. (For the record, I only read the Stochastic Parrots section before this reply.) It's not really a bad paper, but I don't see why it ever carried the esteem it did. Hating on it is like hating on Taylor Swift- she's fine, yes, but for her level of success, one is inclined to question every dumb lyric where others get a pass. (Apologies to Swift fans, substitute a successful artist you don't care for here.)
- dwa3592 3mo ago
  >>The contention that there is no grounding because the training data is linguistic and thus can only reference a world model is disproven in "This sentence has five words"- there's real, grounded information about what "five" means within that sentence. did you think this through? imagine the sentence was "This sentence has four words", now extrapolate that to all the shit that can exist in a dataset and train a model on that dataset - do you know what will happen? - go ahead and think it through.
  - ToValueFunfetti 3mo ago
mike_hearn 3mo ago
The authors were wrong about their core thesis and are now lying about it. That's the only criticism needed. They said, quote: > LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form ... which is presented as unarguable fact, yet is untrue. It was obviously wrong at the time it was written and it's been proven wrong in many ways since. Worse is that they're still at it. In the article she's saying: > Q: What are the most common misconceptions about the “stochastic parrots” metaphor? Bender: I think one of the biggest ones is, “Bender says AI is a stochastic parrot.” Her name is on a paper titled "On the danger of stochastic parrots". It has a section titled "Stochastic parrots" and in section 6.1 it says: > An LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot. She did say that, as clear as day. Now she's trying to rewrite history. Ugly behavior.
- deleted 3mo ago
  [deleted]
- nyeah 3mo ago
  No, they were correct. In fact an LLM stitches together stuff it observed in its training data. That scales up way better than a lot of us expected, but it's still correct. If you train it on lots of working code, then it's useful for coding. If you trained it primarily on non-working code it would produce nonsense.
  - mike_hearn 3mo ago
    It's not correct. Please read some more research papers, this isn't what people working in AI believe at all. You can prove with experiments that different human languages get translated to the same abstract conceptual space in the middle layers, for example. It's why interpretability is so difficult. The claim is odd in another way: you can train a person on non-working code and they'll produce nonsense. That doesn't mean people are just stitching together words they've previously seen.

by dwa3592 3mo ago

CamperBob2 3mo ago

Those costs have to be compared to the way things are currently done without AI. They never are. Ever.

supern0va 3mo ago
>They never are. Ever. And even when they are: they sure seem to bet against Moore's Law or just the general tendency for things to get better/efficient over time. It's frankly remarkable how capable the models have become that we can run locally now on a decent laptop. The same thing happened with image generation. I've had arguments with people that image generators are killing the environment, but I can do it in 20-30 seconds on my GPU. No one bats an eyelash when I play 20-30 minutes or even hours of a video game on my GPU, but the images are burning down the planet. It's slightly maddening.

azakai 3mo ago

My main criticism of the paper is that it says LLMs work "haphazardly", using probabilistic information. That is a hypothesis, but it is stated as a known fact, a fundamental limitation. It is true that LLMs often behave haphazardly, and do rely on statistics. But plenty of research has shown them behaving in methodical ways too. There are findings going both ways! Granted, many of the strongest contradictory results appeared after the Stochastic Parrots paper, so it isn't like they were ignoring the literature at the time. But they did make a very strong claim, and in the half-decade since, a lot of evidence has come out against it.

dwa3592 3mo ago
not sure your criticism makes sense though - they did this pre chatgpt. they are talking about the language models of that time. they did not make predictions about the future.
- mediaman 3mo ago
  They did use RLHF at the time, at which point it is not a pure probabilistic representation of the training corpora. Bizarrely, RLHF never came up in the paper.
- azakai 3mo ago
  They made a claim about language models in general, not just ones that had been released so far. The point of the paper, in fact, is that language models are getting "too big", and another approach is needed to make progress, so they were certainly predicting things about later models. With that said, they talked about "pure" language models, so it is fair to say that they didn't talk about, say, LLMs that are multimodal or that have tool use, which are advances that happened after their paper.
zmgsabst 3mo ago
Statistical operation doesn’t preclude logical processing. We’ve know that since 1943 when McCulloch-Pitts came up with the first “artificial neuron” definition. And since LLMs are a descendant technology — our assumption should be they’re reasoning in some internal learned logic. This is what the evidence supports — eg, the “stochastic parrot” crowd never can explain transfer learning. Whereas for the internal reasoning crowd that is easy: removing your top level judgments from a theory still leaves you with useful terms for describing a new theory — eg, removing your judgments about “which animal is this?” but preserving the underlying structure for representing an image in your new judgments, “is this cancer?” There’s 80 years of reason to think DNNs reason and zero support other than “sTaTs R mAgIc!” to support the stochastic parrot interpretation. Ignorance isn’t argument.

matusp 3mo ago

It is a good blog, not a good paper.

daishi55 3mo ago

My criticism centers on the part of the paper they chose for their title, the “stochastic parrot” metaphor. And my criticism is that if you observe Claude code with opus 4.8 working through an entirely novel problem that nobody has ever worked on before and which certainly wasn’t in its training data, the choice to even metaphorically call them stochastic parrots turned out to be egregiously wrong. And secondarily, and maybe only partially the authors’ fault, is the enormous tidal wave of morons that this paper minted who plague us with their misunderstandings to this day.

dwa3592 3mo ago
oh boy! the lack of critical thinking here is staggering. >>And my criticism is that if you observe Claude code with opus 4.8 working through an entirely novel problem that nobody has ever worked on before and which certainly wasn’t in its training data, the choice to even metaphorically call them stochastic parrots turned out to be egregiously wrong. First of all, for your own benefit - Claude code stopped showing the real reasoning. It only shows a summarized version of it now. So don't ever ask someone to observe the model. Second, do you know when this paper came out? do you know when opus 4.8 came out??? how do you know what is novel? did opus 4.8 tell you it was novel? how do you know no one has worked on it before?
- daishi55 3mo ago
  I strongly suspect my critical thinking skills are better than yours. For example, note my use of the phrase “turned out” and then consider whether your main point about the release date of plus 4.8 vs the paper makes any sense at all? > how do you know what is novel? Because I work on novel ASICs that were created by my company and have never been used or seen outside of my company?

ToValueFunfetti 3mo ago

The contention that there is no grounding because the training data is linguistic and thus can only reference a world model is disproven in "This sentence has five words"- there's real, grounded information about what "five" means within that sentence. While that's a trivial counterexample, I don't know that it's an obvious one (I didn't come up with it myself). It's not a criticism of the paper itself, but multimodal models came shortly after and provide grounding that is more of the sort the paper is getting at, and it didn't seem like anybody updated on that at all. If multimodal models were still stochastic parrots by the original argument, humans would have to be as well; we don't have any way to ground anything beneath sense data and evolution can't have programmed some innate grounding into us because it didn't either. But (and maybe this is my own misperception) nobody threw in the towel at that point. I confess I never read the original paper until now, opting to absorb by osmosis instead, and I was quite surprised that they don't really make a deeper case than that. After just a few paragraphs about how they can't be grounded because humans don't express their thoughts directly, it lurches into a page about how they can be biased by training. And they certainly can be, but that has little to say about their stochastic nature- humans are biased as a rule with no exception. (For the record, I only read the Stochastic Parrots section before this reply.) It's not really a bad paper, but I don't see why it ever carried the esteem it did. Hating on it is like hating on Taylor Swift- she's fine, yes, but for her level of success, one is inclined to question every dumb lyric where others get a pass. (Apologies to Swift fans, substitute a successful artist you don't care for here.)

dwa3592 3mo ago
>>The contention that there is no grounding because the training data is linguistic and thus can only reference a world model is disproven in "This sentence has five words"- there's real, grounded information about what "five" means within that sentence. did you think this through? imagine the sentence was "This sentence has four words", now extrapolate that to all the shit that can exist in a dataset and train a model on that dataset - do you know what will happen? - go ahead and think it through.
- ToValueFunfetti 3mo ago

mike_hearn 3mo ago

The authors were wrong about their core thesis and are now lying about it. That's the only criticism needed. They said, quote: > LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form ... which is presented as unarguable fact, yet is untrue. It was obviously wrong at the time it was written and it's been proven wrong in many ways since. Worse is that they're still at it. In the article she's saying: > Q: What are the most common misconceptions about the “stochastic parrots” metaphor? Bender: I think one of the biggest ones is, “Bender says AI is a stochastic parrot.” Her name is on a paper titled "On the danger of stochastic parrots". It has a section titled "Stochastic parrots" and in section 6.1 it says: > An LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot. She did say that, as clear as day. Now she's trying to rewrite history. Ugly behavior.

deleted 3mo ago
[deleted]
nyeah 3mo ago
No, they were correct. In fact an LLM stitches together stuff it observed in its training data. That scales up way better than a lot of us expected, but it's still correct. If you train it on lots of working code, then it's useful for coding. If you trained it primarily on non-working code it would produce nonsense.
- mike_hearn 3mo ago
  It's not correct. Please read some more research papers, this isn't what people working in AI believe at all. You can prove with experiments that different human languages get translated to the same abstract conceptual space in the middle layers, for example. It's why interpretability is so difficult. The claim is odd in another way: you can train a person on non-working code and they'll produce nonsense. That doesn't mean people are just stitching together words they've previously seen.