8 ms·

I tend to disregard articles that default to the "Stochastic Parrot" argument. These tools are useful now, I don't personally care about achieving actual intell

I tend to disregard articles that default to the "Stochastic Parrot" argument. These tools are useful now, I don't personally care about achieving actual intelligence. I want additional utility for myself and other humans, which these provide now, at scale. By a lot of measures many humans perform at just about the same level, including confidently making up bullshit. This post reads like one of the "Goodbye X online video game" posts. I'll cut them some slack because this is their blog they're venting on and was likely posted here by someone else and not themselves doing some attention seeking, but meh.

LASR 4y ago
It’s pretty ironic that the argument that LLMs are stochastic parrots actually sounds like the actual parrot. I think we’re now way past that now with LLMs now quickly taking on the role of a general reasoning engine.
- shrimpx 4y ago
  LLMs suck at deduction and factual reasoning. How are they reasoning engines?
- cowl 4y ago
  > now quickly taking on the role of a general reasoning engine And this right here is why it's important to emphasize the "stochastic parrot" fact. Because people think this is true and are making decisions based on this misunderstanding.
  - FeepingCreature 4y ago
    Or maybe they just disagree with you?
    - cowl 4y ago
      Who? See: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 A 2021 research paper warning (among other things) precisly about this confusion. > The ersatz fluency and coherence of LMs raises several risks, precisely because humans are prepared to interpret strings belonging to languages they speak as meaningful and corresponding to the communicative intent of some individual or group of individuals who have accountability for what is said Is there any Researcher who maintains that LLM models contain Reasoning and intent? Those who are working on this models are not confused, they know what they are, the Public is confused.
      - pritambaral 4y ago
        > Is there any Researcher who maintains that LLM models contain Reasoning and intent? Researchers with jobs that depend on promulgating this belief. E.g., "Open"AI employees.
      - FeepingCreature 4y ago
        How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that paper, even if I discard the dross (ie. everything except one section on page 7), seems to actually make an argument against X of any strength beyond "it is wrong because it is wrong". My point is this: I disagree with you. This is not because I have "misunderstood" something; it is because I understand the stochastic-parrot argument and think it is erroneous. And the more you talk about "the risk that people will come to falsely believe" rather than actual arguments, the less convincing you sound. This paternalistic tendency is a curse on science and debate in general.
        cowl 4y ago
        Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.
        FeepingCreature 4y ago
        Bad argument is bad argument, doesn't matter who says it.
        usrbinbash 4y ago
        > it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.
        FeepingCreature 4y ago
- layer8 4y ago
  Logical reasoning is the one thing LLMs are really bad at.
- gat1 4y ago
  LLMs cannot be considered as general reasoning engines because they do not make any kind of reasoning though.
- mxmlnkn 4y ago
  Since ChatGPT I've become much more aware of my own thoughts and written text. I'm now often wondering whether I'm just regurgitating the most frequently used next word or phrase or whether it could actually be described as original. Especially, for things like reacting with short answers to chat messages, I am confident that these are only reactionary answers without alternatives, which could have come from ChatGPT trained on my chat log. I feel like knowing and seeing how ChatGPT works can elevate our own thinking process. Or maybe it only is similar to awareness meditation.
  - tigerlily 4y ago
    Now that is a thought that could not possibly have come from GPT. Thanks, this has given me something to think about.
  - a_bonobo 4y ago
    In this vein, ChatGPT is a nice way to start thinking about any topic; ask it about its opinion on anything and it will give you the most run-of-the-mill, middle-of-the-road text that is possible, standing for nothing. If you find yourself agreeing with ChatGPT it may be time to reconsider your own thinking!
  - chomp 4y ago
    LLMs may be an approximation of our knowledge, but understand that there’s more to reasoning than a language model. GPT understands how tokens relate to each other, but humans understand how objects, places, and abstract concepts relate to each other as well. We have a little further to go for AGI. I do believe that GPT4 is a really good approximation of our language though, and feel similarly to you when I respond off the cuff.
- usrbinbash 4y ago
  > I think we’re now way past that now with LLMs now quickly taking on the role of a general reasoning engine. No we're not, and no they are not. An LLM doesn't reason, period. It mimics reasoning ability by stochastically chosing a sequence of tokens. Alot of the time these make sense. At other times, they don't make any sense. I recently asked an LLM: "Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first?" It answered correctly that Mike leaves first. Then I asked: "If the elevator started at the 10th floor, who would have left first?" And the answer was that Mike still leaves first, because he leaves at the 2nd floor, and that's the first floor the elevator reaches. Another time I asked an LLM how many footballs fit in a coffe-mug, and the conversation reached a point where the AI tried to convince me, that coffe-mugs are only slightly smaller than the trunk of a car. Yes, they can also produce the correct answers to both these questions, but the fact that they can also spew such complete illogical nonsense shows that they are not "reasoning" about things. They complete sequences, that's it, period, that's literally the only thing a language model can do. Their apparent emergent abilities look like reasoning, in the same way as Jen from "The IT crowd" can sound like shes speaking Italian, when in fact she has no idea what she is even saying.
  - ianbutler 4y ago
    GPT-4 reasons about a lot of gotcha logic puzzles correctly any pre GPT-4 opinions should be reconsidered, that is effectively two different epochs in the history of AI effectiveness and reasoning. > Me: Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first? > GPT-4: Mike left the elevator first, as he got off at the 2nd floor, while Jenny left at the 9th floor. > Me: If the elevator started at the 10th floor, who would have left first? > GPT-4: If the elevator started at the 10th floor and went downward, then Jenny would have left first, as she got off at the 9th floor, while Mike left at the 2nd floor. > Me: How many footballs fit in a coffe-mug? > GPT-4: A standard football (soccer ball) has a diameter of around 22 centimeters (8.65 inches), while a coffee mug is typically much smaller, with a diameter of around 8-10 centimeters (3-4 inches). Therefore, it is not possible to fit a standard football inside a coffee mug. If you were to use a mini football or a much larger mug, the number of footballs that could fit would depend on the specific sizes of the footballs and the mug. It easily answered all of your questions and produces explanations I would expect most reasonable people to make.
- qayxc 4y ago
  I like that idea a lot. "General reasoning engine" sounds much more sensible than "AGI", especially given that the "I" in AGI is so poorly defined.
cowl 4y ago
Being useful and a stochastic parrot are not mutually exclusive. And I in fact think the opposite. it's Necessary to remind people what it really is, especially in this phase of "Enthusiasm" because I see to many people attributing some meaning or some hidden Insight and especially some innate infallibility to AI nowdays, maybe confused by the name AI.
- ianbutler 4y ago
  Right, but most arguments, including the one here, go something like "AI is a Stochastic Parrot so it's a lie and now I think it's bad and we shouldn't do it." Which is a pretty dumb position imo. Not that I personally think these newer LLMs are a stochastic parrot, or at least not to the degree proponents of the Stochastic Parrot argument would have you believe.
  - cowl 4y ago
    The argument of Stochastic Parrot is not that we shouldn't do it. It's just "Do not attribute any meaning to it". The one in this article is in the same vane that author thought the AI was learning to program when in reality it was just repeating the most statistically probable combination of the code it had seen. That is, "correctness" is not part of "considerations" that the model does. If the majority of the code that it has scanned contains a particular form of logic bug, it will suggest the same logic bug. The trap is in the fact that the AI will write perfect syntax because that is it's bread and butter and people seeing this perfect syntax attribute also perfect logic to it. As long as people are aware of this kind of problem, LLMs are a very useful tool that will save a lot of time. But if applied blindly "Because AI knows best" it will create more problems down the road.
    - ianbutler 4y ago
      That's not what I said the Stochastic Parrot is arguing, I said people use the Stochastic Parrot argument to justify not doing it.
bakugo 4y ago
> These tools are useful now Guns are also useful tools because you can take them into a store and get things for free as a result. But that doesn't make okay to do.
- Eisenstein 4y ago
  Robbing people does not benefit anyone but yourself. This is a terrible analogy.
  - bakugo 4y ago
    I can take the things I stole and sell them to other people, which is exactly what Github is doing here.
    - deleted 4y ago
      [deleted]

LASR 4y ago

It’s pretty ironic that the argument that LLMs are stochastic parrots actually sounds like the actual parrot. I think we’re now way past that now with LLMs now quickly taking on the role of a general reasoning engine.

shrimpx 4y ago
LLMs suck at deduction and factual reasoning. How are they reasoning engines?
cowl 4y ago
> now quickly taking on the role of a general reasoning engine And this right here is why it's important to emphasize the "stochastic parrot" fact. Because people think this is true and are making decisions based on this misunderstanding.
- FeepingCreature 4y ago
  Or maybe they just disagree with you?
  - cowl 4y ago
    Who? See: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 A 2021 research paper warning (among other things) precisly about this confusion. > The ersatz fluency and coherence of LMs raises several risks, precisely because humans are prepared to interpret strings belonging to languages they speak as meaningful and corresponding to the communicative intent of some individual or group of individuals who have accountability for what is said Is there any Researcher who maintains that LLM models contain Reasoning and intent? Those who are working on this models are not confused, they know what they are, the Public is confused.
    - pritambaral 4y ago
      > Is there any Researcher who maintains that LLM models contain Reasoning and intent? Researchers with jobs that depend on promulgating this belief. E.g., "Open"AI employees.
    - FeepingCreature 4y ago
      How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that paper, even if I discard the dross (ie. everything except one section on page 7), seems to actually make an argument against X of any strength beyond "it is wrong because it is wrong". My point is this: I disagree with you. This is not because I have "misunderstood" something; it is because I understand the stochastic-parrot argument and think it is erroneous. And the more you talk about "the risk that people will come to falsely believe" rather than actual arguments, the less convincing you sound. This paternalistic tendency is a curse on science and debate in general.
      - cowl 4y ago
        Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.
        FeepingCreature 4y ago
        Bad argument is bad argument, doesn't matter who says it.
      - usrbinbash 4y ago
        > it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.
        FeepingCreature 4y ago
layer8 4y ago
Logical reasoning is the one thing LLMs are really bad at.
gat1 4y ago
LLMs cannot be considered as general reasoning engines because they do not make any kind of reasoning though.
mxmlnkn 4y ago
Since ChatGPT I've become much more aware of my own thoughts and written text. I'm now often wondering whether I'm just regurgitating the most frequently used next word or phrase or whether it could actually be described as original. Especially, for things like reacting with short answers to chat messages, I am confident that these are only reactionary answers without alternatives, which could have come from ChatGPT trained on my chat log. I feel like knowing and seeing how ChatGPT works can elevate our own thinking process. Or maybe it only is similar to awareness meditation.
- tigerlily 4y ago
  Now that is a thought that could not possibly have come from GPT. Thanks, this has given me something to think about.
- a_bonobo 4y ago
  In this vein, ChatGPT is a nice way to start thinking about any topic; ask it about its opinion on anything and it will give you the most run-of-the-mill, middle-of-the-road text that is possible, standing for nothing. If you find yourself agreeing with ChatGPT it may be time to reconsider your own thinking!
- chomp 4y ago
  LLMs may be an approximation of our knowledge, but understand that there’s more to reasoning than a language model. GPT understands how tokens relate to each other, but humans understand how objects, places, and abstract concepts relate to each other as well. We have a little further to go for AGI. I do believe that GPT4 is a really good approximation of our language though, and feel similarly to you when I respond off the cuff.
usrbinbash 4y ago
> I think we’re now way past that now with LLMs now quickly taking on the role of a general reasoning engine. No we're not, and no they are not. An LLM doesn't reason, period. It mimics reasoning ability by stochastically chosing a sequence of tokens. Alot of the time these make sense. At other times, they don't make any sense. I recently asked an LLM: "Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first?" It answered correctly that Mike leaves first. Then I asked: "If the elevator started at the 10th floor, who would have left first?" And the answer was that Mike still leaves first, because he leaves at the 2nd floor, and that's the first floor the elevator reaches. Another time I asked an LLM how many footballs fit in a coffe-mug, and the conversation reached a point where the AI tried to convince me, that coffe-mugs are only slightly smaller than the trunk of a car. Yes, they can also produce the correct answers to both these questions, but the fact that they can also spew such complete illogical nonsense shows that they are not "reasoning" about things. They complete sequences, that's it, period, that's literally the only thing a language model can do. Their apparent emergent abilities look like reasoning, in the same way as Jen from "The IT crowd" can sound like shes speaking Italian, when in fact she has no idea what she is even saying.
- ianbutler 4y ago
  GPT-4 reasons about a lot of gotcha logic puzzles correctly any pre GPT-4 opinions should be reconsidered, that is effectively two different epochs in the history of AI effectiveness and reasoning. > Me: Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first? > GPT-4: Mike left the elevator first, as he got off at the 2nd floor, while Jenny left at the 9th floor. > Me: If the elevator started at the 10th floor, who would have left first? > GPT-4: If the elevator started at the 10th floor and went downward, then Jenny would have left first, as she got off at the 9th floor, while Mike left at the 2nd floor. > Me: How many footballs fit in a coffe-mug? > GPT-4: A standard football (soccer ball) has a diameter of around 22 centimeters (8.65 inches), while a coffee mug is typically much smaller, with a diameter of around 8-10 centimeters (3-4 inches). Therefore, it is not possible to fit a standard football inside a coffee mug. If you were to use a mini football or a much larger mug, the number of footballs that could fit would depend on the specific sizes of the footballs and the mug. It easily answered all of your questions and produces explanations I would expect most reasonable people to make.
qayxc 4y ago
I like that idea a lot. "General reasoning engine" sounds much more sensible than "AGI", especially given that the "I" in AGI is so poorly defined.

cowl 4y ago

Being useful and a stochastic parrot are not mutually exclusive. And I in fact think the opposite. it's Necessary to remind people what it really is, especially in this phase of "Enthusiasm" because I see to many people attributing some meaning or some hidden Insight and especially some innate infallibility to AI nowdays, maybe confused by the name AI.

ianbutler 4y ago
Right, but most arguments, including the one here, go something like "AI is a Stochastic Parrot so it's a lie and now I think it's bad and we shouldn't do it." Which is a pretty dumb position imo. Not that I personally think these newer LLMs are a stochastic parrot, or at least not to the degree proponents of the Stochastic Parrot argument would have you believe.
- cowl 4y ago
  The argument of Stochastic Parrot is not that we shouldn't do it. It's just "Do not attribute any meaning to it". The one in this article is in the same vane that author thought the AI was learning to program when in reality it was just repeating the most statistically probable combination of the code it had seen. That is, "correctness" is not part of "considerations" that the model does. If the majority of the code that it has scanned contains a particular form of logic bug, it will suggest the same logic bug. The trap is in the fact that the AI will write perfect syntax because that is it's bread and butter and people seeing this perfect syntax attribute also perfect logic to it. As long as people are aware of this kind of problem, LLMs are a very useful tool that will save a lot of time. But if applied blindly "Because AI knows best" it will create more problems down the road.
  - ianbutler 4y ago
    That's not what I said the Stochastic Parrot is arguing, I said people use the Stochastic Parrot argument to justify not doing it.

bakugo 4y ago

> These tools are useful now Guns are also useful tools because you can take them into a store and get things for free as a result. But that doesn't make okay to do.

Eisenstein 4y ago
Robbing people does not benefit anyone but yourself. This is a terrible analogy.
- bakugo 4y ago
  I can take the things I stole and sell them to other people, which is exactly what Github is doing here.
  - deleted 4y ago
    [deleted]