8 ms·

It’s pretty ironic that the argument that LLMs are stochastic parrots actually sounds like the actual parrot. I think we’re now way past that now with LLMs now

by LASR 4y ago

It’s pretty ironic that the argument that LLMs are stochastic parrots actually sounds like the actual parrot. I think we’re now way past that now with LLMs now quickly taking on the role of a general reasoning engine.

shrimpx 4y ago
LLMs suck at deduction and factual reasoning. How are they reasoning engines?
cowl 4y ago
> now quickly taking on the role of a general reasoning engine And this right here is why it's important to emphasize the "stochastic parrot" fact. Because people think this is true and are making decisions based on this misunderstanding.
- FeepingCreature 4y ago
  Or maybe they just disagree with you?
  - cowl 4y ago
    Who? See: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 A 2021 research paper warning (among other things) precisly about this confusion. > The ersatz fluency and coherence of LMs raises several risks, precisely because humans are prepared to interpret strings belonging to languages they speak as meaningful and corresponding to the communicative intent of some individual or group of individuals who have accountability for what is said Is there any Researcher who maintains that LLM models contain Reasoning and intent? Those who are working on this models are not confused, they know what they are, the Public is confused.
    - pritambaral 4y ago
      > Is there any Researcher who maintains that LLM models contain Reasoning and intent? Researchers with jobs that depend on promulgating this belief. E.g., "Open"AI employees.
    - FeepingCreature 4y ago
      How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that paper, even if I discard the dross (ie. everything except one section on page 7), seems to actually make an argument against X of any strength beyond "it is wrong because it is wrong". My point is this: I disagree with you. This is not because I have "misunderstood" something; it is because I understand the stochastic-parrot argument and think it is erroneous. And the more you talk about "the risk that people will come to falsely believe" rather than actual arguments, the less convincing you sound. This paternalistic tendency is a curse on science and debate in general.
      - cowl 4y ago
        Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.
        FeepingCreature 4y ago
        Bad argument is bad argument, doesn't matter who says it.
      - usrbinbash 4y ago
        > it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.
        FeepingCreature 4y ago
        There's a class of statements that can be either interpreted precisely, at which point the claim they make is clearly true but trivial, or interpreted expansively, at which point the claim is significant but no longer clearly true. This is one of those: yes, technically LLMs are token predictors, but technically any nondeterministic Turing machine is a token predictor. The human brain could be viewed as a token predictor [1]. The interesting question is how it comes up with its predictions, and on this the phrase offers no insight at all. [1] https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
    - a_bonobo 4y ago
      >Is there any Researcher who maintains that LLM models contain Reasoning and intent? That's the funny thing about the (in)famous OpenAI letter; the first sentence kind of does this: >AI systems with human-competitive intelligence can pose profound risks to society and humanity, as shown by extensive research[1] 'human-competitive intelligence' sounds like reasoning to me. What's even funnier is that [1] is the stochastic parrot paper, which argues exactly the opposite!
      - usrbinbash 4y ago
        > 'human-competitive intelligence' sounds like reasoning to me. Yes, and when AIs reach that level of intelligence, we can revisit the question. However, as long as LLMs will confidently try to explain to me why several footballs fit in an average coffee mug, I'd say we are still quite some way away from "human-competitive intelligence".
        a_bonobo 4y ago
        Yes, I fully agree with you! there's no reasoning or intelligence in modern LLMs, but the OpenAI open letter and recent comms strongly implies there is
layer8 4y ago
Logical reasoning is the one thing LLMs are really bad at.
gat1 4y ago
LLMs cannot be considered as general reasoning engines because they do not make any kind of reasoning though.
mxmlnkn 4y ago
Since ChatGPT I've become much more aware of my own thoughts and written text. I'm now often wondering whether I'm just regurgitating the most frequently used next word or phrase or whether it could actually be described as original. Especially, for things like reacting with short answers to chat messages, I am confident that these are only reactionary answers without alternatives, which could have come from ChatGPT trained on my chat log. I feel like knowing and seeing how ChatGPT works can elevate our own thinking process. Or maybe it only is similar to awareness meditation.
- tigerlily 4y ago
  Now that is a thought that could not possibly have come from GPT. Thanks, this has given me something to think about.
- a_bonobo 4y ago
  In this vein, ChatGPT is a nice way to start thinking about any topic; ask it about its opinion on anything and it will give you the most run-of-the-mill, middle-of-the-road text that is possible, standing for nothing. If you find yourself agreeing with ChatGPT it may be time to reconsider your own thinking!
- chomp 4y ago
  LLMs may be an approximation of our knowledge, but understand that there’s more to reasoning than a language model. GPT understands how tokens relate to each other, but humans understand how objects, places, and abstract concepts relate to each other as well. We have a little further to go for AGI. I do believe that GPT4 is a really good approximation of our language though, and feel similarly to you when I respond off the cuff.
usrbinbash 4y ago
> I think we’re now way past that now with LLMs now quickly taking on the role of a general reasoning engine. No we're not, and no they are not. An LLM doesn't reason, period. It mimics reasoning ability by stochastically chosing a sequence of tokens. Alot of the time these make sense. At other times, they don't make any sense. I recently asked an LLM: "Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first?" It answered correctly that Mike leaves first. Then I asked: "If the elevator started at the 10th floor, who would have left first?" And the answer was that Mike still leaves first, because he leaves at the 2nd floor, and that's the first floor the elevator reaches. Another time I asked an LLM how many footballs fit in a coffe-mug, and the conversation reached a point where the AI tried to convince me, that coffe-mugs are only slightly smaller than the trunk of a car. Yes, they can also produce the correct answers to both these questions, but the fact that they can also spew such complete illogical nonsense shows that they are not "reasoning" about things. They complete sequences, that's it, period, that's literally the only thing a language model can do. Their apparent emergent abilities look like reasoning, in the same way as Jen from "The IT crowd" can sound like shes speaking Italian, when in fact she has no idea what she is even saying.
- ianbutler 4y ago
  GPT-4 reasons about a lot of gotcha logic puzzles correctly any pre GPT-4 opinions should be reconsidered, that is effectively two different epochs in the history of AI effectiveness and reasoning. > Me: Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first? > GPT-4: Mike left the elevator first, as he got off at the 2nd floor, while Jenny left at the 9th floor. > Me: If the elevator started at the 10th floor, who would have left first? > GPT-4: If the elevator started at the 10th floor and went downward, then Jenny would have left first, as she got off at the 9th floor, while Mike left at the 2nd floor. > Me: How many footballs fit in a coffe-mug? > GPT-4: A standard football (soccer ball) has a diameter of around 22 centimeters (8.65 inches), while a coffee mug is typically much smaller, with a diameter of around 8-10 centimeters (3-4 inches). Therefore, it is not possible to fit a standard football inside a coffee mug. If you were to use a mini football or a much larger mug, the number of footballs that could fit would depend on the specific sizes of the footballs and the mug. It easily answered all of your questions and produces explanations I would expect most reasonable people to make.
qayxc 4y ago
I like that idea a lot. "General reasoning engine" sounds much more sensible than "AGI", especially given that the "I" in AGI is so poorly defined.

by LASR 4y ago

shrimpx 4y ago

LLMs suck at deduction and factual reasoning. How are they reasoning engines?

cowl 4y ago

> now quickly taking on the role of a general reasoning engine And this right here is why it's important to emphasize the "stochastic parrot" fact. Because people think this is true and are making decisions based on this misunderstanding.

FeepingCreature 4y ago
Or maybe they just disagree with you?
- cowl 4y ago
  Who? See: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 A 2021 research paper warning (among other things) precisly about this confusion. > The ersatz fluency and coherence of LMs raises several risks, precisely because humans are prepared to interpret strings belonging to languages they speak as meaningful and corresponding to the communicative intent of some individual or group of individuals who have accountability for what is said Is there any Researcher who maintains that LLM models contain Reasoning and intent? Those who are working on this models are not confused, they know what they are, the Public is confused.
  - pritambaral 4y ago
    > Is there any Researcher who maintains that LLM models contain Reasoning and intent? Researchers with jobs that depend on promulgating this belief. E.g., "Open"AI employees.
  - FeepingCreature 4y ago
    How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that paper, even if I discard the dross (ie. everything except one section on page 7), seems to actually make an argument against X of any strength beyond "it is wrong because it is wrong". My point is this: I disagree with you. This is not because I have "misunderstood" something; it is because I understand the stochastic-parrot argument and think it is erroneous. And the more you talk about "the risk that people will come to falsely believe" rather than actual arguments, the less convincing you sound. This paternalistic tendency is a curse on science and debate in general.
    - cowl 4y ago
      Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.
      - FeepingCreature 4y ago
        Bad argument is bad argument, doesn't matter who says it.
    - usrbinbash 4y ago
      > it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.
      - FeepingCreature 4y ago
        There's a class of statements that can be either interpreted precisely, at which point the claim they make is clearly true but trivial, or interpreted expansively, at which point the claim is significant but no longer clearly true. This is one of those: yes, technically LLMs are token predictors, but technically any nondeterministic Turing machine is a token predictor. The human brain could be viewed as a token predictor [1]. The interesting question is how it comes up with its predictions, and on this the phrase offers no insight at all. [1] https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
  - a_bonobo 4y ago
    >Is there any Researcher who maintains that LLM models contain Reasoning and intent? That's the funny thing about the (in)famous OpenAI letter; the first sentence kind of does this: >AI systems with human-competitive intelligence can pose profound risks to society and humanity, as shown by extensive research[1] 'human-competitive intelligence' sounds like reasoning to me. What's even funnier is that [1] is the stochastic parrot paper, which argues exactly the opposite!
    - usrbinbash 4y ago
      > 'human-competitive intelligence' sounds like reasoning to me. Yes, and when AIs reach that level of intelligence, we can revisit the question. However, as long as LLMs will confidently try to explain to me why several footballs fit in an average coffee mug, I'd say we are still quite some way away from "human-competitive intelligence".
      - a_bonobo 4y ago
        Yes, I fully agree with you! there's no reasoning or intelligence in modern LLMs, but the OpenAI open letter and recent comms strongly implies there is

layer8 4y ago

Logical reasoning is the one thing LLMs are really bad at.

gat1 4y ago

LLMs cannot be considered as general reasoning engines because they do not make any kind of reasoning though.

mxmlnkn 4y ago

Since ChatGPT I've become much more aware of my own thoughts and written text. I'm now often wondering whether I'm just regurgitating the most frequently used next word or phrase or whether it could actually be described as original. Especially, for things like reacting with short answers to chat messages, I am confident that these are only reactionary answers without alternatives, which could have come from ChatGPT trained on my chat log. I feel like knowing and seeing how ChatGPT works can elevate our own thinking process. Or maybe it only is similar to awareness meditation.

tigerlily 4y ago
Now that is a thought that could not possibly have come from GPT. Thanks, this has given me something to think about.
a_bonobo 4y ago
In this vein, ChatGPT is a nice way to start thinking about any topic; ask it about its opinion on anything and it will give you the most run-of-the-mill, middle-of-the-road text that is possible, standing for nothing. If you find yourself agreeing with ChatGPT it may be time to reconsider your own thinking!
chomp 4y ago
LLMs may be an approximation of our knowledge, but understand that there’s more to reasoning than a language model. GPT understands how tokens relate to each other, but humans understand how objects, places, and abstract concepts relate to each other as well. We have a little further to go for AGI. I do believe that GPT4 is a really good approximation of our language though, and feel similarly to you when I respond off the cuff.

usrbinbash 4y ago

> I think we’re now way past that now with LLMs now quickly taking on the role of a general reasoning engine. No we're not, and no they are not. An LLM doesn't reason, period. It mimics reasoning ability by stochastically chosing a sequence of tokens. Alot of the time these make sense. At other times, they don't make any sense. I recently asked an LLM: "Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first?" It answered correctly that Mike leaves first. Then I asked: "If the elevator started at the 10th floor, who would have left first?" And the answer was that Mike still leaves first, because he leaves at the 2nd floor, and that's the first floor the elevator reaches. Another time I asked an LLM how many footballs fit in a coffe-mug, and the conversation reached a point where the AI tried to convince me, that coffe-mugs are only slightly smaller than the trunk of a car. Yes, they can also produce the correct answers to both these questions, but the fact that they can also spew such complete illogical nonsense shows that they are not "reasoning" about things. They complete sequences, that's it, period, that's literally the only thing a language model can do. Their apparent emergent abilities look like reasoning, in the same way as Jen from "The IT crowd" can sound like shes speaking Italian, when in fact she has no idea what she is even saying.

ianbutler 4y ago
GPT-4 reasons about a lot of gotcha logic puzzles correctly any pre GPT-4 opinions should be reconsidered, that is effectively two different epochs in the history of AI effectiveness and reasoning. > Me: Mike leaves the elevator at the 2nd floor. Jenny leaves at the 9th floor. Who left the elevator first? > GPT-4: Mike left the elevator first, as he got off at the 2nd floor, while Jenny left at the 9th floor. > Me: If the elevator started at the 10th floor, who would have left first? > GPT-4: If the elevator started at the 10th floor and went downward, then Jenny would have left first, as she got off at the 9th floor, while Mike left at the 2nd floor. > Me: How many footballs fit in a coffe-mug? > GPT-4: A standard football (soccer ball) has a diameter of around 22 centimeters (8.65 inches), while a coffee mug is typically much smaller, with a diameter of around 8-10 centimeters (3-4 inches). Therefore, it is not possible to fit a standard football inside a coffee mug. If you were to use a mini football or a much larger mug, the number of footballs that could fit would depend on the specific sizes of the footballs and the mug. It easily answered all of your questions and produces explanations I would expect most reasonable people to make.

qayxc 4y ago

I like that idea a lot. "General reasoning engine" sounds much more sensible than "AGI", especially given that the "I" in AGI is so poorly defined.