4 ms·

There's some argument to be made that a form of reasoning happens in a roundabout way when the AI is told to explain it's reasoning. For example if you tell it

There's some argument to be made that a form of reasoning happens in a roundabout way when the AI is told to explain it's reasoning. For example if you tell it "Do <thing>" and then open a new context and say "Do <thing>, explain your reasoning beforehand." you will often get a more accurate response. Granted, it's not that any "Hmm, let me think about that." Deep Thought reasoning occurs, but simply that predicting what the reasoning would look like and then predicting what comes after that reasoning results in a more accurate - and ironically, reasoned - response. Kinda funny actually, it's a bit like how in Hitchiker's Guide they just had to tell the probability machine to calculate the odds of an improbability drive in order to create it.

FishInTheWater 3y ago
This is where the terminology becomes a bit annoying, but there is a key difference in the kinds of reasoning at work here. When you ask LLMs to provide a reasoning, the actual reasoning performed is linguistic; The LLM has (is) a model about language and performs some (limited) reasoning on that model to get an output. But that is explicitly different from reasoning about the abstract question at hand, thus the answer is mostly a guess. The key difference to observe is that "semantic reasoners" like computer algebra or prolog, always maintain correctness within the axioms provided. They may slow down significantly as questions get more complex, but they do not start providing wrong answers. Computers are flawless mathematicians, provided they are programmed correctly. LLMs do provide increasingly more-wrong answers as the question gets more complex. Thus we can observe that LLMs do not abstractly reason about the question and it's model.
- famouswaffles 3y ago
  >Thus we can observe that LLMs do not abstractly reason about the question and it's model. Your conclusion makes no sense. Humans provide increasingly wrong answers as questions get more complex too. Jumping from that to "incapable of abstract reasoning" is silly. You have not "trivially proven" anything at all >The LLM has (is) a model about language and performs some (limited) reasoning on that model to get an output. LLMs generalize to non linguistic patterns. https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/
  - FishInTheWater 3y ago
    Humans provide increasingly wrong answers as questions get more complex too. Human this, Human that. LLMs aren't humans. "My model is crap but the human brain isn't very good at this either" is irrelevant when we have machines that are not only very good at these tasks but almost perfect at them. Humans make such mistakes precisely because they are not perfect reasoning machines. To compare LLMs to humans is not only disingenuous, but proves my point. (And no, I will not humour you with an argument about how the amount of wrong answers is drastically lower from human mathematicians) Jumping from that to "incapable of abstract reasoning" is silly. They are language models. It is explicitly what they are designed to do. If these LLMs are not, as I claim, reasoning on language rather than the abstract model of the query, then how come they fail miserably in exactly the ways you would expect where that the case? LLMs generalize to non linguistic patterns. Yes, congratulations, if you turn a problem into a linguistic one LLMs can deal with them. This does not in any way go against what I said about the capabilities of LLMs. The same levels of actual abstract reasoning can be achieved on a graphing calculator running off literal potatoes.
    - famouswaffles 3y ago
      >Human this, Human that. LLMs aren't humans. You said you trivially proved something and made up nonsensical lines of reasoning to justify it. If your "proof" can't port to Humans then it's not proof. You are just rambling. >Humans make such mistakes precisely because they are not perfect reasoning machines. Nobody is calling LLMs perfect reasoning machines. Your "point" was that they don't reason at all which of none of your ramblings has been able to "prove". >If these LLMs are not, as I claim, reasoning on language rather than the abstract model of the query, then how come they fail miserably in exactly the ways you would expect where that the case? They don't. The idea that you must make no mistake reasoning before you can be considered to be reasoning has no ground. >LLMs generalize to non linguistic patterns. Yes, congratulations, if you turn a problem into a linguistic one LLMs can deal with them. Can you read ? Did you even bother looking at the link? LLMs don't need patterns to be linguistic to reason over them lol. None of those patterns are turned linguistic. Some of them are arbitrary numbers that resemble nothing like the data they've been trained on.
- JustBreath 3y ago
  Seems like a blurry line between "reason" and "guessing." Kind of like how an educated guess by a professional is often more accurate than a well reasoned opinion of a layman. The professional may not have reasoned it so much as intuited, but within that intuition is a lot of wisdom. I suppose "predicting" is a more precise word than guessing or reasoning. Guessing implies an arbitrary nature, reasoning implies understanding the concepts at some level.

FishInTheWater 3y ago

This is where the terminology becomes a bit annoying, but there is a key difference in the kinds of reasoning at work here. When you ask LLMs to provide a reasoning, the actual reasoning performed is linguistic; The LLM has (is) a model about language and performs some (limited) reasoning on that model to get an output. But that is explicitly different from reasoning about the abstract question at hand, thus the answer is mostly a guess. The key difference to observe is that "semantic reasoners" like computer algebra or prolog, always maintain correctness within the axioms provided. They may slow down significantly as questions get more complex, but they do not start providing wrong answers. Computers are flawless mathematicians, provided they are programmed correctly. LLMs do provide increasingly more-wrong answers as the question gets more complex. Thus we can observe that LLMs do not abstractly reason about the question and it's model.

famouswaffles 3y ago
>Thus we can observe that LLMs do not abstractly reason about the question and it's model. Your conclusion makes no sense. Humans provide increasingly wrong answers as questions get more complex too. Jumping from that to "incapable of abstract reasoning" is silly. You have not "trivially proven" anything at all >The LLM has (is) a model about language and performs some (limited) reasoning on that model to get an output. LLMs generalize to non linguistic patterns. https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/
- FishInTheWater 3y ago
  Humans provide increasingly wrong answers as questions get more complex too. Human this, Human that. LLMs aren't humans. "My model is crap but the human brain isn't very good at this either" is irrelevant when we have machines that are not only very good at these tasks but almost perfect at them. Humans make such mistakes precisely because they are not perfect reasoning machines. To compare LLMs to humans is not only disingenuous, but proves my point. (And no, I will not humour you with an argument about how the amount of wrong answers is drastically lower from human mathematicians) Jumping from that to "incapable of abstract reasoning" is silly. They are language models. It is explicitly what they are designed to do. If these LLMs are not, as I claim, reasoning on language rather than the abstract model of the query, then how come they fail miserably in exactly the ways you would expect where that the case? LLMs generalize to non linguistic patterns. Yes, congratulations, if you turn a problem into a linguistic one LLMs can deal with them. This does not in any way go against what I said about the capabilities of LLMs. The same levels of actual abstract reasoning can be achieved on a graphing calculator running off literal potatoes.
  - famouswaffles 3y ago
    >Human this, Human that. LLMs aren't humans. You said you trivially proved something and made up nonsensical lines of reasoning to justify it. If your "proof" can't port to Humans then it's not proof. You are just rambling. >Humans make such mistakes precisely because they are not perfect reasoning machines. Nobody is calling LLMs perfect reasoning machines. Your "point" was that they don't reason at all which of none of your ramblings has been able to "prove". >If these LLMs are not, as I claim, reasoning on language rather than the abstract model of the query, then how come they fail miserably in exactly the ways you would expect where that the case? They don't. The idea that you must make no mistake reasoning before you can be considered to be reasoning has no ground. >LLMs generalize to non linguistic patterns. Yes, congratulations, if you turn a problem into a linguistic one LLMs can deal with them. Can you read ? Did you even bother looking at the link? LLMs don't need patterns to be linguistic to reason over them lol. None of those patterns are turned linguistic. Some of them are arbitrary numbers that resemble nothing like the data they've been trained on.
JustBreath 3y ago
Seems like a blurry line between "reason" and "guessing." Kind of like how an educated guess by a professional is often more accurate than a well reasoned opinion of a layman. The professional may not have reasoned it so much as intuited, but within that intuition is a lot of wisdom. I suppose "predicting" is a more precise word than guessing or reasoning. Guessing implies an arbitrary nature, reasoning implies understanding the concepts at some level.