5 ms·

It sounds like you think this research is wrong? (it claims llms can not reason) https://arstechnica.com/ai/2024/10/llms-cant-perform-genuine-logical-reasoning

by nox101 2y ago

It sounds like you think this research is wrong? (it claims llms can not reason) https://arstechnica.com/ai/2024/10/llms-cant-perform-genuine-logical-reasoning-apple-researchers-suggest/ https://arstechnica.com/ai/2024/10/llms-cant-perform-genuine... or do you maybe think no logical reasoning is needed to do everything a human can do? Tho humans seem to be able to do logical reasoning

astrange 2y ago
It says "current" LLMs can't "genuinely" reason. Also, one of the researchers then posted an internship for someone to work on LLM reasoning. I think the paper should've included controls, because we don't know how strong the result is. They certainly may have proven that humans can't reason either.
- mannykannot 2y ago
  If they had human controls, they might well show that some humans can’t do any better, but based on how they generated test cases, it seems unlikely to me that doing so would prove that humans cannot reason (of course, if that’s actually the case, we cannot trust ourselves to devise, execute and interpret these tests in the first place!) Some people will use any limitation of LLMs to deny there is anything to see here, while others will call this ‘moving the goalposts’, but the most interesting questions, I believe, involve figuring out what the differences are, putting aside the question of whether LLMs are or are not AGIs.
CSMastermind 2y ago
The later. While I generally do suspect that we need to invent some new technique in the realm of AI in order for software to do everything a human can do, I use analogies like chess engines to caution myself from certainty.
bbor 2y ago
I’ll pop in with a friendly “that research is definitely wrong”. If they want to prove that LLMs can’t reason, shouldn’t they stringently define that word somewhere in their paper? As it stands, they’re proving something small (some of today’s LLMs have XYZ weaknesses) and claiming something big (humans have an ineffable calculator-soul). LLMs absolutely 100% can reason, if we take the dictionary definition; it’s trivial to show their ability to answer non-memorized questions, and the only way to do that is some sort of reasoning. I personally don’t think they’re the most efficient tool for deliberative derivation of concepts, but I also think any sort of categorical prohibition is anti-scientific. What is the brain other than a neural network? Even if we accept the most fringe, anthropocentric theories like Penrose & Hammerhoff’s quantum tubules, that’s just a neural network with fancy weights. How could we possibly hope to forbid digital recreations of our brains from “truly” or “really” mimicking them?
- visarga 2y ago
  Chasing our own tail with concepts like "reasoning". Let's move the concept a bit - "search". Can LLMs search for novel ideas and discoveries? They do under the right circumstances. You got to provide idea testing environments, the missing ingredient. Search and learn, it's what humans do and AI can do as well. The whole issue with "reasoning" is that is an incompletely defined concept. Over what domain, what problem space, and what kind of experimental access do we define "reasoning"? Search is better as a concept because it comes packed with all these things, and without conceptual murkiness. Search is scientifically studied to a greater extent. I don't think we doubt LLMs can learn given training data, we already accuse them of being mere interpolators or parrots. And we can agree to some extent the LLMs can recombine concepts correctly. So they got down the learning part. And for the searching part, we can probably agree its a matter of access to the search space not AI. It's an environment problem, and even a social one. Search is usually more extended than the lifetime of any agent, so it has to be a cultural process, where language plays a central role. When you break reasoning/progress/intelligence into "search and learn" it becomes much more tractable and useful. We can also make more grounded predictions on AI, considering the needs for search that are implied, not just the needs for learning. How much search did AlphaZero need to beat us at go? How much search did humans pack in our 200K years history over 10,000 generations? What was the cost of that journey of search? That kind of questions. In my napkin estimations we solved 1:10000 of the problem by learning, search is 10000x to a million times harder.
  - shkkmo 2y ago
    You can't breakdown cognition into just "search" and "learn" without either ridiculously overloading those concepts or leaving a ton out.
- shkkmo 2y ago
  > LLMs absolutely 100% can reason, if we take the dictionary definition; it’s trivial to show their ability to answer non-memorized questions, and the only way to do that is some sort of reasoning. Um... What? That is a huge leap to make. 'Reasoning' is a specific type of thought process and humans regularly make complicated decisions without doing it. We uses hunches and intuition and gut feelings. We make all kinds of snap assessments that we don't have time to reason through. As such, answering novel questions doesn't necessarily show a system is capable of reasoning. I see absolutely nothing resumbling an argument for humans having an "ineffable calculator soul", I think that might be you projecting. There is no 'categorical prohibition', only an analysis of the current flaws of specific models. Personally, my skepticism about imminent AGI has to do believing we may be underestimating the complexity of the software running on our brain. We've reached the point where we can create digital "brains", or atleast portions of them. We may be missing some other pieces of a digital brain, or we may just not have the right software to run on it yet. I suspect it is both but that we'll have fully functional digital brains well before we figure out the software to run on them.

by nox101 2y ago

astrange 2y ago

It says "current" LLMs can't "genuinely" reason. Also, one of the researchers then posted an internship for someone to work on LLM reasoning. I think the paper should've included controls, because we don't know how strong the result is. They certainly may have proven that humans can't reason either.

mannykannot 2y ago
If they had human controls, they might well show that some humans can’t do any better, but based on how they generated test cases, it seems unlikely to me that doing so would prove that humans cannot reason (of course, if that’s actually the case, we cannot trust ourselves to devise, execute and interpret these tests in the first place!) Some people will use any limitation of LLMs to deny there is anything to see here, while others will call this ‘moving the goalposts’, but the most interesting questions, I believe, involve figuring out what the differences are, putting aside the question of whether LLMs are or are not AGIs.

CSMastermind 2y ago

The later. While I generally do suspect that we need to invent some new technique in the realm of AI in order for software to do everything a human can do, I use analogies like chess engines to caution myself from certainty.

bbor 2y ago

I’ll pop in with a friendly “that research is definitely wrong”. If they want to prove that LLMs can’t reason, shouldn’t they stringently define that word somewhere in their paper? As it stands, they’re proving something small (some of today’s LLMs have XYZ weaknesses) and claiming something big (humans have an ineffable calculator-soul). LLMs absolutely 100% can reason, if we take the dictionary definition; it’s trivial to show their ability to answer non-memorized questions, and the only way to do that is some sort of reasoning. I personally don’t think they’re the most efficient tool for deliberative derivation of concepts, but I also think any sort of categorical prohibition is anti-scientific. What is the brain other than a neural network? Even if we accept the most fringe, anthropocentric theories like Penrose & Hammerhoff’s quantum tubules, that’s just a neural network with fancy weights. How could we possibly hope to forbid digital recreations of our brains from “truly” or “really” mimicking them?

visarga 2y ago
Chasing our own tail with concepts like "reasoning". Let's move the concept a bit - "search". Can LLMs search for novel ideas and discoveries? They do under the right circumstances. You got to provide idea testing environments, the missing ingredient. Search and learn, it's what humans do and AI can do as well. The whole issue with "reasoning" is that is an incompletely defined concept. Over what domain, what problem space, and what kind of experimental access do we define "reasoning"? Search is better as a concept because it comes packed with all these things, and without conceptual murkiness. Search is scientifically studied to a greater extent. I don't think we doubt LLMs can learn given training data, we already accuse them of being mere interpolators or parrots. And we can agree to some extent the LLMs can recombine concepts correctly. So they got down the learning part. And for the searching part, we can probably agree its a matter of access to the search space not AI. It's an environment problem, and even a social one. Search is usually more extended than the lifetime of any agent, so it has to be a cultural process, where language plays a central role. When you break reasoning/progress/intelligence into "search and learn" it becomes much more tractable and useful. We can also make more grounded predictions on AI, considering the needs for search that are implied, not just the needs for learning. How much search did AlphaZero need to beat us at go? How much search did humans pack in our 200K years history over 10,000 generations? What was the cost of that journey of search? That kind of questions. In my napkin estimations we solved 1:10000 of the problem by learning, search is 10000x to a million times harder.
- shkkmo 2y ago
  You can't breakdown cognition into just "search" and "learn" without either ridiculously overloading those concepts or leaving a ton out.
shkkmo 2y ago
> LLMs absolutely 100% can reason, if we take the dictionary definition; it’s trivial to show their ability to answer non-memorized questions, and the only way to do that is some sort of reasoning. Um... What? That is a huge leap to make. 'Reasoning' is a specific type of thought process and humans regularly make complicated decisions without doing it. We uses hunches and intuition and gut feelings. We make all kinds of snap assessments that we don't have time to reason through. As such, answering novel questions doesn't necessarily show a system is capable of reasoning. I see absolutely nothing resumbling an argument for humans having an "ineffable calculator soul", I think that might be you projecting. There is no 'categorical prohibition', only an analysis of the current flaws of specific models. Personally, my skepticism about imminent AGI has to do believing we may be underestimating the complexity of the software running on our brain. We've reached the point where we can create digital "brains", or atleast portions of them. We may be missing some other pieces of a digital brain, or we may just not have the right software to run on it yet. I suspect it is both but that we'll have fully functional digital brains well before we figure out the software to run on them.