7 ms·

> LLMs are PhD-level reasoners in math and science, yet they fail at children's puzzles. How is this possible? Because they are not. Pattern matching question

> LLMs are PhD-level reasoners in math and science, yet they fail at children's puzzles. How is this possible? Because they are not. Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. It’s the same reason why most of the people who pass your leetcode tests don’t actually know how to build anything real. They are taught to the test not taught to reality.

wiseowise 1y ago
[flagged]
- bloqs 1y ago
  please consider a less emotive, flaming/personal tone in the future, hacker news is much more readable without it! I would broadly agree that it's a bit far, but the OPs point does have some validity, its often the same formulaic methodology
ACCount37 1y ago
"Not understanding or reasoning" is anthropocentric cope. There is very little practical difference between "understanding" and "reasoning" implemented in human mind and that implemented in LLMs. One notable difference, however, is that LLMs disproportionately suck at spatial reasoning. Which shouldn't be surprising, considering that their training datasets are almost entirely text. The ultimate wordcel makes for a poor shape rotator. All ARC-AGI tasks are "spatial reasoning" tasks. They aren't in any way special. They just force LLMs to perform in an area they're spectacularly weak at. And LLMs aren't good enough yet to be able to brute force through this innate deficiency with raw intelligence.
- HighGoldstein 1y ago
  > There is very little practical difference between "understanding" and "reasoning" implemented in human mind and that implemented in LLMs. Source?
  - NooneAtAll3 1y ago
    ...literally benchmarks the post is all about? practical difference is about results - and results are here
  - ACCount37 1y ago
    The primary source is: measured LLM performance on once-human-exclusive tasks - such as high end natural language processing or commonsense reasoning. Those things were once thought to require a human mind - clearly, not anymore. Human commonsense knowledge can be both captured and applied by a learning algorithm trained on nothing but a boatload of text. But another important source is: loads and loads of mech interpret research that tried to actually pry the black box open and see what happens on the inside. This found some amusing artifacts - such as latent world models that can be extracted from the hidden state, or neural circuits corresponding to high level abstracts being chained together to obtain the final outputs. Very similar to human "abstract thinking" in function - despite being implemented on a substrate of floating point math and not wet meat.
    - freejazz 1y ago
      I haven't seen LLMs perform common sense reasoning. Feel free to share some links. Your post reads like anthropomorphized nonsense.
      - ACCount37 1y ago
        What? Do you even know what "commonsense reasoning" means?
        freejazz 1y ago
        Do you?
        ACCount37 1y ago
        https://en.wikipedia.org/wiki/Commonsense_reasoning https://en.wikipedia.org/wiki/Commonsense_reasoning
- dwallin 1y ago
  Very much agree with this. Looking at the dimensionality of a given problem space is a very helpful heuristic when analyzing how likely an llm is going to be suitable/reliable for that task. Consider how important positional encodings are LLM performance. You also then have an attention model that operates in that 1-dimensional space. With multidimensional data significant transformations to encode into a higher dimensional abstraction needs to happen within the model itself, before the model can even attempt to intelligently manipulate it.
- fumeux_fume 1y ago
  For many people, the difference between how a language model solves a problem and how a human solves a problem is actually very important.
gwd 1y ago
> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Do submarines swim? I don't really care if it gets me where I want to go. The fact is that just two days ago, I asked Claude to look at some reasonably complicated concurrent code to which I had added a new feature, and asked it to list what tests needed to be added; and then when I asked GPT-5 to add them, it one-shot nailed the implementations. I've written a gist of it here: https://gitlab.com/-/snippets/4889253 https://gitlab.com/-/snippets/4889253 Seriously just even read the description of the test it's trying to write. In order to one-shot that code, it had to understand: - How the cache was supposed to work - How conceptually to set up the scenario described - How to assemble golang's concurrency primitives (channels, goroutines, and waitgroups), in the correct order, to achieve the goal. Did it have a library of concurrency testing patterns in its head? Probably -- so do I. Had it ever seen my exact package before in its training? Never. I just don't see how you can argue with a straight face that this is "pattern matching". If that's pattern matching, then pattern matching is not an insult. If anything, the examples in this article are the opposite. Take the second example, which is basically 'assemble these assorted pieces into a rectangle'. Nearly every adult has assembled a minimum of dozens of things in their lives; many have assembled thousands of things. So it's humans in this case who are simply "pattern matching questions on a contrived test", and the LLMs, which almost certainly didn't have a lot of "assemble these items" in their training data, that are reasoning out what's going on from first principles.
- Akronymus 1y ago
  > I just don't see how you can argue with a straight face that this is "pattern matching". If that's pattern matching, then pattern matching is not an insult. IMO its still "just" a, very good, autocomplete. No actual reasoning, but lots of statistics on what is the next token to spit out.
  - NoahZuniga 1y ago
    > Do submarines swim? That's the main point of the parent comment. Arguing about the definition of "reasoning" or "pattern matching" is just a waste of time. What really matters is if it produces helpful output. Arguing about that is way better! Instead of saying: "It's just pattern matching -> It won't improve the world", make an argument like: "AI's seem to have trouble specializing like humans -> adopting AI will increase error rates in business processes -> due to the amount of possible edge cases, most people will get into an edge case with no hope of escaping it -> many people's lives will get worse". The first example relies on us agreeing on the definition of pattern matching, and then taking a conclusion based on how those words feel. This has no hope of convincing me if I don't like your definition! The second one is an argument that could potentially convince me, even if I'm an AI optimist. It is also just by itself an interesting line of reasoning.
pessimizer 1y ago
> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Pattern matching is definitely the same thing as understanding and reasoning. The problem is that LLMs can't recognize patterns that are longer than a few paragraphs, because the tokens would have to be far too long. LLMs are a thing we are lucky to have because we have very fast computers and very smart mathematicians making very hard calculations very efficient and parallelizable. But they sit on top of a bed of an enormous amount of human written knowledge, and can only stretch so far from that bed before completely falling apart. Humans don't use tokenizers. The goal right now is to build a scaffolding of these dummies in order to get really complicated work done, but that work is only ever going to accidentally be correct because of an accumulation of errors. This may be enough for a lot if we try it 1000x and run manually-tuned algos over the output to find the good ones. But this is essentially manual work, done in the traditional way. edit: sorry, you're never going to convince me these things are geniuses when I chat to them for a couple of back and forth exchanges and they're already obviously losing track of everything, even what they just said. The good thing is that what they are is enough to do a lot, if you're a person who can be satisfied that they're not going to be your god anytime soon.
faangguyindia 1y ago
>Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. I think most of the problem i solve is also a pattern matching. The problems i am good at solving are the ones i've seen before or the ones i can break into problems i've seen before.
artrockalter 1y ago
> It’s the same reason why most of the people who pass your leetcode tests don’t actually know how to build anything real. They are taught to the test not taught to reality. True, and "Agentic Workflows" are now playing the same role as "Agile" in that both take the idea that if you have many people/LLMs that can solve toy problems but not real ones then you can still succeed by breaking down the real problems into toy problems and assigning them out.
Grimblewald 1y ago
I fully agree, and any doubters, I can assure you that an LLM is bested by most undergraduates when it comes to reasoning. LLM's will universally get smoked by any PhD. They have great big wealth of knowledge to draw on but critical thinking is sorely lacking. LLM's strength is being an interactive encyclopedia, not a decision making thing.

wiseowise 1y ago

[flagged]

bloqs 1y ago
please consider a less emotive, flaming/personal tone in the future, hacker news is much more readable without it! I would broadly agree that it's a bit far, but the OPs point does have some validity, its often the same formulaic methodology

ACCount37 1y ago

"Not understanding or reasoning" is anthropocentric cope. There is very little practical difference between "understanding" and "reasoning" implemented in human mind and that implemented in LLMs. One notable difference, however, is that LLMs disproportionately suck at spatial reasoning. Which shouldn't be surprising, considering that their training datasets are almost entirely text. The ultimate wordcel makes for a poor shape rotator. All ARC-AGI tasks are "spatial reasoning" tasks. They aren't in any way special. They just force LLMs to perform in an area they're spectacularly weak at. And LLMs aren't good enough yet to be able to brute force through this innate deficiency with raw intelligence.

HighGoldstein 1y ago
> There is very little practical difference between "understanding" and "reasoning" implemented in human mind and that implemented in LLMs. Source?
- NooneAtAll3 1y ago
  ...literally benchmarks the post is all about? practical difference is about results - and results are here
- ACCount37 1y ago
  The primary source is: measured LLM performance on once-human-exclusive tasks - such as high end natural language processing or commonsense reasoning. Those things were once thought to require a human mind - clearly, not anymore. Human commonsense knowledge can be both captured and applied by a learning algorithm trained on nothing but a boatload of text. But another important source is: loads and loads of mech interpret research that tried to actually pry the black box open and see what happens on the inside. This found some amusing artifacts - such as latent world models that can be extracted from the hidden state, or neural circuits corresponding to high level abstracts being chained together to obtain the final outputs. Very similar to human "abstract thinking" in function - despite being implemented on a substrate of floating point math and not wet meat.
  - freejazz 1y ago
    I haven't seen LLMs perform common sense reasoning. Feel free to share some links. Your post reads like anthropomorphized nonsense.
    - ACCount37 1y ago
      What? Do you even know what "commonsense reasoning" means?
      - freejazz 1y ago
        Do you?
        ACCount37 1y ago
        https://en.wikipedia.org/wiki/Commonsense_reasoning https://en.wikipedia.org/wiki/Commonsense_reasoning
dwallin 1y ago
Very much agree with this. Looking at the dimensionality of a given problem space is a very helpful heuristic when analyzing how likely an llm is going to be suitable/reliable for that task. Consider how important positional encodings are LLM performance. You also then have an attention model that operates in that 1-dimensional space. With multidimensional data significant transformations to encode into a higher dimensional abstraction needs to happen within the model itself, before the model can even attempt to intelligently manipulate it.
fumeux_fume 1y ago
For many people, the difference between how a language model solves a problem and how a human solves a problem is actually very important.

gwd 1y ago

> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Do submarines swim? I don't really care if it gets me where I want to go. The fact is that just two days ago, I asked Claude to look at some reasonably complicated concurrent code to which I had added a new feature, and asked it to list what tests needed to be added; and then when I asked GPT-5 to add them, it one-shot nailed the implementations. I've written a gist of it here: https://gitlab.com/-/snippets/4889253 https://gitlab.com/-/snippets/4889253 Seriously just even read the description of the test it's trying to write. In order to one-shot that code, it had to understand: - How the cache was supposed to work - How conceptually to set up the scenario described - How to assemble golang's concurrency primitives (channels, goroutines, and waitgroups), in the correct order, to achieve the goal. Did it have a library of concurrency testing patterns in its head? Probably -- so do I. Had it ever seen my exact package before in its training? Never. I just don't see how you can argue with a straight face that this is "pattern matching". If that's pattern matching, then pattern matching is not an insult. If anything, the examples in this article are the opposite. Take the second example, which is basically 'assemble these assorted pieces into a rectangle'. Nearly every adult has assembled a minimum of dozens of things in their lives; many have assembled thousands of things. So it's humans in this case who are simply "pattern matching questions on a contrived test", and the LLMs, which almost certainly didn't have a lot of "assemble these items" in their training data, that are reasoning out what's going on from first principles.

Akronymus 1y ago
> I just don't see how you can argue with a straight face that this is "pattern matching". If that's pattern matching, then pattern matching is not an insult. IMO its still "just" a, very good, autocomplete. No actual reasoning, but lots of statistics on what is the next token to spit out.
- NoahZuniga 1y ago
  > Do submarines swim? That's the main point of the parent comment. Arguing about the definition of "reasoning" or "pattern matching" is just a waste of time. What really matters is if it produces helpful output. Arguing about that is way better! Instead of saying: "It's just pattern matching -> It won't improve the world", make an argument like: "AI's seem to have trouble specializing like humans -> adopting AI will increase error rates in business processes -> due to the amount of possible edge cases, most people will get into an edge case with no hope of escaping it -> many people's lives will get worse". The first example relies on us agreeing on the definition of pattern matching, and then taking a conclusion based on how those words feel. This has no hope of convincing me if I don't like your definition! The second one is an argument that could potentially convince me, even if I'm an AI optimist. It is also just by itself an interesting line of reasoning.

pessimizer 1y ago

> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Pattern matching is definitely the same thing as understanding and reasoning. The problem is that LLMs can't recognize patterns that are longer than a few paragraphs, because the tokens would have to be far too long. LLMs are a thing we are lucky to have because we have very fast computers and very smart mathematicians making very hard calculations very efficient and parallelizable. But they sit on top of a bed of an enormous amount of human written knowledge, and can only stretch so far from that bed before completely falling apart. Humans don't use tokenizers. The goal right now is to build a scaffolding of these dummies in order to get really complicated work done, but that work is only ever going to accidentally be correct because of an accumulation of errors. This may be enough for a lot if we try it 1000x and run manually-tuned algos over the output to find the good ones. But this is essentially manual work, done in the traditional way. edit: sorry, you're never going to convince me these things are geniuses when I chat to them for a couple of back and forth exchanges and they're already obviously losing track of everything, even what they just said. The good thing is that what they are is enough to do a lot, if you're a person who can be satisfied that they're not going to be your god anytime soon.

faangguyindia 1y ago

>Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. I think most of the problem i solve is also a pattern matching. The problems i am good at solving are the ones i've seen before or the ones i can break into problems i've seen before.

artrockalter 1y ago

> It’s the same reason why most of the people who pass your leetcode tests don’t actually know how to build anything real. They are taught to the test not taught to reality. True, and "Agentic Workflows" are now playing the same role as "Agile" in that both take the idea that if you have many people/LLMs that can solve toy problems but not real ones then you can still succeed by breaking down the real problems into toy problems and assigning them out.

Grimblewald 1y ago

I fully agree, and any doubters, I can assure you that an LLM is bested by most undergraduates when it comes to reasoning. LLM's will universally get smoked by any PhD. They have great big wealth of knowledge to draw on but critical thinking is sorely lacking. LLM's strength is being an interactive encyclopedia, not a decision making thing.