8 ms·

> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Do submarines swim? I don't really care if it gets me wh

by gwd 1y ago

> Pattern matching questions on a contrived test is not the same thing as understanding or reasoning. Do submarines swim? I don't really care if it gets me where I want to go. The fact is that just two days ago, I asked Claude to look at some reasonably complicated concurrent code to which I had added a new feature, and asked it to list what tests needed to be added; and then when I asked GPT-5 to add them, it one-shot nailed the implementations. I've written a gist of it here: https://gitlab.com/-/snippets/4889253 https://gitlab.com/-/snippets/4889253 Seriously just even read the description of the test it's trying to write. In order to one-shot that code, it had to understand: - How the cache was supposed to work - How conceptually to set up the scenario described - How to assemble golang's concurrency primitives (channels, goroutines, and waitgroups), in the correct order, to achieve the goal. Did it have a library of concurrency testing patterns in its head? Probably -- so do I. Had it ever seen my exact package before in its training? Never. I just don't see how you can argue with a straight face that this is "pattern matching". If that's pattern matching, then pattern matching is not an insult. If anything, the examples in this article are the opposite. Take the second example, which is basically 'assemble these assorted pieces into a rectangle'. Nearly every adult has assembled a minimum of dozens of things in their lives; many have assembled thousands of things. So it's humans in this case who are simply "pattern matching questions on a contrived test", and the LLMs, which almost certainly didn't have a lot of "assemble these items" in their training data, that are reasoning out what's going on from first principles.

Akronymus 1y ago
> I just don't see how you can argue with a straight face that this is "pattern matching". If that's pattern matching, then pattern matching is not an insult. IMO its still "just" a, very good, autocomplete. No actual reasoning, but lots of statistics on what is the next token to spit out.
- NoahZuniga 1y ago
  > Do submarines swim? That's the main point of the parent comment. Arguing about the definition of "reasoning" or "pattern matching" is just a waste of time. What really matters is if it produces helpful output. Arguing about that is way better! Instead of saying: "It's just pattern matching -> It won't improve the world", make an argument like: "AI's seem to have trouble specializing like humans -> adopting AI will increase error rates in business processes -> due to the amount of possible edge cases, most people will get into an edge case with no hope of escaping it -> many people's lives will get worse". The first example relies on us agreeing on the definition of pattern matching, and then taking a conclusion based on how those words feel. This has no hope of convincing me if I don't like your definition! The second one is an argument that could potentially convince me, even if I'm an AI optimist. It is also just by itself an interesting line of reasoning.
- ozgung 1y ago
  No it's not "just a very good autocomplete". I don't know why people repeat this thing (it's wrong) but I find it an extremely counterproductive position. Some people just love to dismiss the capabilities of AI with a very shallow understanding of how it works. Why? It generates words one by one, like we all do. This doesn't mean it does just that and nothing else. It's the mechanics of how they are trained and how they do inference. And most importantly how they communicate with us. It doesn't define what they are or their limits. This is reductionism. Ignoring the mathematical complexity of a giant neural network.
  - deleted 1y ago
    [deleted]
  - karmakaze 1y ago
    I can't say for certain that our wetware isn't "just a very good autocomplete".
    - esafak 1y ago
      A very good autocomplete is realized by developing an understanding.
  - Bjartr 1y ago
    > like we all do Do we though? Sure, we communicate sequentially, but that doesn't mean that our internal effort is piecewise and linear. A modern transformer LLM however is. Each token is sampled from a population exclusively dependent on the tokens that came before it. Mechanistically speaking, it works similarly to autocomplete, but at a very different scale. Now how much of an unavoidable handicap this incurs, if any, is absolutely up for debate. But yes, taking this mechanistic truth and only considering it in a shallow manner underestimates the capability of LLMs by a large degree.
    - kenjackson 1y ago
      Our thinking is also based only on events that occurred previously in time. We don’t use events in the future.
HarHarVeryFunny 1y ago
> Do submarines swim? It doesn't matter HOW LLMs "swim" as long as they can, but the point being raised is whether they actually can. It's as if LLMs can swim in the ocean, in rough surf, but fail to swim in rivers or swimming pools, because they don't have a generalized ability to swim - they've just been RL-trained on the solution steps to swimming in surf, but since those exact conditions don't exist in a river (which might seem like a less challenging environment), they fail there. So, the question that might be asked is when LLMs are trained to perform well in these vertical domains like math and programming, where it's easy to verify results and provide outcome- or process-based RL rewards, are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Does the LLM have the capability to reason/swim, or is it really just an expert system that has been given the rules to reason/swim in certain cases, but would need to be similarly hand fed the reasoning steps to be successful in other cases? I think the answer is pretty obvious given that LLM's can't learn at runtime - can't try out some reasoning generalization they may have arrived at, find that it doesn't work in a specific case, then explore the problem and figure it out for next time. Given that it's Demis Hassabis who it pointing out this deficiency of LLMs (and has a 5-10 year plan/timeline to fix it - AGI), not some ill-informed LLM critic, it seems silly to deny it.
- naasking 1y ago
  > are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Are you sure there's a real difference? Do you have a definition of "reasoning" that excludes this?
  - freejazz 1y ago
    It seems readily apparent there is a difference given their inability to do tasks we would otherwise reasonably describe as achievable via basic reasoning on the same facts.
    - naasking 1y ago
amelius 1y ago
Most of our coding is just plumbing. Getting data from one place to where it needs to be. There is no advanced reasoning necessary. Just a good idea of the structure of the code and the data-structures. Even high school maths tests are way harder than what most professional programmers do on a daily basis.
OtherShrezzing 1y ago
>and the LLMs, which almost certainly didn't have a lot of "assemble these items" in their training data I don't think this assumption is sound. Humans write a huge amount on "assemble components x and y to make entity z". I'd expect all LLMs to have consumed every IKEA type instruction manual, the rules for Jenga, all geometry textbooks and papers ever written.
vlovich123 1y ago
I could be mistaken but generally LLMs cannot tackle out-of-domain problems whereas humans do seem to have that capability. Relatedly, the energy costs are wildly different suggesting that LLMs are imitating some kind of thought but not simulating it. They’re doing a remarkable job of passing the Turing test but that says more about the limitations of the Turing test than it does about the capabilities of the LLMs.
Grimblewald 1y ago
> just read the description Seems fine enough to me. Wanna really challenge an LLM? get it to make an image stitching algorithm that isn't shit. Implement the results from brown et al https://link.springer.com/article/10.1007/s11263-006-0002-3 https://link.springer.com/article/10.1007/s11263-006-0002-3 and I'll be impressed. This is a paper from 2007 and there are plenty of packages available to help make it all happen through some API calls and a bit of cleverness on the coders part, and so far not a single LLM has gotten close to an acceptable implementation. Not a single one. Now, why is it so hard? Because there's not public code for good quality high performance image stitching on the level of the image composite editor microsoft research once hosted. There's nothing for the LLM's to draw on and they fundamentally lack reasoning / planning other than something that superficially resembles it, but it falls apart for out of domain things where humans still do fine even if new to the task.
washadjeffmad 1y ago
I tend to skip these threads because there's a gulf in understanding between those who have accepted the marketing and those who are hoping to profit from them. "PhD level reasoning" means they have asked PhDs questions and finetuned on their responses. It does not mean that every response is "PhD level", that the models only provide responses that PhDs have validated, or that every response is correct. It is "Lysol kills 99.9% of germs" logic. The domains I work in and contribute to largely view being paid to write out their thought processes when answering questions models answer incorrectly as a novelty, like grading homework with some minor bragging rights.

by gwd 1y ago

Akronymus 1y ago

> I just don't see how you can argue with a straight face that this is "pattern matching". If that's pattern matching, then pattern matching is not an insult. IMO its still "just" a, very good, autocomplete. No actual reasoning, but lots of statistics on what is the next token to spit out.

NoahZuniga 1y ago
> Do submarines swim? That's the main point of the parent comment. Arguing about the definition of "reasoning" or "pattern matching" is just a waste of time. What really matters is if it produces helpful output. Arguing about that is way better! Instead of saying: "It's just pattern matching -> It won't improve the world", make an argument like: "AI's seem to have trouble specializing like humans -> adopting AI will increase error rates in business processes -> due to the amount of possible edge cases, most people will get into an edge case with no hope of escaping it -> many people's lives will get worse". The first example relies on us agreeing on the definition of pattern matching, and then taking a conclusion based on how those words feel. This has no hope of convincing me if I don't like your definition! The second one is an argument that could potentially convince me, even if I'm an AI optimist. It is also just by itself an interesting line of reasoning.
ozgung 1y ago
No it's not "just a very good autocomplete". I don't know why people repeat this thing (it's wrong) but I find it an extremely counterproductive position. Some people just love to dismiss the capabilities of AI with a very shallow understanding of how it works. Why? It generates words one by one, like we all do. This doesn't mean it does just that and nothing else. It's the mechanics of how they are trained and how they do inference. And most importantly how they communicate with us. It doesn't define what they are or their limits. This is reductionism. Ignoring the mathematical complexity of a giant neural network.
- deleted 1y ago
  [deleted]
- karmakaze 1y ago
  I can't say for certain that our wetware isn't "just a very good autocomplete".
  - esafak 1y ago
    A very good autocomplete is realized by developing an understanding.
- Bjartr 1y ago
  > like we all do Do we though? Sure, we communicate sequentially, but that doesn't mean that our internal effort is piecewise and linear. A modern transformer LLM however is. Each token is sampled from a population exclusively dependent on the tokens that came before it. Mechanistically speaking, it works similarly to autocomplete, but at a very different scale. Now how much of an unavoidable handicap this incurs, if any, is absolutely up for debate. But yes, taking this mechanistic truth and only considering it in a shallow manner underestimates the capability of LLMs by a large degree.
  - kenjackson 1y ago
    Our thinking is also based only on events that occurred previously in time. We don’t use events in the future.

HarHarVeryFunny 1y ago

> Do submarines swim? It doesn't matter HOW LLMs "swim" as long as they can, but the point being raised is whether they actually can. It's as if LLMs can swim in the ocean, in rough surf, but fail to swim in rivers or swimming pools, because they don't have a generalized ability to swim - they've just been RL-trained on the solution steps to swimming in surf, but since those exact conditions don't exist in a river (which might seem like a less challenging environment), they fail there. So, the question that might be asked is when LLMs are trained to perform well in these vertical domains like math and programming, where it's easy to verify results and provide outcome- or process-based RL rewards, are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Does the LLM have the capability to reason/swim, or is it really just an expert system that has been given the rules to reason/swim in certain cases, but would need to be similarly hand fed the reasoning steps to be successful in other cases? I think the answer is pretty obvious given that LLM's can't learn at runtime - can't try out some reasoning generalization they may have arrived at, find that it doesn't work in a specific case, then explore the problem and figure it out for next time. Given that it's Demis Hassabis who it pointing out this deficiency of LLMs (and has a 5-10 year plan/timeline to fix it - AGI), not some ill-informed LLM critic, it seems silly to deny it.

naasking 1y ago
> are they really learning to reason, or are they just learning to pattern match to steer generation in the direction of problem-specific reasoning steps that they had been trained on? Are you sure there's a real difference? Do you have a definition of "reasoning" that excludes this?
- freejazz 1y ago
  It seems readily apparent there is a difference given their inability to do tasks we would otherwise reasonably describe as achievable via basic reasoning on the same facts.
  - naasking 1y ago

amelius 1y ago

Most of our coding is just plumbing. Getting data from one place to where it needs to be. There is no advanced reasoning necessary. Just a good idea of the structure of the code and the data-structures. Even high school maths tests are way harder than what most professional programmers do on a daily basis.

OtherShrezzing 1y ago

>and the LLMs, which almost certainly didn't have a lot of "assemble these items" in their training data I don't think this assumption is sound. Humans write a huge amount on "assemble components x and y to make entity z". I'd expect all LLMs to have consumed every IKEA type instruction manual, the rules for Jenga, all geometry textbooks and papers ever written.

vlovich123 1y ago

I could be mistaken but generally LLMs cannot tackle out-of-domain problems whereas humans do seem to have that capability. Relatedly, the energy costs are wildly different suggesting that LLMs are imitating some kind of thought but not simulating it. They’re doing a remarkable job of passing the Turing test but that says more about the limitations of the Turing test than it does about the capabilities of the LLMs.

Grimblewald 1y ago

> just read the description Seems fine enough to me. Wanna really challenge an LLM? get it to make an image stitching algorithm that isn't shit. Implement the results from brown et al https://link.springer.com/article/10.1007/s11263-006-0002-3 https://link.springer.com/article/10.1007/s11263-006-0002-3 and I'll be impressed. This is a paper from 2007 and there are plenty of packages available to help make it all happen through some API calls and a bit of cleverness on the coders part, and so far not a single LLM has gotten close to an acceptable implementation. Not a single one. Now, why is it so hard? Because there's not public code for good quality high performance image stitching on the level of the image composite editor microsoft research once hosted. There's nothing for the LLM's to draw on and they fundamentally lack reasoning / planning other than something that superficially resembles it, but it falls apart for out of domain things where humans still do fine even if new to the task.

washadjeffmad 1y ago

I tend to skip these threads because there's a gulf in understanding between those who have accepted the marketing and those who are hoping to profit from them. "PhD level reasoning" means they have asked PhDs questions and finetuned on their responses. It does not mean that every response is "PhD level", that the models only provide responses that PhDs have validated, or that every response is correct. It is "Lysol kills 99.9% of germs" logic. The domains I work in and contribute to largely view being paid to write out their thought processes when answering questions models answer incorrectly as a novelty, like grading homework with some minor bragging rights.