6 ms·

Humans are notoriously bad at formal logic. The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s

Humans are notoriously bad at formal logic. The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. That looks a lot more like pattern matching than rule application. Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. The slow, deliberate, step-by-step stuff is effortful and error-prone, and people avoid it when they can. And even when they do engage it, they’re often confabulating a logical-sounding justification for a conclusion they already reached by other means. So maybe the honest answer is: the gap between what LLMs do and what most humans do most of the time might be smaller than people assume. The story that humans have access to some pure deductive engine and LLMs are just faking it with statistics might be flattering to humans more than it’s accurate. Where I’d still flag a possible difference is something like adaptability. A person can learn a totally new formal system and start applying its rules, even if clumsily. Whether LLMs can genuinely do that outside their training distribution or just interpolate convincingly is still an open question. But then again, how often do humans actually reason outside their own “training distribution”? Most human insight happens within well-practiced domains.

lich_king 7mo ago
> The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. I've never heard about the Wason selection task, looked it up, and could tell the right answer right away. But I can also tell you why: because I have some familiarity with formal logic and can, in your words, pattern-match the gotcha that "if x then y" is distinct from "if not x then not y". In contrast to you, this doesn't make me believe that people are bad at logic or don't really think. It tells me that people are unfamiliar with "gotcha" formalities introduced by logicians that don't match the everyday use of language. If you added a simple additional to the problem, such as "Note that in this context, 'if' only means that...", most people would almost certainly answer it correctly. Mind you, I'm not arguing that human thinking is necessarily more profound from what what LLMs could ever do. However, judging from the output, LLMs have a tenuous grasp on reality, so I don't think that reductionist arguments along the lines of "humans are just as dumb" are fair. There's a difference that we don't really know how to overcome.
- edanm 7mo ago
  Agree with much of your comment. Though note that as GP said, on the Wason selection task, people famously do much better when it's framed in a social context. That at least partially undermines your theory that its lack of familiarity with the terminology of formal logic.
  - the_mar 7mo ago
    I for the life of me could not solve the <18 example from wikipedia. but the number/color one is super easy
  - deleted 7mo ago
    [deleted]
  - Ajedi32 7mo ago
    Maybe the social version just creates a context where "if x then y" obviously does not include "if not x then not y". Everyone knows people over the drinking age can drink both alcoholic and non-alcoholic drinks, so you obviously don't have to check the person drinking the soft drink to make sure they aren't an adult.
- lugu 7mo ago
  Your response contains a performative contradiction: you are asserting that humans are naturally logical while simultaneously committing several logical errors to defend that claim.
  - jacquesm 7mo ago
    This comment would be a lot more useful with an enumeration of those logical errors.
    - lugu 7mo ago
      commenter’s specific claim—that adding a note about the definition of "if" would solve the problem—is a moving the goalposts fallacy and a tautology. The comment also suffers from hasty generalization (in their experience the test isn't hard) and special pleading (double standard for LLM and humans).
jonahx 7mo ago
> The story that humans have access to some pure deductive engine and LLMs are just faking it with statistics might be flattering to humans more than it’s accurate. Your point rings true with most human reasoning most of the time. Still, at least some humans do have the capability to run that deductive engine, and it seems to be a key part (though not the only part) of scientific and mathematical reasoning. Even informal experimentation and iteration rest on deductive feedback loops.
- Nevermark 7mo ago
  The fact that humans can learn to do X, sometimes well, often badly, and while many don’t, strongly supports the conjecture that X is not how they naturally do things. I can perform symbolic calculations too. But most people have limited versions of this skill, and many people who don’t learn to think symbolically have full lives. I think it is fair to say humans don’t naturally think in formal or symbolic reasoning terms. People pattern match, Another clue is humans have to practice things, become familiar with them to reason even somewhat reliable about them. Even if they already learned some formal reasoning. —- Higher level reasoning is always implemented as specific forms of lower order reasoning. There is confusion about substrate processing vs. what higher order processes can be created with that substrate. We can “just” be doing pattern matching from an implementation view, and yet go far “beyond” pattern matching with specific compositions of pattern matching, from a capability view. How else could neurons think? We are “only” neurons. Yet we far surpass the kinds of capabilities neurons have.
  - jonahx 7mo ago
    I don't disagree with any of that. My comment was only in relation to the question of human-specific capability that current LLMs may not be able to duplicate. I was not making the value judgments you seem to have read.
- mikkupikku 7mo ago
  When people do math or rigorous deductive reasoning, are we sure they aren't just pattern matching with a set of carefully chosen interacting patterns that have been refined by ancient philosophers as being useful patterns that produce consistent results when applied in correctly patterned ways?
nextaccountic 7mo ago
> Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. The slow, deliberate, step-by-step stuff is effortful and error-prone, and people avoid it when they can. And even when they do engage it, they’re often confabulating a logical-sounding justification for a conclusion they already reached by other means. Some references on that https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow https://thedecisionlab.com/reference-guide/philosophy/system-1-and-system-2-thinking https://thedecisionlab.com/reference-guide/philosophy/system... System 1 really looks like a LLM (indeed completing a phrase is an example of what it can do, like, "you either die a hero, or you live enough to become the _"). It's largely unconscious and runs all the time, pattern matching on random stuff System 2 is something else and looks like a supervisor system, a higher level stuff that can be consciously directed through your own will But the two systems run at the same time and reinforce each other
- drdaeman 7mo ago
  In my naive understanding, neither requires any will or consciousness. S1 is “bare” language production, picking words or concepts to say or think by a fancy pattern prediction. There’s no reasoning at this level, just blabbering. However, language by itself weeds out too obvious nonsense purely statistically (some concepts are rarely in the same room), but we may call that “mindlessly” - that’s why even early LLMs produced semi-meaningful texts. S2 is a set of patterns inside the language (“logic”), that biases S1 to produce reasoning-like phrases. Doesn’t require any consciousness or will, just concepts pushing S1 towards a special structure, simply backing one keeps them “in mind” and throws in the mix. I suspect S2 has a spectrum of rigorousness, because one can just throw in some rules (like “if X then Y, not Y therefore not X”) or may do fancier stuff (imposing a larger structure to it all, like formulating and testing a null hypothesis). Either way it all falls down onto S1 for a ultimate decision-making, a sense of what sounds right (allowing us our favorite logical flaws), thus the fancier the rules (patterns of “thought”) the more likely reasoning will be sounder. S2 doesn’t just rely but is a part of S1-as-language, though, because it’s a phenomena born out (and inside) the language. Whether it’s willfully “consciously” engaged or if it works just because S1 predicts logical thinking concept as appropriate for certain lines of thinking and starts to involve probably doesn’t even matter - it mainly depends on whatever definition of “will” we would like to pick (there are many). LLMs and humans can hypothetically do both just fine, but when it comes to checking, humans currently excel because (I suspect) they have a “wider” language in S1, that doesn’t only include word-concepts but also sensory concepts (like visuospatial thinking). Thus, as I get it, the world models idea.
rhubarbtree 7mo ago
Brilliant insight. The success of LLM reasoning, ie “telling yourself a story”, has greatly increased my belief that humans are actually much less impressive than they seem. I do think it’s mostly pattern matching and a bunch of interacting streams analogous to LLM tokens. Obviously the implementations are different, because nature has to be robust and learn online, but I do not think we are as different from these machines as most people assume. There’s a reason Hofstadter et al. reacted as they did even to the earlier models.
- pixl97 7mo ago
  This is why I also think humans being logical inference machines is mostly not true. We are seemingly capable of it, but there must be some cost that keeps it from being commonly used. While humans did seemingly evolve socially very fast, with the tools we seem to have had for a few hundred thousand years it could have been far faster if there were not some other limitations that are being applied.
  - rhubarbtree 7mo ago
    Agreed. This also explains why maths is so difficult for humans. It doesn't come "naturally" to use, we have to force ourselves to use it and it "makes our head hurt".
ChildOfChaos 7mo ago
I remember reading about this in a book, 'The enigma of reason', basically it was saying that reasoning was exactly that, we decided and then we came up with a reason for what we had decided and usually not the other way around. This is because, the 'reasoning' part of our brain came from evolution when we started to communicate with others, we needed to explain our behaviour. Which is fascinating if you think of the implications of that. In the most part we think we are being logical, but in reality we are pattern matching/impulsive and using our reasoning/logic to come up for excuses for why we have chosen what we had already decided. It explains a lot about the world and why it's so hard to reason with someone, we are assuming the decision came from reason in the first place, which when you look at such peoples choices, makes sense as it's clear it didn't.
bwfan123 7mo ago
> But then again, how often do humans actually reason outside their own “training distribution”? Most human insight happens within well-practiced domains. Humans can produce new concepts and then symbolize them for communication purposes. The meaning of concepts is grounded in operational definitions - in a manner that anyone can understand because they are operational, and can be reproduced in theory by anyone. For example, euclid invented the concepts of a point, angle and line to operationally represent geometry in the real world. These concepts were never "there" to begin with. They were created from scratch to "build" a world-model that helps humans navigate the real world. Euclid went outside his "training distribution" to invent point, angle, and line. Humans have this ability to construct new concepts by interaction with the real world - bringing the "unknown" into the "known" so-to-speak. Animals have this too via evolution, but it is unclear if animals can symbolize their concepts and skills to the extent that humans can.
- perfmode 7mo ago
  > Humans can produce new concepts and then symbolize them for communication purposes. Sure, but the question is how often this actually happens versus how often people are doing something closer to recombination and pattern-matching within familiar territory. The point was about the base rate of genuine novel reasoning in everyday human cognition, and I don't think this addresses that. > Euclid invented the concepts of a point, angle and line to operationally represent geometry in the real world. These concepts were never "there" to begin with. This isn't really true though. Egyptian and Babylonian surveyors were working with geometric concepts long before Euclid. What Euclid did was axiomatize and systematize knowledge that was already in wide practical use. That's a real achievement, but it's closer to "sophisticated refinement within a well-practiced domain" than to reasoning from scratch outside a training distribution. If anything the example supports the parent comment. There's also something off about saying points and lines were "never there." Humans have spatial perception. Geometric intuitions come from embodied experience of edges, boundaries, trajectories. Formalizing those intuitions is real work, but it's not the same as generating something with no prior basis. The deeper issue is you're pointing to one of the most extraordinary intellectual achievements in human history and treating it as representative of human cognition generally. The whole point, drawing on Kahneman, is that most of what we call reasoning is fast associative pattern-matching, and that the slow deliberate stuff is rarer and more error-prone than people assume. The fact that Euclid existed doesn't tell us much about what the other billions of humans are doing cognitively on a Tuesday afternoon.

lich_king 7mo ago

> The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. I've never heard about the Wason selection task, looked it up, and could tell the right answer right away. But I can also tell you why: because I have some familiarity with formal logic and can, in your words, pattern-match the gotcha that "if x then y" is distinct from "if not x then not y". In contrast to you, this doesn't make me believe that people are bad at logic or don't really think. It tells me that people are unfamiliar with "gotcha" formalities introduced by logicians that don't match the everyday use of language. If you added a simple additional to the problem, such as "Note that in this context, 'if' only means that...", most people would almost certainly answer it correctly. Mind you, I'm not arguing that human thinking is necessarily more profound from what what LLMs could ever do. However, judging from the output, LLMs have a tenuous grasp on reality, so I don't think that reductionist arguments along the lines of "humans are just as dumb" are fair. There's a difference that we don't really know how to overcome.

edanm 7mo ago
Agree with much of your comment. Though note that as GP said, on the Wason selection task, people famously do much better when it's framed in a social context. That at least partially undermines your theory that its lack of familiarity with the terminology of formal logic.
- the_mar 7mo ago
  I for the life of me could not solve the <18 example from wikipedia. but the number/color one is super easy
- deleted 7mo ago
  [deleted]
- Ajedi32 7mo ago
  Maybe the social version just creates a context where "if x then y" obviously does not include "if not x then not y". Everyone knows people over the drinking age can drink both alcoholic and non-alcoholic drinks, so you obviously don't have to check the person drinking the soft drink to make sure they aren't an adult.
lugu 7mo ago
Your response contains a performative contradiction: you are asserting that humans are naturally logical while simultaneously committing several logical errors to defend that claim.
- jacquesm 7mo ago
  This comment would be a lot more useful with an enumeration of those logical errors.
  - lugu 7mo ago
    commenter’s specific claim—that adding a note about the definition of "if" would solve the problem—is a moving the goalposts fallacy and a tautology. The comment also suffers from hasty generalization (in their experience the test isn't hard) and special pleading (double standard for LLM and humans).

jonahx 7mo ago

> The story that humans have access to some pure deductive engine and LLMs are just faking it with statistics might be flattering to humans more than it’s accurate. Your point rings true with most human reasoning most of the time. Still, at least some humans do have the capability to run that deductive engine, and it seems to be a key part (though not the only part) of scientific and mathematical reasoning. Even informal experimentation and iteration rest on deductive feedback loops.

Nevermark 7mo ago
The fact that humans can learn to do X, sometimes well, often badly, and while many don’t, strongly supports the conjecture that X is not how they naturally do things. I can perform symbolic calculations too. But most people have limited versions of this skill, and many people who don’t learn to think symbolically have full lives. I think it is fair to say humans don’t naturally think in formal or symbolic reasoning terms. People pattern match, Another clue is humans have to practice things, become familiar with them to reason even somewhat reliable about them. Even if they already learned some formal reasoning. —- Higher level reasoning is always implemented as specific forms of lower order reasoning. There is confusion about substrate processing vs. what higher order processes can be created with that substrate. We can “just” be doing pattern matching from an implementation view, and yet go far “beyond” pattern matching with specific compositions of pattern matching, from a capability view. How else could neurons think? We are “only” neurons. Yet we far surpass the kinds of capabilities neurons have.
- jonahx 7mo ago
  I don't disagree with any of that. My comment was only in relation to the question of human-specific capability that current LLMs may not be able to duplicate. I was not making the value judgments you seem to have read.
mikkupikku 7mo ago
When people do math or rigorous deductive reasoning, are we sure they aren't just pattern matching with a set of carefully chosen interacting patterns that have been refined by ancient philosophers as being useful patterns that produce consistent results when applied in correctly patterned ways?

nextaccountic 7mo ago

> Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. The slow, deliberate, step-by-step stuff is effortful and error-prone, and people avoid it when they can. And even when they do engage it, they’re often confabulating a logical-sounding justification for a conclusion they already reached by other means. Some references on that https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow https://thedecisionlab.com/reference-guide/philosophy/system-1-and-system-2-thinking https://thedecisionlab.com/reference-guide/philosophy/system... System 1 really looks like a LLM (indeed completing a phrase is an example of what it can do, like, "you either die a hero, or you live enough to become the _"). It's largely unconscious and runs all the time, pattern matching on random stuff System 2 is something else and looks like a supervisor system, a higher level stuff that can be consciously directed through your own will But the two systems run at the same time and reinforce each other

drdaeman 7mo ago
In my naive understanding, neither requires any will or consciousness. S1 is “bare” language production, picking words or concepts to say or think by a fancy pattern prediction. There’s no reasoning at this level, just blabbering. However, language by itself weeds out too obvious nonsense purely statistically (some concepts are rarely in the same room), but we may call that “mindlessly” - that’s why even early LLMs produced semi-meaningful texts. S2 is a set of patterns inside the language (“logic”), that biases S1 to produce reasoning-like phrases. Doesn’t require any consciousness or will, just concepts pushing S1 towards a special structure, simply backing one keeps them “in mind” and throws in the mix. I suspect S2 has a spectrum of rigorousness, because one can just throw in some rules (like “if X then Y, not Y therefore not X”) or may do fancier stuff (imposing a larger structure to it all, like formulating and testing a null hypothesis). Either way it all falls down onto S1 for a ultimate decision-making, a sense of what sounds right (allowing us our favorite logical flaws), thus the fancier the rules (patterns of “thought”) the more likely reasoning will be sounder. S2 doesn’t just rely but is a part of S1-as-language, though, because it’s a phenomena born out (and inside) the language. Whether it’s willfully “consciously” engaged or if it works just because S1 predicts logical thinking concept as appropriate for certain lines of thinking and starts to involve probably doesn’t even matter - it mainly depends on whatever definition of “will” we would like to pick (there are many). LLMs and humans can hypothetically do both just fine, but when it comes to checking, humans currently excel because (I suspect) they have a “wider” language in S1, that doesn’t only include word-concepts but also sensory concepts (like visuospatial thinking). Thus, as I get it, the world models idea.

rhubarbtree 7mo ago

Brilliant insight. The success of LLM reasoning, ie “telling yourself a story”, has greatly increased my belief that humans are actually much less impressive than they seem. I do think it’s mostly pattern matching and a bunch of interacting streams analogous to LLM tokens. Obviously the implementations are different, because nature has to be robust and learn online, but I do not think we are as different from these machines as most people assume. There’s a reason Hofstadter et al. reacted as they did even to the earlier models.

pixl97 7mo ago
This is why I also think humans being logical inference machines is mostly not true. We are seemingly capable of it, but there must be some cost that keeps it from being commonly used. While humans did seemingly evolve socially very fast, with the tools we seem to have had for a few hundred thousand years it could have been far faster if there were not some other limitations that are being applied.
- rhubarbtree 7mo ago
  Agreed. This also explains why maths is so difficult for humans. It doesn't come "naturally" to use, we have to force ourselves to use it and it "makes our head hurt".

ChildOfChaos 7mo ago

I remember reading about this in a book, 'The enigma of reason', basically it was saying that reasoning was exactly that, we decided and then we came up with a reason for what we had decided and usually not the other way around. This is because, the 'reasoning' part of our brain came from evolution when we started to communicate with others, we needed to explain our behaviour. Which is fascinating if you think of the implications of that. In the most part we think we are being logical, but in reality we are pattern matching/impulsive and using our reasoning/logic to come up for excuses for why we have chosen what we had already decided. It explains a lot about the world and why it's so hard to reason with someone, we are assuming the decision came from reason in the first place, which when you look at such peoples choices, makes sense as it's clear it didn't.

bwfan123 7mo ago

> But then again, how often do humans actually reason outside their own “training distribution”? Most human insight happens within well-practiced domains. Humans can produce new concepts and then symbolize them for communication purposes. The meaning of concepts is grounded in operational definitions - in a manner that anyone can understand because they are operational, and can be reproduced in theory by anyone. For example, euclid invented the concepts of a point, angle and line to operationally represent geometry in the real world. These concepts were never "there" to begin with. They were created from scratch to "build" a world-model that helps humans navigate the real world. Euclid went outside his "training distribution" to invent point, angle, and line. Humans have this ability to construct new concepts by interaction with the real world - bringing the "unknown" into the "known" so-to-speak. Animals have this too via evolution, but it is unclear if animals can symbolize their concepts and skills to the extent that humans can.

perfmode 7mo ago
> Humans can produce new concepts and then symbolize them for communication purposes. Sure, but the question is how often this actually happens versus how often people are doing something closer to recombination and pattern-matching within familiar territory. The point was about the base rate of genuine novel reasoning in everyday human cognition, and I don't think this addresses that. > Euclid invented the concepts of a point, angle and line to operationally represent geometry in the real world. These concepts were never "there" to begin with. This isn't really true though. Egyptian and Babylonian surveyors were working with geometric concepts long before Euclid. What Euclid did was axiomatize and systematize knowledge that was already in wide practical use. That's a real achievement, but it's closer to "sophisticated refinement within a well-practiced domain" than to reasoning from scratch outside a training distribution. If anything the example supports the parent comment. There's also something off about saying points and lines were "never there." Humans have spatial perception. Geometric intuitions come from embodied experience of edges, boundaries, trajectories. Formalizing those intuitions is real work, but it's not the same as generating something with no prior basis. The deeper issue is you're pointing to one of the most extraordinary intellectual achievements in human history and treating it as representative of human cognition generally. The whole point, drawing on Kahneman, is that most of what we call reasoning is fast associative pattern-matching, and that the slow deliberate stuff is rarer and more error-prone than people assume. The fact that Euclid existed doesn't tell us much about what the other billions of humans are doing cognitively on a Tuesday afternoon.