4 ms·

> LLMs have "dead reasoning zones" — areas in their weights where logic doesn't work. Humans have dead knowledge zones (things we don't know), but not dead reas

> LLMs have "dead reasoning zones" — areas in their weights where logic doesn't work. Humans have dead knowledge zones (things we don't know), but not dead reasoning zones. blank stare

mjburgess 1y ago
We have dead-zones in adductive reasoning, not in induction or deduction. Almost all failures of reasoning in people are in abducing what model describes the situation at hand. eg., we can apply the rule, "-A cannot follow from A", etc. regardless of the A eg., we always know that if the number of apples is 2, then it cannot be any of "all numbers without 2" -- which quantifies over all numbers You will not find a "gap" for a given number, whereas with LLMs, gaps of this kind are common
- rel_ic 1y ago
  > we can apply the rule, "-A cannot follow from A", etc. regardless of the A You can't think of any domains where we are unable to apply this rule? I feel like I'm surrounded by people claiming "A, therefore -A!!" And if I'm one of them, and this were a reasoning dead-zone for me, I wouldn't be able to tell!
  - mjburgess 1y ago
    That's an abductive failure to recognise that something is A, and something else is not-A I dont see cases where people recognise the contradiction and then perform it.
    - virgilp 1y ago
      How can you know? One could argue that the entire phenomenon of cognitive dissonance is "people (internally) recognize the contradiction and then perform it"
    - rel_ic 1y ago
      People who know alcohol is bad for them and don't want to keep being drunks but keep drinking, people who believe phones are bad for their kids but still buy them, people who understand AI will significantly degrade the environment if it becomes ubiquitous but still work to help it become ubiquitous... Mathematicians who publish proofs that are later proven inconsistent! I suspect we have fundamentally different views of how humans work. I see our behavior and beliefs as _mostly_ irrational, with only a few "reasoning live-zones" where, with great effort, we can achieve logical thought.
      - kandel 1y ago
        Well most of your examples are about failure to act based on reasoning rather than failure to reason, except the mathematics one which is unfair as research mathematics is a very hard task - either subtle errors, reasoning in new fields, or extremely long chains of reasoning.

> LLMs have "dead reasoning zones" — areas in their weights where logic doesn't work. Humans have dead knowledge zones (things we don't know), but not dead reasoning zones. blank stare

mjburgess 1y ago

We have dead-zones in adductive reasoning, not in induction or deduction. Almost all failures of reasoning in people are in abducing what model describes the situation at hand. eg., we can apply the rule, "-A cannot follow from A", etc. regardless of the A eg., we always know that if the number of apples is 2, then it cannot be any of "all numbers without 2" -- which quantifies over all numbers You will not find a "gap" for a given number, whereas with LLMs, gaps of this kind are common

rel_ic 1y ago
> we can apply the rule, "-A cannot follow from A", etc. regardless of the A You can't think of any domains where we are unable to apply this rule? I feel like I'm surrounded by people claiming "A, therefore -A!!" And if I'm one of them, and this were a reasoning dead-zone for me, I wouldn't be able to tell!
- mjburgess 1y ago
  That's an abductive failure to recognise that something is A, and something else is not-A I dont see cases where people recognise the contradiction and then perform it.
  - virgilp 1y ago
    How can you know? One could argue that the entire phenomenon of cognitive dissonance is "people (internally) recognize the contradiction and then perform it"
  - rel_ic 1y ago
    People who know alcohol is bad for them and don't want to keep being drunks but keep drinking, people who believe phones are bad for their kids but still buy them, people who understand AI will significantly degrade the environment if it becomes ubiquitous but still work to help it become ubiquitous... Mathematicians who publish proofs that are later proven inconsistent! I suspect we have fundamentally different views of how humans work. I see our behavior and beliefs as _mostly_ irrational, with only a few "reasoning live-zones" where, with great effort, we can achieve logical thought.
    - kandel 1y ago
      Well most of your examples are about failure to act based on reasoning rather than failure to reason, except the mathematics one which is unfair as research mathematics is a very hard task - either subtle errors, reasoning in new fields, or extremely long chains of reasoning.