4 ms·

> It’s clear transformers can’t understand either case. They’re not architecturally designed to. What does this follow from? > The emergent behavior of appear

• PaulDavisThe1st 3y ago
  Well, I see where you're trying to with this, but you can't get away with this sort of sleight of hand in order to dismiss some of the central questions of cognitive science as if they've already been answered by LLMs. One of those central questions is: how does (and does) the human brain perform reasoning? We know the brain is capable of all kinds of autonomous behavior that does not require reasoning, but more of "if this then that" (with a huge boatload of "this" and "that" being multi-dimensional, multi-variate). But that doesn't help to explain what happens when we actually have the experience of reasoning about a problem, and for once I am not talking about qualia. What is the brain doing when a person is thinking "hard" about how to solve a hitherto unknown problem or situation? The central problem of LLMs (or at least, a central problem) is that they only model the autonomous aspects of speech behavior (which may indeed make up more of speech behavior than we might have suspected before them). That still leaves the sort of speech behavior that emerges from intense, focused, concentrated thought, particular in response to a novel question or situation.