3 ms·

Hinton's opinions on LLMs are frankly bonkers. Just because you're famous — and intelligent and successful — doesn't mean you can't be completely wrong. Also:

Hinton's opinions on LLMs are frankly bonkers. Just because you're famous — and intelligent and successful — doesn't mean you can't be completely wrong. Also: what's his rationale? It's no use simply claiming something without evidence. And as far as I (and seemingly most others) can see, there's no such evidence other than that they can sometimes output sentences that happen to be true. But so can Wikipedia — does that mean Wikipedia is reasoning? Also, any form of reasoning in the usual sense of the word would surely require the ability to allocate arbitrary amounts of computation (i.e. thought) to each question. LLMs don't do this — they don't sit and ponder; each token takes exactly the same amount of computation to produce. Once they hit an 'end of text' token, they're done. Even empirically speaking, LLMs' ability to reason can be seen to be nonexistent. Just try asking basic mathematics questions. As soon as you ask anything for which the answer isn't available — practically verbatim — on the web already, it produces intelligent-sounding gibberish. This whole idea that 'LLMs must be able to reason because in order to learn to fake reasoning you must learn to actually reason' is like some kind of inverted no true Scotsman fallacy.

sitkack 2y ago
Does slime mold reason? Yes, Hinton can be wrong, is wrong on many things like his misunderstanding on Chomsky and language. But I also think he has spent thousands of hours testing these systems scientifically. Your last sentence puts a lot of words in peoples mouths. But to continue down that line, fake reasoning and actual reasoning sounds like the Chinese Room. Is that the argument you are making? We don't understand our own mental processes well enough, so I try to not anthropomorphize reasoning and cognition.
- xanderlewis 2y ago
  > Your last sentence puts a lot of words in peoples mouths. Well, it’s the most common sentiment I see on both here and (before I gave up) the AI-centred parts of reddit. It’s not quite the Chinese Room, since LLMs can’t even simulate reasoning very well. So there’s no need to debate the distinction between ‘fake reasoning and actual reasoning’ — there may or may not be a difference, but it’s not the point I’m making. As for Hinton: I’m sure he has. But inventors are often not experts on their own creations/discoveries, and are probably just as prone to FUD and panic in the face of surprising developments as the rest of us. No one predicted that autoregressive transformers would get us this far, least of all the experts whose decades of work lead us to this point.

sitkack 2y ago

Does slime mold reason? Yes, Hinton can be wrong, is wrong on many things like his misunderstanding on Chomsky and language. But I also think he has spent thousands of hours testing these systems scientifically. Your last sentence puts a lot of words in peoples mouths. But to continue down that line, fake reasoning and actual reasoning sounds like the Chinese Room. Is that the argument you are making? We don't understand our own mental processes well enough, so I try to not anthropomorphize reasoning and cognition.

xanderlewis 2y ago
> Your last sentence puts a lot of words in peoples mouths. Well, it’s the most common sentiment I see on both here and (before I gave up) the AI-centred parts of reddit. It’s not quite the Chinese Room, since LLMs can’t even simulate reasoning very well. So there’s no need to debate the distinction between ‘fake reasoning and actual reasoning’ — there may or may not be a difference, but it’s not the point I’m making. As for Hinton: I’m sure he has. But inventors are often not experts on their own creations/discoveries, and are probably just as prone to FUD and panic in the face of surprising developments as the rest of us. No one predicted that autoregressive transformers would get us this far, least of all the experts whose decades of work lead us to this point.