4 ms·
Maybe, but the fact remains that LLMs don’t grok semantics and thus do not reason.
by eternalban 4y ago
Maybe, but the fact remains that LLMs don’t grok semantics and thus do not reason.
- aftbit 4y agoRight... but do you? Or is this all just a game of syntax that _feels_ like semantics from the inside?
- danbruc 4y agoCould be. Does not even have to feel like anything, this could be a independent additional component. This goes into the direction of the Chinese room, how do you figure out if someone understands something? I would say you ask questions and judge by the answers.
- mcguire 4y agoIf the answers include hallucinations of basic facts, there is likely something seriously wrong with the Chinese room's thinking somewhere.
- eternalban 4y agoTwo questions that I keep asking myself: - What distinguishes thinking and computing? - What is more fundamental? Form or meaning? > is this all just a game of syntax that _feels_ like semantics from the inside? The point is that in case of LLMs, there is no mystery here: it is syntactic templates (x) statistics. But do we know that 'syntax' is all there is to our mental reasoning? We don't know this (yet). I have flirted with the notion - heretical for me since I 'believe' in primacy of meaning, not form - and can kinda squint and see meaning possibly being a 'virtual' phenomena and possibly simply a 'narrative' of 'arcs' connecting 'facts' mapped to a syntactic template. That meaning is simply a process. [So, not even "42" ..]
- danbruc 4y agoWhat does grok semantics mean, how would you test this? I am not trying to argue that language model are close to human level reasoning or whatever, but it is not obvious to me that a language model of some sort is fundamentally unable to to achieve this.
- AlotOfReading 4y agoThe question of whether language models can "reason" ultimately comes back to the chinese room argument. There aren't any universally accepted resolutions to that argument either, nor whether those limitations (if they exist) are fundamental. However, the current incarnation of LLMs like ChatGPT fail at straightforward reasoning around things like categorizing sailfish and mathematical proofs. I'm incredibly impressed at how powerful the simplistic methods they use are, but the deficiencies make it clear that it's not abstractly reasoning from definitions in the same way humans are able to.
- mcguire 4y agoA fun thing to do: Ask ChatGPT a question, and ask for references. Then try to track down the references. I just asked a simple question about the Thirty Years War, and with the answer it gave me two valid references (including C.V. Wedgewood, yay), and one that was completely made up. As far as I understand, LLMs have no model of reality to base results on; they're just an enormously complex statistical model of "what is the next word in the series".
- vannevar 4y agoIt's also pretty easy to lead ChatGPT into giving contradictory answers to the same question, just by suggesting the answer in the question.