3 ms·
My personal take is that "inside" much of language (especially written language) is a kernel of predicate logic. Most pieces of text end up taking a proposition
by cmrdporcupine 4y ago
My personal take is that "inside" much of language (especially written language) is a kernel of predicate logic. Most pieces of text end up taking a propositional form, eventually. So if you train the right kind of learning machine on an absolutely huge pile of text, it will eventually "learn" some amount of logical constructions, and start producing things that at least look like logical reasoning, in response to logical queries.
Same with training it on programming languages, which have a similar underlying form.
What I've found though is that this "reasoning" starts to get exposed as shallow fairly quickly. It can only "backtrack" through logical relationships a couple levels deep. E.g. I give GPT4 a Rust program with a borrow checker problem and ask it to "fix" it and watch it flail around in circles for 3 or 4 tries before just producing complete nonsense because it gets lost.
It looks like human intelligence like you say, but it just falls over after a few short steps.
Yes, most humans will do the same thing. Except unlike an LLM, humans are able to sense when they get lost and attempt to sometimes retrace or correct their steps. But I think there's something intrinsically missing in LLMs that prevents them from doing that -- they don't "know" why they arrived at some step, and don't seem to be able to retrace and fix. Ask them to fix something they get wrong, and they start to degrade in quality quite quickly, I find. Self-awareness/consciousness may be the kernel of that missing piece.
I think maybe these systems might be more useful once they're tied to an actual rigorous logical deduction system, and "taught" how to use them. Let the LLM "read" the prompt, recognize that it needs to do some logical constraint solving or deduction, etc., create the right terms or horne clauses or whatever in that subsystem, query it, and then let the LLM in turn produce the textual response.