4 ms·
> Large language models (LLMs) are like prediction engines — when given a prompt, they generate a response by predicting what words are likely to come next. As
by agentultra 3y ago
> Large language models (LLMs) are like prediction engines — when given a prompt, they generate a response by predicting what words are likely to come next. As a result, they’ve been extremely capable on language and creative tasks, but weaker in areas like reasoning and math. In order to help solve more complex problems with advanced reasoning and logic capabilities, relying solely on LLM output isn’t enough.
And yet I've heard AI folks argue that LLM's do reasoning. I think it still has a long way to go before we can use inference models, even highly sophisticated ones like LLMs, to predict the proof we would have written.
It will be a very good day when we can dispatch trivial theorems to such a program and expect it will use tactics and inference to prove it for us. In such cases I don't think we'd even care all that much how complicated a proof it generates.
Although I don't think they will get to the level where they will write proofs that we consider, beautiful, and explain the argument in an elegant way; we'll probably still need humans for that for a while.
Neat to read about small steps like this.
- twayt 3y agoI don’t think they’re mutually exclusive. Next word prediction IS reasoning. It cannot do arbitrarily complex reasoning but many people have used the next word prediction mechanism to chain together multiple outputs to produce something akin to reasoning. What definition of reasoning are you operating on?
- TacticalCoder 3y ago> Next word prediction IS reasoning I can write a program in less than 100 lines that can do next work prediction and I guarantee you it's not going to be reasoning. Note that I'm not saying LLMs are or are not reasoning. I'm saying "next word prediction" is not anywhere near sufficient to determine if something is able to reason or not.
- twayt 3y agoAny program you write is encoded reasoning. I’d argue if-then statements are reasoning too. Even if you do write a garbage next word predictor, it would still be reasoning. It’s just a qualitative assessment that it would be good reasoning. Again, what exactly is your definition of reasoning? It seems to be not well defined enough to have a discussion about in this context.
- agentultra 3y agoSemantic reasoning, being able to understand what a symbol means and ascertain truth from expressions (which can also mean manipulating expressions in order to derive that truth). As far as I understand tensors and transformers that's... not what they're doing.
- twayt 3y agoIf you understand transformers, you’d know that they’re doing precisely that. They’re taking a sequence of tokens (symbols), manipulating them (matrix multiplication is ultimately just moving things around and re-weighting - the same operations that you call symbol manipulations can be encoded or at least approximated there) and output a sequence of other tokens (symbols) that make sense to humans. You use the term “ascertain truth” lightly. Unless you’re operating in an axiomatic system or otherwise have access to equipment to query the real world, you can’t really “ascertain truth”. Try using ChatGPT with gpt4 enabled and present it with a novel scenario with well defined rules. That scenario surely isn’t present in its training data but it will able to show signs of making inferences and breaking the problem down. It isn’t just regurgitating memorizing text.
- agentultra 3y agoOh cool, so we can ask it to give us a proof of the Erdős–Gyárfás conjecture? I’ve seen it confidently regurgitate incorrect proofs of linear algebra theorems. I’m just not confident it’s doing the kind of reasoning needed for us to trust that it can prove theorems formally.
- Sharlin 3y agoDo you also deem humans incapable of reasoning unless they can prove the Erdős–Gyárfás conjecture? Like, talk about moving the goalposts!
- twayt 3y agoJust because it makes mistakes on a domain that may not be part of it's data and/or architectural capabilities doesn't mean it can't do what humans consider "reasoning". Once again, I implore you to come up with a working definition of "reasoning" so that we can have a real discussion about this. Many undergraduates also confidently regurgitate incorrect proofs of linear algebra theorems, do you consider them completely lacking in reasoning ability?
- hutzlibu 3y ago"In such cases I don't think we'd even care all that much how complicated a proof it generates." I think a proof is only useful, if you can validate it. If a LLM spits out something very complicated, then it will take a loooong time, before I would trust that.
- Closi 3y agoLLMs can reason, and it’s surprising. I think some people get caught up on the “next word prediction” point, because this is just the mechanism. For the next word prediction to work, the LLM has all sorts of internal representations of the world inside it which is where the capability comes from. Human reasoning probably comes from evolution (genetic survival/replication), and then somehow thought was an emergent behaviour that unexpectedly came from that process. A thinking machine wasn’t designed, it just kind of came to be over millennia. Seems to be kind of the same with AI, but the first example of these emergent behaviours seems to be coming out of the back of building a next-word-guesser. It’s a little unexpected, but a simple framework seems to be allowing a neural net to somehow build representations of the world inside it. GPT is just a next word guesser, but humans are just big piles of cells trying to replicate and not die.
- hackefeller 3y agoDo you think the "next word prediction" argument is so popular because we want to believe our intelligence is more complex than it is?