5 ms·
> But then I started asking myself how can we figure out the limits of its ability to reason Third paragraph. The entire article is based on the premise LLMs a
by gwervc 2y ago
> But then I started asking myself how can we figure out the limits of its ability to reason
Third paragraph. The entire article is based on the premise LLMs are supposed to reason, which is wrong. They don't, they're tools to generate text.
- stavros 2y agoI really hate this reductive, facile, "um akshually" take. If the text that the text-generating tool generates contains reasoning, then the text generation tool can be said to be reasoning, can't it. That's like saying "humans aren't supposed to reason, they're supposed to make sounds with their mouths".
- rambambram 2y ago> If the text that the text-generating tool generates contains reasoning, then the text generation tool can be said to be reasoning, can't it. I don't know... you're still describing a talking parrot here, if you'd ask me.
- stavros 2y agoWhat's the difference between a human and a talking parrot that can answer any question you ask it?
- cj 2y agoThe talking parrot can only answer by repeating something it heard before. Another question you could ask is “What’s the difference between a conversation between 2 people and a conversation between 2 parrots who can answer any question?”
- stavros 2y agoIt doesn't only repeat things it's heard before. Have you ever talked with an LLM?
- throwaway11460 2y agoI had ChatGPT design a programming language for a specific uncommon purpose and then use it. It's creative enough.
- rambambram 2y agoCan any question be answered? As long as any reaction on a question is considered an answer, then I see no difference between a human and a parrot.
- dTal 2y agoI feel the use of the word "parrot" is unintentionally apt, given that parrots were long thought to be mere mimics but were ultimately shown to have (at least the capacity for) real linguistic understanding.
- wbogusz 2y agoI’m not a fan of the talking parrot argument, especially when you’re pointing it at models of scale. The only thing separating a talking parrot and humans is our accuracy in shaping our words to the context in which they’re spoken. Sure it’s easy to liken a low resource model to a talking parrot, the output seems no better than selective repetition of training data. But is that really so different from a baby whose first words are mimics from the environment around them? I would argue that as we learn language we implicitly develop the neural circuitry to continue to improve our lexical outputs, this circuitry being concepts like foresight, reasoning, emotion, logic, etc and that while we can take explicit action to teach these ideas, they naturally develop in isolation as well. I don’t think language models, especially at scale, are much different. They would seem to similarly acquire implicit circuitry like we do as they are exposed to more data. As I see it, the main difference in what exactly that circuitry accomplishes and looks like in final output has more to do with the limited styles of data we can provide and the limitations of fine tuning we can apply on top. Humans would seem to share a lot in common with talking parrots, we just have a lot more capable hardware to select what we repeat.
- rambambram 2y agoWhat if we were talking with each other and the right answer for me would be to kiss you on the cheek? Then what?
- Findecanor 2y agoEven if the generated text contains reasoning, could the LLM understand and apply it?
- stavros 2y agoIf I tell GPT-4 to print something, it understands it needs to check if my printer is turned on first and turn it on if it's not, so, yes? Also, if the generated text contains reasoning, what's your definition of "understanding"? Is it "must be made of the same stuff brains are"?
- RandomLensman 2y agoLLMs fail at so many reasoning tasks (not unlike humans to be fair) that they are either incapable or really poor at reasoning. As far as reasoning machines go, I suspect LLMs will be a dead end. Reasoning here meaning, for example, given a certain situation or issue described being able to answer questions about implications, applications, and outcome of such a situation. In my experience things quickly degenerate into technobabble for non-trivial issues (also not unlike humans).
- stavros 2y agoIf you're contending that LLMs are incapable of reasoning, you're saying that there's no reasoning task that an LLM can do. Is that what you're saying? Because I can easily find an example to prove you wrong.
- RandomLensman 2y agoIt could be that all reasoning displayed is showing existing information - so there would be no reasoning, but that aside, what I meant is being able to reason in any consistent way. Like a machine that only sometimes gets an addition right isn't really capable of addition.
- stavros 2y ago
- rdedev 2y agoAt some point if you need to generate better text you need to start creating a model of how the world works along with some amount of reasoning. The "it's just a token generator" argument fails to get this part. That being said I don't think just scaling LLMs are going to get us AGI but I don't have any real arguments to support that