4 ms·
You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world mo
by prasoonds 2y ago
You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world model which is fundamental to reasoning. That in-turn might mean that the model is reasoning.
Others have argued against the "inner world model" theory and suggested that solving reasoning tasks is merely an extension of the "stochastic parrot" scenario - i.e., they claim that no such world model exists and that the model has rote memorized these reasoning scenarios.
- zarzavat 2y agoWhy not just give it a series of directions e.g. go one step forwards, turn right, etc like a LOGO program, and then ask it if it is back to the start or not. If the series of instructions is randomly generated then this is impossible to solve without a world model.
- prasoonds 2y agoPeople have tried this! And for not-complex problems, the newer models do give the right answer and yet usually fail on the complex tasks. Prompting does help though - if you ask the model to check it's own answer, it can often catch errors it made and improve. But the interesting part is precisely how does the model succeed at all at any spatial reasoning task? Surely it hasn't seen all spatial reasoning tasks in its training set. So, what's the other hypothesis? As you say, people suggest that there could be a world model that is constructed internally by the LLM. However, as you can see right here in this thread that people disagree that you need a world model for solving such tasks.