10 ms·
My impression is that LLMs “pattern-match” on a less abstract level than general-purpose reasoning requires. They capture a large number of typical reasoning pa
by layer8 2y ago
My impression is that LLMs “pattern-match” on a less abstract level than general-purpose reasoning requires. They capture a large number of typical reasoning patterns through their training, but it is not sufficiently decoupled, or generalized, from what the reasoning is about in each of the concrete instances that occur in the training data. As a result, the apparent reasoning capability that LLMs exhibit significantly depends on what they are asked to reason about, and even depends on representational aspects like the sentence patterns used in the query. LLMs seem to be largely unable to symbolically abstract (as opposed to interpolate) from what is exemplified in the training data.
- arketyp 2y agoI agree. Although I wonder sometimes how much "about" is involved in my own abstract reasoning.
- astromaniak 2y agoFor some reasons LLMs get a lot of attention. But.. while simplicity is great it has limits. To make model reason you have to put it in a loop with fallbacks. It has to try possibilities and fallback from false branches. Which can be done on a level higher. This can be either algorithm, another model, or another thread in the same model. To some degree it can be done by prompting in the same thread. Like asking LLMs to first print high level algorithm and then do it step by step.
- layer8 2y agoIteration is important, but I don’t think that it can substantively compensate for the abstraction limitations outlined in the GP comment.
- refulgentis 2y ago> To make model reason you have to put it in a loop with fallbacks Source? TFA, i.e. the thing we're commenting on, tried to, and seems to, show the opposite
- cma 2y agoThe comment may just be pattern matching on the topic, not reasoning about TFA.
- astromaniak 2y agoWhen the task, or part of it, is np complete there is no way around. Model has to try all options till it find working one. In a loop. And this can be multi-step with partial fallback. That's how humans are thinking. They can see only to some depth. They may first determine promising directions. Select one, go dipper. Fallback if it doesn't work. Pattern matching mentioned is simplest one step solution. LLMs are doing it with no problems.
- llm_trw 2y agoLLMs get a lot of attention because they were the first architecture that could scale to a trillion parameters while still improving with every added parameter.
- Salgat 2y agoLLMs already do this. Their many wide layers allow for this, and as a final fallback, their output adjusts based on every token they generate (it's not all decided at once). All your statement really means is a vague "well it should do it more!" which yeah, is the goal of each iteration of GPT etc.
- kgeist 2y agoAren't humans also prone to this? Take the classic trick question, for example: "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?" Most people give a wrong answer because they, too, "pattern match".
- mecsred 2y agoI think you could reasonably describe that as "pattern matching at the wrong level". If you tell most people the first answer is wrong they will go up a level or two and work out the correct answer.
- nwhitehead 2y agoYou would think so... I asked this question in a college-level class with clickers. For the initial question I told them, "This is a trick question, your first answer might not be right". Still less than 10% of students got the right answer.
- adamisom 2y agoStudents are more arrogant than LLMs (but still more generally intelligent)
- roenxi 2y agoThis idea that students are "more generally intelligent" requires specific arguments to support. ChatGPT alone has a vast breadth of knowledge that is impossible for a human to keep up with as well as superior skills to a student in any number of fields. I can find evidence that it has an IQ of around 124 [0] which is going to stretch most students (although in fairness the same article also speculates an IQ of 0). Students can keep ahead of it with training in specific fields and it has a few weaknesses in specific skills but I think someone could make a reasonable claim that ChatGPT has superior general intelligence. [0] https://medium.com/@soltrinox/the-i-q-of-gpt4-is-124-approx-2a29b7e5821e https://medium.com/@soltrinox/the-i-q-of-gpt4-is-124-approx-...
- slashdave 2y agoGiven that LLMs are trained on tokens and not symbols, that is, they have no context or world model, I would say that LLMs cannot symbolically abstract, given that they have neither a mechanism nor the data to do so.
- Salgat 2y agoI wonder if reasoning is intrinsically tied to the need for a model to be able to perform well with extremely limited training data. Current LLMs require the sum total of all human knowledge to work correctly (even a few decades ago, there was no where near enough data to train one to be useful, regardless of available computing), meanwhile humans only need access to a few books and conversations with uneducated people growing up to achieve brilliant ideas.
- ekianjo 2y ago> Current LLMs require the sum total of all human knowledge to work correctly the total sum of human knowledge is not found in the digital world.
- Salgat 2y agoI'd argue the vast majority of it is, from newspaper articles spanning back centuries to the vast majority of books written, down to every conversation someone has had on public social media websites such as Reddit.
- xkcd1963 2y agoA unicellular organism alone has already more variables to deal with than an LLM, now multiply that by the amount of cells a brain has.