8 ms·

Can LLMs Reason and Plan?

• greenbit 3y ago
  These things are just statistical language models, aren't they? To the extent that when people reason or plan and then verbalize those plans, they tend to leave certain trajectories through language space, a statistical model could presumably reproduce similar, plausible sounding narratives. Doesn't mean that any kind of agency ever actually thought about those plans though, in the way that a person would.
  • greydius 3y ago
    "plausible sounding" is probably the best two word summary of LLM output. A good one word summary might be "bullshit".
    • Buttons840 3y ago
      I relied on GPT4 to learn linear algebra. I had a text book and GPT4. One difference is the text book was sometimes wrong (and GPT4 is the one that spotted the error in the book). If it is "bullshit", it's very helpful bullshit. When I ask GPT4 to write some code to perform a specific task, and it does so, and the code works and correctly performs the task, is that bullshit? It's true that LLMs are not reliable enough to trust blindly, the same applies to humans.
  • ryanklee 3y ago
    > To the extent that when people reason or plan and then verbalize those plans This is a large and dubious assumption. Most of what humans do require step coordination, but almost none of it is a result of explicit planning or verbalization.
• haltist 3y ago
  LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.
  • sublinear 3y ago
    I would hope the next generation of softwares engineers are not the kind to blindly buy into media hype.
    • dkersten 3y ago
      The next generation (going by some comments I’ve read on HN) are already non-programmers gluing (I assume terrible) LLM-generated code together until it (appears like it) works. Hell, my non-programmer brother recently sent me a message like “can you think of a way to fix this script, ChatGPT isn’t managing” (sends me a badly written AI-generated script)
      • ponderings 3y ago
        > “can you think of a way to fix this script, ChatGPT isn’t managing” haha, who's the glue coder now! edit: I chat with a 13 year old glue coder one time. He couldn't really code at all but he could find the components and was excellent at prompting people for help. He showed many things like map api's interacting with many other things. How long did that take you to put together? uhh 2 hours. My mind was blown.
  • jasfi 3y ago
    What about an LLM writing an algorithm than can play Sudoku? We build things that we ourselves can't do all the time. Edit: I updated my sentence to "an LLM writing an algorithm" to make myself clearer. After reading my own sentence I realized it wasn't, sorry!
    • wildrhythms 3y ago
      Of course we can write an algorithm to solve Sudoku. The commenter specifically is talking about LLMs, not algorithms in general.
      • jasfi 3y ago
        I'm making the point that they're not as limited as they seem at first glance.
• xkcd1963 3y ago
  No.
• deleted 3y ago
  [deleted]
• epups 3y ago
  The article itself is very assertive and makes a lot of generalizations, but if you look at the source of their claims [1] you see that in the first study they are using GPT-3.5 and achieve only a 5% score on a reasoning test that largely relies on spatial intuition - some boxes need to be stacked and unstacked sequentially in a convoluted task. Then, they get criticised so they come up with another paper in which they use GPT-4, which has an improved performance of 30% - a 6-fold increase, which the author describes as "modest". They then decide to change the test to a much harder and more convoluted version, where (surprise!) performance drops down once again [2]. I would also have welcomed a comparison to humans. If we apply this test to 100 humans, can we conclude humans don't reason if only 30 get it right? [1] https://arxiv.org/abs/2206.10498 https://arxiv.org/abs/2206.10498 [2] https://arxiv.org/abs/2305.15771 https://arxiv.org/abs/2305.15771
  • 3abiton 3y ago
    Interesting take. I feel in this discussion, many people are approaching it from the theoretical limitations of LLMs, and you seem the only one taking the experimental approach. Funny enough, many who doubted LLMs capabilities 4 years ago, have come around their emergent capabilities, yet with much skepticism, simply because we still don't understand how these emergent abilties work. I haven't seen a paper comparing the threshold of performance with the LLMs increased capabilities, and what parameter (and their weights) come into play to influence the performance.
  • ryanklee 3y ago
    > would also have welcomed a comparison to humans. Much of the criticism and skepticism around LLMs rests on a double-standard that itself rests on an almost embarrassing lack of understanding of how humans themselves operate.
    • xkcd1963 3y ago
      Absolutely!
• deleted 3y ago
  [deleted]
• jasfi 3y ago
  LLMs can write a plan, and they can write about their reasoning. The question is, are they good enough? We may need another generation of LLMs that are more competent.
  • BoorishBears 3y ago
    Actually LLMs can write a better plan than the author: The prompt format made absolutely no sense considering they had decided to translate away from the PDDL to arbitrary natural language. So to avoid triggering their overused Clever Hans defense, I fed GPT 4 their prompt with only the instruction: "Think critically about how we could represent these rules in a way that's clearer to an LLM. Lean into using coding style identifiers where possible, and JSON formatting" (Hopefully the author won't claim telling a model to generate JSON is secretly telling it how to move blocks!) In a fresh context window at 0 temp and gpt-4-0613 I entered the JSON formatted rules it generated, along with the instruction: Return a JSON array of [{[<valid actions>],<internal thought>, <action>}] that results in goal state ... the resulting answer solved their failed few-shot example with zero-shot _ Also funny blunder: their chain of thought example generates thoughts... after the action. Surely the author understands a transformer model can't rely on an ungenerated token to affect the action taken? Edit: Also their "disguised difficulty" version was wrong? > To perform Attack action, the following facts need to be true: Province object, Planet object, Harmony. The single word "Harmony" somehow replaced replaces "Your hands are empty"? Before that they claim it's meant to be a 1:1 replacement of entities, but the actual disguised versions are not longer valid instructions
    • jasfi 3y ago
      I'm not surprised, GPT-4 really is very good, you just have to prompt it well enough. Often you don't even have to be a prompt guru.
• stormfather 3y ago
  One at least has to admit that in many cases they already do an effective job at simulating planning. I'm not convinced there's a practical distinction between that and "truly planning".
• famouswaffles 3y ago
  A test that doesn't properly inform you someone/thing's abilities is a rubbish test. From their paper, you would think being able to pilot robots to handle chores or stack objects or whatever would be well beyond LLMs. But it's not. https://tidybot.cs.princeton.edu/ https://tidybot.cs.princeton.edu/ https://innermonologue.github.io/ https://innermonologue.github.io/ Anyone who wants LLMs to plan and is actually interested in teasing the extent of those abilities knows how to structure planning requests. It's extra funny because humans can't actually generate plans the way he tests LLMs to either. Also Seeing output from GPT that demonstrates intelligence, reasoning, or whatever, and saying it is not real reasoning/Intelligence etc, is like looking at a plane soar and saying that the plane is fake flying. And this isn't a nature versus artificial thing either. The origin point is entirely arbitrary. You could just as easily move the origin to Bees and say, "oh, birds aren't really flying". You could move it to planes and say, "oh, helicopters aren't really flying." It's a very meaningless statement. Internal processes are entirely irrelevant.
  • haukurb 3y ago
    Their example prompt is so bad that I'm split between them being wildly incompetent at understanding how transformers or autoregressiveness works (and general LLM dynamics) or they are deliberately obfuscating the task and prompt representation in bad faith. If one really wants to know if LLMs "are capable of planning", one should keep an open mind for how planning behavior can manifest in textual form, then actually try to find any manifestations. Imposing one's view of how planning should look like is bad science. All they've proved is that their task format sucks for current generation LLMs.
• jamesblonde 3y ago
  I tend to live the "theory-of-mind" reasoning tasks as a test for the ability of a LLM to reason about the "state" of other minds. It's challenging enough for humans. https://www.hopsworks.ai/dictionary/theory-of-mind-tasks https://www.hopsworks.ai/dictionary/theory-of-mind-tasks
• Communitivity 3y ago
  "Plan" means a lot of things. There has been some research in applying GPT-4 to Hierarchical Task Networks (HTN), one means of doing computerized semi-automated/automated planning of a complex task as a tree of less and less complex tasks [1]. There are other types of planning. Automated planning works better as there are more defined the tasks in a plan, less ambiguity in dependencies, more separate between the tasks. The OP article touches on that, noting LLMs are good at extracting planning knowledge but not good in their experience at creating executable plans. This is why I think the hybrid approach is best, using an LLM to inform and tweak other planning tools in order to create an executable plan. [1] https://github.com/DaemonIB/GPT-HTN-Planner https://github.com/DaemonIB/GPT-HTN-Planner
• fnordpiglet 3y ago
  The obsession with whether LLMs can out perform classical AI, algorithms, solvers, and other optimizers fascinates me. Intuitively of course they aren’t doing something similar to a solver. They will never play chess better than the best for purpose chess playing system. They will never reason better than a classical reasoning system. That misses the point entirely. They are a fascinating augmentation to existing capabilities that fills an ability to operate and do some approximation to reason and plan in an abstract semantic space, using natural language in a semantic way, to perform an abductive style of “reasoning” that has alluded us to date. Need to play chess? Use a chess playing system. But you can glue together special purpose systems for planning, reasoning, optimizing, calculating, etc, with an LLM in ways that are much more flexible and adaptable in a real world context than we’ve ever been able to achieve. That’s the magic of them. The fact they can do pretty alright at some of these tasks is interesting and shows how powerful language is that embedded in its semantic structure is an awful lot of what you need to reason, plan, etc. But beyond the academic question, why would you not just use a provably optimal algorithm for a specific task? Interestingly, given a set of operations in context, LLMs are pretty good at identifying a situation where an operation is applicable and making the API call, and as we learn how to embed them, they’ll be able to defer to and be deferred to more accurately.
  • Difwif 3y ago
    People are interested because of the possibility of an architecture for AGI. A system capable of generalizing to all types of problems and also being able to leverage everything you said as a tool (or even build the tool) is extremely valuable. We're not even close to this result yet but LLMs seem the closest.
    • fnordpiglet 3y ago
      I agree, but I think there’s an overly large emphasis on picking at LLMs (in)ability to reason in a structured way independent of its semantic language expectation model. It’s usually used as a “see LLM sucks” while the other side holds to “see it can do everything.” Neither is right. LLMs are amazing, but they can’t do everything, even if they can seem to do many things most of the time. While interesting to investigate the boundaries, I think the passion should be more directed towards “how do we wire the piece together and for what and where”
• kat_rebelo 3y ago
  LLM's only work on the data they have been trained on so all outputs are merely based on information that has already been written about by a human. Furthermore, LLM's do not truly "understand" even first order causal relationships, meaning whatever it plans will have no foresight to evaluate how a plan it generates will impact downstream components of a complex system. LLM's live in "the world that has been written about", not the real world, and thus cannot formulate new ideas or hypothesis other than by accident. This, coupled with the lack of an ontological system for evaluating the validity of the statements it makes about a complex system, and, its lack of causal reasoning, means they cannot effectively plan. I've worked on research related to causality that used LLM's (admittedly, pre ChatGPT and using much smaller models) and it was not uncommon to see extremely bogus causal relationships inferred such as "rising cost of living in NYC caused a flood in Argentina".
  • theonlybutlet 3y ago
    Neither can a human. Unless the human subconsciously re-evaluates their output and refines it before speaking, much like running the LLM output through the model again. Or if they've learnt it through past experience reinforcing that pathway much like LLM learning and adjusting weights to factor that in so it would impact its output in future.
• deleted 3y ago
  [deleted]