3 ms·

> it proves nothing about reason, which LLMs have clearly shown needs to be distinguished from consciousness. Uh, they have? Are you saying they know how to r

> it proves nothing about reason, which LLMs have clearly shown needs to be distinguished from consciousness. Uh, they have? Are you saying they know how to reason? Because if so, why is it that when I give a state of the art model documentation lacking examples for a new library and ask it to write something, it cannot even begin to do that, even if the documentation is in the training data? A model that can reason should be able to understand the documentation and create novel examples. It cannot. This happened to me just the other day. If the model can reason, examples of the language, which it has, and the expository documentation should have been sufficient. Instead, the model repeatedly inserted bullshitted code in the style of the language I wanted, but with library calls and names based on a version of the library for another language. This is evidence of reasoning ability? Claude Sonnet 3.7 and Gemini Pro both exhibited this behavior last week. I think this technology is fundamentally the same as it has been since GPT2

smithkl42 1y ago
Absolutely LLMs can reason. There are limitations on their ability to reason, as you and everyone else has discovered. But they can absolutely reason about both concepts and the physical world in ways that, say, animals can't - even though presumably animals have at least some sort of self-consciousness and LLM's do not.
slippybit 1y ago
> A model that can reason should be able to understand the documentation and create novel examples. It cannot. That's due to limitations imposed for "security". "Here's a new X, do Y with it" can result in holes bigger and more complex than anyone can currently handle "in time". It's not about "abilities" with LLMs for now, but about functions that work within the range of edge cases, sometimes including them, some other times not. You could still guide it to fulfill the task, though. It just cannot be allowed to do it on it's own but since just "forbidding" an LLM to do something is about as effective as doing that to a child with mischievous older brothers, the only ways to actually do it result in "bullshitted" code and "hallucinations". If I understood the problem correctly, that is.

smithkl42 1y ago

Absolutely LLMs can reason. There are limitations on their ability to reason, as you and everyone else has discovered. But they can absolutely reason about both concepts and the physical world in ways that, say, animals can't - even though presumably animals have at least some sort of self-consciousness and LLM's do not.

slippybit 1y ago

> A model that can reason should be able to understand the documentation and create novel examples. It cannot. That's due to limitations imposed for "security". "Here's a new X, do Y with it" can result in holes bigger and more complex than anyone can currently handle "in time". It's not about "abilities" with LLMs for now, but about functions that work within the range of edge cases, sometimes including them, some other times not. You could still guide it to fulfill the task, though. It just cannot be allowed to do it on it's own but since just "forbidding" an LLM to do something is about as effective as doing that to a child with mischievous older brothers, the only ways to actually do it result in "bullshitted" code and "hallucinations". If I understood the problem correctly, that is.