6 ms·

Isn't reasoning a little strong word? I mean finding evidence of any reasoning is the first point and I haven't seen anything that points to that. (note that I

by hggigg 2y ago

Isn't reasoning a little strong word? I mean finding evidence of any reasoning is the first point and I haven't seen anything that points to that. (note that I mean structured reasoning to solve new problems here, not just the correct answer and steps for existing problems).

ben_w 2y ago
Depends on which definition of "reasoning" you prefer. If you need reasoning to be a "conscious" process, then we've got another problem because there's about 40* different meanings of that, too. * Wikipedia cites: Vimal RL, Sansthana DA (2010). "On the Quest of Defining Consciousness" (PDF). Mind and Matter. 8 (1): 93–121 but then gives a dead link, so I couldn't check it
ziofill 2y ago
I agree. Reasoning should be robust to perturbations of the way in which the same problem is cast, but LLMs have problems with that.
- krisoft 2y ago
  Does human reasoning survive that test? My reasoning for example is roboust to some extent but you can easily confuse me with appropriately (or innapriately) worded questions. Not even talking about if you grab me by the ankle and swing me around with 10g acceleration i won’t even be able to answer the simplest questions. So clearly my reasoning is highly sensitive to certain perturbations.
  - d4mi3n 2y ago
    Not disagreeing with you, but a point to consider for reasoning is that we consider it a key cornerstone of sapience and sentience. I think one could argue that a LLM could be considered sapient (able to solve problems) in a general sense, but probably not sentient (able to sustain undifferentiated consciousness, continue to absorb and apply information, etc). Part of the difficulty of these conversations, though, is many of these definitions were made before our math was advanced enough to approximate any of these criteria. We humans have also been notorious for excluding other things that could be considered sapience (e.g. elephants, dolphins, corvids, etc). In all cases, I think this is going to be a difficult topic to find consensus on until we better understand how our biology results in the emergent properties of our own sapience.
  - ziofill 2y ago
    I see what you are saying. I guess it all depends on how complex one makes that test. But arguably vanilla LLMs (0-shot, no CoT, no rewrite memory) are much worse than humans at this.
swatcoder 2y ago
The field of AI likes to borrow pop psychology/philosophy language and use it for its own narrow jargon. It's been doing that for almost a century now. That's how it builds enthusiasm and raises funds, and also how it shoots itself in the foot and invites the AI winters that hollow it out again. The field named itself that very way to in response to a early 20th century boom of interest (money) in "intelligence" as something quantifiable (and therefore potentially computable). Tons of money was flying around based on the idea that people could be quantitatively measured for work-suitability, and researchers in computation saw opportunity to get research funds to explore that in their own budding endeavor, so they adopted the language and made a very long bet that they might eventually figure it out. At this point, there is largely consensus within the field that current models perform a process that the the field has collectively decided to call "reasoning" as part of their tradition of repurposing psychology and philosophy language as jargon. There's no putting that cat back in the bag. It's a term of art within the field now. Frustratingly, it's easier to just accept that this is just a novel sense of the word than to try to reconcile it with senses that we might be familiar with from lay usage or from other fields.
- d4mi3n 2y ago
  I find it apropos that this topic (disagreement about if LLMs can "reason") boils down to being an issue of one of the hard problems in computer science: naming things.
  - ben_w 2y ago
    And the linked topic, off by one errors. One could also the hallucinated citations are null pointers. At least cache invalidation is easier: the users can backtrack up a conversation as far as they need to in order to un-cache things from the context window.
- astrange 2y ago
  > The field of AI likes to borrow pop psychology/philosophy language and use it for its own narrow jargon. They also like to invent metaphors for how the brain works, and then forget that it's a metaphor and decide that humans either literally have one, that they should program one into AIs, or that AIs must have invented one on their own. This is why you keep seeing them talk about "world models", which probably don't actually exist and which people don't actually need in order to live their life.
krisoft 2y ago
> finding evidence of any reasoning is the first point Happy to oblige. Oxford Languages dictionary defines “reasoning” as “the action of thinking about something in a logical, sensible way.” This was my prompt to chatgpt: “Hey chatgpt! On hacker news user hggigg said this: Isn't reasoning a little strong word? I mean finding evidence of any reasoning is the first point and I haven't seen anything that points to that. Based on this comment what do you think is the likely attitude of hggigg about large language models? (Answer with a single sentence)” This was the response: “hggigg likely expresses skepticism about large language models' ability to genuinely reason.” That sounds like reasoning to me. It was an action of thinking about something in a logical, sensible way. The something (a snippet of your comment) was novel. I don’t know what you mean by “structured”. Does it provide you evidence of reasoning for you? (If not, what would?)
- swatcoder 2y ago
  > It was an action of thinking about something in a logical, sensible way If only it was so simple! Empirically, you streamed a short text into a black box system and received a short text out. The output text was related and coherent, and -- after the fact -- you assessed it to be logically, sensibly applicable given the input. On your way to using that to exemplify "reasoning" you took for granted that there was "thinking", and that the "thinking" was both "logical" and "sensible". Any number of systems might deliver the empirical result you measured, and many of them would involve none of that. LLM's are sophisticated and highly capable, but the discussion about what language to use for what they do isn't nearly so simple as you suggest.
  - wbl 2y ago
    And how do I know you think?
    - deleted 2y ago
      [deleted]
    - swatcoder 2y ago
      Are you sure you do? You're just responding to a block of text on an internet forum with trivial signup flow, known to be populated by bots of greater or less sophistication, as everywhere else on the internet.

by hggigg 2y ago

ben_w 2y ago

Depends on which definition of "reasoning" you prefer. If you need reasoning to be a "conscious" process, then we've got another problem because there's about 40* different meanings of that, too. * Wikipedia cites: Vimal RL, Sansthana DA (2010). "On the Quest of Defining Consciousness" (PDF). Mind and Matter. 8 (1): 93–121 but then gives a dead link, so I couldn't check it

ziofill 2y ago

I agree. Reasoning should be robust to perturbations of the way in which the same problem is cast, but LLMs have problems with that.

krisoft 2y ago
Does human reasoning survive that test? My reasoning for example is roboust to some extent but you can easily confuse me with appropriately (or innapriately) worded questions. Not even talking about if you grab me by the ankle and swing me around with 10g acceleration i won’t even be able to answer the simplest questions. So clearly my reasoning is highly sensitive to certain perturbations.
- d4mi3n 2y ago
  Not disagreeing with you, but a point to consider for reasoning is that we consider it a key cornerstone of sapience and sentience. I think one could argue that a LLM could be considered sapient (able to solve problems) in a general sense, but probably not sentient (able to sustain undifferentiated consciousness, continue to absorb and apply information, etc). Part of the difficulty of these conversations, though, is many of these definitions were made before our math was advanced enough to approximate any of these criteria. We humans have also been notorious for excluding other things that could be considered sapience (e.g. elephants, dolphins, corvids, etc). In all cases, I think this is going to be a difficult topic to find consensus on until we better understand how our biology results in the emergent properties of our own sapience.
- ziofill 2y ago
  I see what you are saying. I guess it all depends on how complex one makes that test. But arguably vanilla LLMs (0-shot, no CoT, no rewrite memory) are much worse than humans at this.

swatcoder 2y ago

The field of AI likes to borrow pop psychology/philosophy language and use it for its own narrow jargon. It's been doing that for almost a century now. That's how it builds enthusiasm and raises funds, and also how it shoots itself in the foot and invites the AI winters that hollow it out again. The field named itself that very way to in response to a early 20th century boom of interest (money) in "intelligence" as something quantifiable (and therefore potentially computable). Tons of money was flying around based on the idea that people could be quantitatively measured for work-suitability, and researchers in computation saw opportunity to get research funds to explore that in their own budding endeavor, so they adopted the language and made a very long bet that they might eventually figure it out. At this point, there is largely consensus within the field that current models perform a process that the the field has collectively decided to call "reasoning" as part of their tradition of repurposing psychology and philosophy language as jargon. There's no putting that cat back in the bag. It's a term of art within the field now. Frustratingly, it's easier to just accept that this is just a novel sense of the word than to try to reconcile it with senses that we might be familiar with from lay usage or from other fields.

d4mi3n 2y ago
I find it apropos that this topic (disagreement about if LLMs can "reason") boils down to being an issue of one of the hard problems in computer science: naming things.
- ben_w 2y ago
  And the linked topic, off by one errors. One could also the hallucinated citations are null pointers. At least cache invalidation is easier: the users can backtrack up a conversation as far as they need to in order to un-cache things from the context window.
astrange 2y ago
> The field of AI likes to borrow pop psychology/philosophy language and use it for its own narrow jargon. They also like to invent metaphors for how the brain works, and then forget that it's a metaphor and decide that humans either literally have one, that they should program one into AIs, or that AIs must have invented one on their own. This is why you keep seeing them talk about "world models", which probably don't actually exist and which people don't actually need in order to live their life.

krisoft 2y ago

> finding evidence of any reasoning is the first point Happy to oblige. Oxford Languages dictionary defines “reasoning” as “the action of thinking about something in a logical, sensible way.” This was my prompt to chatgpt: “Hey chatgpt! On hacker news user hggigg said this: Isn't reasoning a little strong word? I mean finding evidence of any reasoning is the first point and I haven't seen anything that points to that. Based on this comment what do you think is the likely attitude of hggigg about large language models? (Answer with a single sentence)” This was the response: “hggigg likely expresses skepticism about large language models' ability to genuinely reason.” That sounds like reasoning to me. It was an action of thinking about something in a logical, sensible way. The something (a snippet of your comment) was novel. I don’t know what you mean by “structured”. Does it provide you evidence of reasoning for you? (If not, what would?)

swatcoder 2y ago
> It was an action of thinking about something in a logical, sensible way If only it was so simple! Empirically, you streamed a short text into a black box system and received a short text out. The output text was related and coherent, and -- after the fact -- you assessed it to be logically, sensibly applicable given the input. On your way to using that to exemplify "reasoning" you took for granted that there was "thinking", and that the "thinking" was both "logical" and "sensible". Any number of systems might deliver the empirical result you measured, and many of them would involve none of that. LLM's are sophisticated and highly capable, but the discussion about what language to use for what they do isn't nearly so simple as you suggest.
- wbl 2y ago
  And how do I know you think?
  - deleted 2y ago
    [deleted]
  - swatcoder 2y ago
    Are you sure you do? You're just responding to a block of text on an internet forum with trivial signup flow, known to be populated by bots of greater or less sophistication, as everywhere else on the internet.