4 ms·

This article has aged very poorly. It's hard to look at LLMs and argue they don't have reasoning capability.

m00x 4y ago
They don't have any reasoning capability. There is nowhere in their architecture that would give them the ability to reason. All they do is predict what would be the most likely word to go next in the sentence, considering the current text and the input text.
- randomdata 4y ago
  What is reasoning other than predicting what next step is most likely to produce a desired outcome? That's all it feels like the mind is doing. Is there a more formal take?
  - m00x 4y ago
    > Reason is the capacity of consciously applying logic by drawing conclusions from new or existing information, with the aim of seeking the truth. ChatGPT doesn't care what word it puts in front of you, and if it's true or not. Reasoning must have a goal, and current LLMs don't. All it does it repeat what are the most common words that they've seen. It also doesn't think. It's a completely forward process. It's definitely a part of human thinking, but not the reasoning part.
    - randomdata 4y ago
      One of the challenges faced by ChatGPT was in seeing it produce conversational-style output instead of simply the most common words that it saw. I'm not sure your definition really clears up how reasoning or thinking differ from "basic" statistical processes. Granted, perhaps it is not explainable. If it were we would understand it well enough to replicate it.
      - m00x 4y ago
        > One of the challenges faced by ChatGPT was in seeing it produce conversational-style output instead of simply the most common words that it saw. Do you have a source for that? I've implemented GPT from scratch and I don't see anywhere that doesn't just take the current output + input as attention and produces the next word. The loss being if it correctly guessed what would be the next word according to context and position.
        randomdata 4y ago
        There were some papers linked here yesterday. I'm not good at memorizing URLs. Sorry.
        fiso64 4y ago
        ChatGPT was also trained with reinforcement learning to produce outputs human raters preferred
        m00x 4y ago
        It's definitely getting closer in this case, but still chatGPT still look for any truth in its response.
  - akomtu 4y ago
    If GPT is fed all the DNA sequencies, it will confidently continue the AGCCTG sequence, but it won't find the inner principles behind such sequencies, it won't tell us any interesting revelations about them. Will it even notice the concept of codons? Maybe there is a hidden message creatively encoded in these sequences, but GPT will never notice it. Maybe these DNAs really represent a stellar map on a 10 dimensonal manifold, or some other crazy thing, but GPT will never see it. That's because GPT doesn't build a mental model to find structure in the data, and there are infinitely many possible models. Dealing with mental models is what I'd call reasoning, and I believe it's solvable with our tech. The source of such models is the upper abstract mind that deals with ideas, and that's a much harder problem to solve. I'd make a guess that this boundary between rational reasoning mind and the upper abstract mind is the boundary between integer and real numbers.

m00x 4y ago

They don't have any reasoning capability. There is nowhere in their architecture that would give them the ability to reason. All they do is predict what would be the most likely word to go next in the sentence, considering the current text and the input text.

randomdata 4y ago
What is reasoning other than predicting what next step is most likely to produce a desired outcome? That's all it feels like the mind is doing. Is there a more formal take?
- m00x 4y ago
  > Reason is the capacity of consciously applying logic by drawing conclusions from new or existing information, with the aim of seeking the truth. ChatGPT doesn't care what word it puts in front of you, and if it's true or not. Reasoning must have a goal, and current LLMs don't. All it does it repeat what are the most common words that they've seen. It also doesn't think. It's a completely forward process. It's definitely a part of human thinking, but not the reasoning part.
  - randomdata 4y ago
    One of the challenges faced by ChatGPT was in seeing it produce conversational-style output instead of simply the most common words that it saw. I'm not sure your definition really clears up how reasoning or thinking differ from "basic" statistical processes. Granted, perhaps it is not explainable. If it were we would understand it well enough to replicate it.
    - m00x 4y ago
      > One of the challenges faced by ChatGPT was in seeing it produce conversational-style output instead of simply the most common words that it saw. Do you have a source for that? I've implemented GPT from scratch and I don't see anywhere that doesn't just take the current output + input as attention and produces the next word. The loss being if it correctly guessed what would be the next word according to context and position.
      - randomdata 4y ago
        There were some papers linked here yesterday. I'm not good at memorizing URLs. Sorry.
      - fiso64 4y ago
        ChatGPT was also trained with reinforcement learning to produce outputs human raters preferred
        m00x 4y ago
        It's definitely getting closer in this case, but still chatGPT still look for any truth in its response.
- akomtu 4y ago
  If GPT is fed all the DNA sequencies, it will confidently continue the AGCCTG sequence, but it won't find the inner principles behind such sequencies, it won't tell us any interesting revelations about them. Will it even notice the concept of codons? Maybe there is a hidden message creatively encoded in these sequences, but GPT will never notice it. Maybe these DNAs really represent a stellar map on a 10 dimensonal manifold, or some other crazy thing, but GPT will never see it. That's because GPT doesn't build a mental model to find structure in the data, and there are infinitely many possible models. Dealing with mental models is what I'd call reasoning, and I believe it's solvable with our tech. The source of such models is the upper abstract mind that deals with ideas, and that's a much harder problem to solve. I'd make a guess that this boundary between rational reasoning mind and the upper abstract mind is the boundary between integer and real numbers.