4 ms·

You may be mistaking reasoning ability with sheer volume of more patterns to match on. You can only reason when you have awareness within which to reason, whic

You may be mistaking reasoning ability with sheer volume of more patterns to match on. You can only reason when you have awareness within which to reason, which these systems do not.

nl 4y ago
> You may be mistaking reasoning ability with sheer volume of more patterns to match on. It's not my work. And no, they probed this theory. The OCWCourses benchmark was new questions they generated from MIT Open Courseware that didn't exist prior to this benchmark. > You can only reason when you have awareness within which to reason, which these systems do not. I don't understand this objection at all. Reasoning seems roughly the same as logical inference, and logical inference is a computational process.
- gibsonf1 4y ago
  The LLM has no idea what words mean, so its quite impossible for them to reason about things. They just statistically mash up patterns of words from massive input with no clue whatsoever what those patterns of words actually mean in the real world.
  - nl 4y ago
    > The LLM has no idea what words mean Well that isn't true for any sensible definition of "idea of what words mean". If you examine embeddings produced by LLMs you find similar words in similar locations in the multi-dimensional space. > They just statistically mash up patterns of words from massive input with no clue whatsoever what those patterns of words actually mean in the real world. This is simply not true. As multiple papers (eg [1]) have shown these models can do tasks that require chained reasoning and give results that can only be explained by this ability. There's even a hint of a possible explanation in that paper: > For certain tasks, there may be natural intuitions for why emergence requires a model larger than a particular threshold scale. For instance, if a multi-step reasoning task requires l steps of sequential computation, this might require a model with a depth of at least O(l) layers. Your understanding might have been correct for LSTMs trained before 2018 or so, and it was a reasonable model in the BERT era of early transformers. But it needs to be updated now with more recent results. [1] https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html https://ai.googleblog.com/2022/11/characterizing-emergent-ph...
    - deleted 4y ago
      [deleted]
  - nuancebydefault 4y ago
    Define "mean". Then show/prove how a computer algo like GPT does not have that ability.
drakmo 4y ago
think about someone who looses the ability to store memories in long term memory and only has short term memory available. What would reasoning feel like for this person and for interactors with that person? What kind of awareness does one have when no long term contexts exist but only a single short term context?
deleted 4y ago
[deleted]
kilgnad 4y ago
No this is categorically wrong. There is science showing that LLMs build actual models around the subject it is learning. It is akin to someone looking over the shoulder at your math homework and deriving understanding just from that. This isn't a mistake. This is what the current science says. What makes me curious is that chatGPT even has output that displays complete understanding and is so novel it is impossible for the output to be anything else other then actual understanding. Yet people are still running around downplaying it all. The proof is right there. You can interact with chatGPT. The science is also there. Yet people just want to downplay it. Perhaps it's fear of the future and what it means for humanity?
phkahler 4y ago
>> You may be mistaking reasoning ability with sheer volume of more patterns to match on. You can only reason when you have awareness within which to reason, which these systems do not. What exactly is "reasoning"? Seem, like the ability to consider options and choose one that fits the situation best. One might even call it "prediction". I generally argue that these big brains of ours (and science too) are intended to predict the future of various hypotheticals in order to choose the path forward with the best outcome. Another frightening thing about chatGPT is that it has no "motivation". A lot of people think that AI will need some kind of motivation in order to be useful. How much of what we do as humans is actually done on autopilot? And then there's the NPC meme. I'm not sure these concepts (reason, thought, motivation) are so well defined that we will be certain when AI attains them or not.
nuancebydefault 4y ago
As soon as sheer volume of pattern matching results in answers indistinguishable from huge levels of awareness/reasoning, what is still the point in saying the algorithm is not "aware" or reasoning ? And believe me, the areas in which this appears the case increase day by day. Also, what makes you sure that awareness and reasoning is more than mere pattern matching? We humans find ourselves so special that it is simply hard to imagine that we would consist purely of a huge set of chemical and physical reactions.

You may be mistaking reasoning ability with sheer volume of more patterns to match on. You can only reason when you have awareness within which to reason, which these systems do not.

nl 4y ago

> You may be mistaking reasoning ability with sheer volume of more patterns to match on. It's not my work. And no, they probed this theory. The OCWCourses benchmark was new questions they generated from MIT Open Courseware that didn't exist prior to this benchmark. > You can only reason when you have awareness within which to reason, which these systems do not. I don't understand this objection at all. Reasoning seems roughly the same as logical inference, and logical inference is a computational process.

gibsonf1 4y ago
The LLM has no idea what words mean, so its quite impossible for them to reason about things. They just statistically mash up patterns of words from massive input with no clue whatsoever what those patterns of words actually mean in the real world.
- nl 4y ago
  > The LLM has no idea what words mean Well that isn't true for any sensible definition of "idea of what words mean". If you examine embeddings produced by LLMs you find similar words in similar locations in the multi-dimensional space. > They just statistically mash up patterns of words from massive input with no clue whatsoever what those patterns of words actually mean in the real world. This is simply not true. As multiple papers (eg [1]) have shown these models can do tasks that require chained reasoning and give results that can only be explained by this ability. There's even a hint of a possible explanation in that paper: > For certain tasks, there may be natural intuitions for why emergence requires a model larger than a particular threshold scale. For instance, if a multi-step reasoning task requires l steps of sequential computation, this might require a model with a depth of at least O(l) layers. Your understanding might have been correct for LSTMs trained before 2018 or so, and it was a reasonable model in the BERT era of early transformers. But it needs to be updated now with more recent results. [1] https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html https://ai.googleblog.com/2022/11/characterizing-emergent-ph...
  - deleted 4y ago
    [deleted]
- nuancebydefault 4y ago
  Define "mean". Then show/prove how a computer algo like GPT does not have that ability.

drakmo 4y ago

think about someone who looses the ability to store memories in long term memory and only has short term memory available. What would reasoning feel like for this person and for interactors with that person? What kind of awareness does one have when no long term contexts exist but only a single short term context?

deleted 4y ago

[deleted]

kilgnad 4y ago

No this is categorically wrong. There is science showing that LLMs build actual models around the subject it is learning. It is akin to someone looking over the shoulder at your math homework and deriving understanding just from that. This isn't a mistake. This is what the current science says. What makes me curious is that chatGPT even has output that displays complete understanding and is so novel it is impossible for the output to be anything else other then actual understanding. Yet people are still running around downplaying it all. The proof is right there. You can interact with chatGPT. The science is also there. Yet people just want to downplay it. Perhaps it's fear of the future and what it means for humanity?

phkahler 4y ago

>> You may be mistaking reasoning ability with sheer volume of more patterns to match on. You can only reason when you have awareness within which to reason, which these systems do not. What exactly is "reasoning"? Seem, like the ability to consider options and choose one that fits the situation best. One might even call it "prediction". I generally argue that these big brains of ours (and science too) are intended to predict the future of various hypotheticals in order to choose the path forward with the best outcome. Another frightening thing about chatGPT is that it has no "motivation". A lot of people think that AI will need some kind of motivation in order to be useful. How much of what we do as humans is actually done on autopilot? And then there's the NPC meme. I'm not sure these concepts (reason, thought, motivation) are so well defined that we will be certain when AI attains them or not.

nuancebydefault 4y ago

As soon as sheer volume of pattern matching results in answers indistinguishable from huge levels of awareness/reasoning, what is still the point in saying the algorithm is not "aware" or reasoning ? And believe me, the areas in which this appears the case increase day by day. Also, what makes you sure that awareness and reasoning is more than mere pattern matching? We humans find ourselves so special that it is simply hard to imagine that we would consist purely of a huge set of chemical and physical reactions.