4 ms·

Who? See: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 A 2021 research paper warning (among other th

by cowl 4y ago

Who? See: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 A 2021 research paper warning (among other things) precisly about this confusion. > The ersatz fluency and coherence of LMs raises several risks, precisely because humans are prepared to interpret strings belonging to languages they speak as meaningful and corresponding to the communicative intent of some individual or group of individuals who have accountability for what is said Is there any Researcher who maintains that LLM models contain Reasoning and intent? Those who are working on this models are not confused, they know what they are, the Public is confused.

pritambaral 4y ago
> Is there any Researcher who maintains that LLM models contain Reasoning and intent? Researchers with jobs that depend on promulgating this belief. E.g., "Open"AI employees.
FeepingCreature 4y ago
How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that paper, even if I discard the dross (ie. everything except one section on page 7), seems to actually make an argument against X of any strength beyond "it is wrong because it is wrong". My point is this: I disagree with you. This is not because I have "misunderstood" something; it is because I understand the stochastic-parrot argument and think it is erroneous. And the more you talk about "the risk that people will come to falsely believe" rather than actual arguments, the less convincing you sound. This paternalistic tendency is a curse on science and debate in general.
- cowl 4y ago
  Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.
  - FeepingCreature 4y ago
    Bad argument is bad argument, doesn't matter who says it.
- usrbinbash 4y ago
  > it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.
  - FeepingCreature 4y ago
    There's a class of statements that can be either interpreted precisely, at which point the claim they make is clearly true but trivial, or interpreted expansively, at which point the claim is significant but no longer clearly true. This is one of those: yes, technically LLMs are token predictors, but technically any nondeterministic Turing machine is a token predictor. The human brain could be viewed as a token predictor [1]. The interesting question is how it comes up with its predictions, and on this the phrase offers no insight at all. [1] https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
    - usrbinbash 4y ago
      > The human brain could be viewed as a token predictor No it really couldn't, because "generating and updating a 'mental model' of the environment." is as different from predicting the next token in a sequence, as a bees dance is from a structured human language. The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs that we make sense of not as abstract data points, but as experiences in a world of which we are part of. We also have a pre-existing model summarizing our experience in the world, including their degradation, our agency in that world, and our intentionality in that world. The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals, in itself already shows how far a humans mental modeling is from the linear action of a language model.
      - FeepingCreature 4y ago
        But the human mental model is purely internal. For that matter, there is strong evidence that LLMs generate mental models internally. [1] Our interface to motor actions is not dissimilar to a token predictor. > The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs So just like multimodal language models, for instance GPT-4? > as experiences in a world of which we are part of. > The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals Unfalsifiable! GPT-4 can talk about its experiences all day long. What's more, GPT-4 can act agentic if prompted correctly. [2] How do you qualify a "real goal"? [1] https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... [2] https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain
a_bonobo 4y ago
>Is there any Researcher who maintains that LLM models contain Reasoning and intent? That's the funny thing about the (in)famous OpenAI letter; the first sentence kind of does this: >AI systems with human-competitive intelligence can pose profound risks to society and humanity, as shown by extensive research[1] 'human-competitive intelligence' sounds like reasoning to me. What's even funnier is that [1] is the stochastic parrot paper, which argues exactly the opposite!
- usrbinbash 4y ago
  > 'human-competitive intelligence' sounds like reasoning to me. Yes, and when AIs reach that level of intelligence, we can revisit the question. However, as long as LLMs will confidently try to explain to me why several footballs fit in an average coffee mug, I'd say we are still quite some way away from "human-competitive intelligence".
  - a_bonobo 4y ago
    Yes, I fully agree with you! there's no reasoning or intelligence in modern LLMs, but the OpenAI open letter and recent comms strongly implies there is

by cowl 4y ago

pritambaral 4y ago

> Is there any Researcher who maintains that LLM models contain Reasoning and intent? Researchers with jobs that depend on promulgating this belief. E.g., "Open"AI employees.

FeepingCreature 4y ago

How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that paper, even if I discard the dross (ie. everything except one section on page 7), seems to actually make an argument against X of any strength beyond "it is wrong because it is wrong". My point is this: I disagree with you. This is not because I have "misunderstood" something; it is because I understand the stochastic-parrot argument and think it is erroneous. And the more you talk about "the risk that people will come to falsely believe" rather than actual arguments, the less convincing you sound. This paternalistic tendency is a curse on science and debate in general.

cowl 4y ago
Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.
- FeepingCreature 4y ago
  Bad argument is bad argument, doesn't matter who says it.
usrbinbash 4y ago
> it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.
- FeepingCreature 4y ago
  There's a class of statements that can be either interpreted precisely, at which point the claim they make is clearly true but trivial, or interpreted expansively, at which point the claim is significant but no longer clearly true. This is one of those: yes, technically LLMs are token predictors, but technically any nondeterministic Turing machine is a token predictor. The human brain could be viewed as a token predictor [1]. The interesting question is how it comes up with its predictions, and on this the phrase offers no insight at all. [1] https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
  - usrbinbash 4y ago
    > The human brain could be viewed as a token predictor No it really couldn't, because "generating and updating a 'mental model' of the environment." is as different from predicting the next token in a sequence, as a bees dance is from a structured human language. The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs that we make sense of not as abstract data points, but as experiences in a world of which we are part of. We also have a pre-existing model summarizing our experience in the world, including their degradation, our agency in that world, and our intentionality in that world. The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals, in itself already shows how far a humans mental modeling is from the linear action of a language model.
    - FeepingCreature 4y ago
      But the human mental model is purely internal. For that matter, there is strong evidence that LLMs generate mental models internally. [1] Our interface to motor actions is not dissimilar to a token predictor. > The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs So just like multimodal language models, for instance GPT-4? > as experiences in a world of which we are part of. > The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals Unfalsifiable! GPT-4 can talk about its experiences all day long. What's more, GPT-4 can act agentic if prompted correctly. [2] How do you qualify a "real goal"? [1] https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... [2] https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain

a_bonobo 4y ago

>Is there any Researcher who maintains that LLM models contain Reasoning and intent? That's the funny thing about the (in)famous OpenAI letter; the first sentence kind of does this: >AI systems with human-competitive intelligence can pose profound risks to society and humanity, as shown by extensive research[1] 'human-competitive intelligence' sounds like reasoning to me. What's even funnier is that [1] is the stochastic parrot paper, which argues exactly the opposite!

usrbinbash 4y ago
> 'human-competitive intelligence' sounds like reasoning to me. Yes, and when AIs reach that level of intelligence, we can revisit the question. However, as long as LLMs will confidently try to explain to me why several footballs fit in an average coffee mug, I'd say we are still quite some way away from "human-competitive intelligence".
- a_bonobo 4y ago
  Yes, I fully agree with you! there's no reasoning or intelligence in modern LLMs, but the OpenAI open letter and recent comms strongly implies there is