5 ms·

How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that

How is that different from "This evidence for X raises the risk that people falsely believe that X"? That's an argument for X, not against. And nothing in that paper, even if I discard the dross (ie. everything except one section on page 7), seems to actually make an argument against X of any strength beyond "it is wrong because it is wrong". My point is this: I disagree with you. This is not because I have "misunderstood" something; it is because I understand the stochastic-parrot argument and think it is erroneous. And the more you talk about "the risk that people will come to falsely believe" rather than actual arguments, the less convincing you sound. This paternalistic tendency is a curse on science and debate in general.

cowl 4y ago
Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.
- FeepingCreature 4y ago
  Bad argument is bad argument, doesn't matter who says it.
usrbinbash 4y ago
> it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.
- FeepingCreature 4y ago
  There's a class of statements that can be either interpreted precisely, at which point the claim they make is clearly true but trivial, or interpreted expansively, at which point the claim is significant but no longer clearly true. This is one of those: yes, technically LLMs are token predictors, but technically any nondeterministic Turing machine is a token predictor. The human brain could be viewed as a token predictor [1]. The interesting question is how it comes up with its predictions, and on this the phrase offers no insight at all. [1] https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
  - usrbinbash 4y ago
    > The human brain could be viewed as a token predictor No it really couldn't, because "generating and updating a 'mental model' of the environment." is as different from predicting the next token in a sequence, as a bees dance is from a structured human language. The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs that we make sense of not as abstract data points, but as experiences in a world of which we are part of. We also have a pre-existing model summarizing our experience in the world, including their degradation, our agency in that world, and our intentionality in that world. The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals, in itself already shows how far a humans mental modeling is from the linear action of a language model.
    - FeepingCreature 4y ago
      But the human mental model is purely internal. For that matter, there is strong evidence that LLMs generate mental models internally. [1] Our interface to motor actions is not dissimilar to a token predictor. > The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs So just like multimodal language models, for instance GPT-4? > as experiences in a world of which we are part of. > The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals Unfalsifiable! GPT-4 can talk about its experiences all day long. What's more, GPT-4 can act agentic if prompted correctly. [2] How do you qualify a "real goal"? [1] https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... [2] https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain
      - usrbinbash 4y ago
        > For that matter, there is strong evidence that LLMs generate mental models internally. Limited models, such as those representing the state of a game that it was trained to do: Yes. This is how we hope deep learning systems work in general. But I am not talking about limited models. I am talking about ad-hoc models, built from ingesting the context and semantic meaning of a string of tokens, that can simulate reality and allows drawing logical conclusions from it. In regard to my example given elsewhere in this HN thread: I know that Mike exits the elevator first because I build a mental model of what the tokens in the question represent. I can draw conclusions from that model, including new conclusions whos token-representation would be unlikely in the LLMs model, which doesn't explain anything about reality, but explains how tokens are usually ordered in the training set.

cowl 4y ago

Well what you are saying is that the argument of the creators of the model is erroneous and you know better. More power to you.

FeepingCreature 4y ago
Bad argument is bad argument, doesn't matter who says it.

usrbinbash 4y ago

> it is because I understand the stochastic-parrot argument and think it is erroneous. Okay then, what exactly about it is erroneous? Because stochastically sorting the set M of known tokens by likelyhood of being the next, is literally what LLMs do.

FeepingCreature 4y ago
There's a class of statements that can be either interpreted precisely, at which point the claim they make is clearly true but trivial, or interpreted expansively, at which point the claim is significant but no longer clearly true. This is one of those: yes, technically LLMs are token predictors, but technically any nondeterministic Turing machine is a token predictor. The human brain could be viewed as a token predictor [1]. The interesting question is how it comes up with its predictions, and on this the phrase offers no insight at all. [1] https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
- usrbinbash 4y ago
  > The human brain could be viewed as a token predictor No it really couldn't, because "generating and updating a 'mental model' of the environment." is as different from predicting the next token in a sequence, as a bees dance is from a structured human language. The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs that we make sense of not as abstract data points, but as experiences in a world of which we are part of. We also have a pre-existing model summarizing our experience in the world, including their degradation, our agency in that world, and our intentionality in that world. The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals, in itself already shows how far a humans mental modeling is from the linear action of a language model.
  - FeepingCreature 4y ago
    But the human mental model is purely internal. For that matter, there is strong evidence that LLMs generate mental models internally. [1] Our interface to motor actions is not dissimilar to a token predictor. > The mental model we build and update is not just based on a linear stream, but many parallel and even contradictory sensory inputs So just like multimodal language models, for instance GPT-4? > as experiences in a world of which we are part of. > The simple fact that we don't just complete streams, but do so with goals, both immediate and long term, and fit our actions into these goals Unfalsifiable! GPT-4 can talk about its experiences all day long. What's more, GPT-4 can act agentic if prompted correctly. [2] How do you qualify a "real goal"? [1] https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... [2] https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain
    - usrbinbash 4y ago
      > For that matter, there is strong evidence that LLMs generate mental models internally. Limited models, such as those representing the state of a game that it was trained to do: Yes. This is how we hope deep learning systems work in general. But I am not talking about limited models. I am talking about ad-hoc models, built from ingesting the context and semantic meaning of a string of tokens, that can simulate reality and allows drawing logical conclusions from it. In regard to my example given elsewhere in this HN thread: I know that Mike exits the elevator first because I build a mental model of what the tokens in the question represent. I can draw conclusions from that model, including new conclusions whos token-representation would be unlikely in the LLMs model, which doesn't explain anything about reality, but explains how tokens are usually ordered in the training set.