3 ms·

They are quite literally trained to predict a distribution that could supply the next token given previous context. This is their primary objective.

pdonis 3y ago
> They are quite literally trained to predict a distribution that could supply the next token given previous context. "Previous context" here means text. It does not mean "the actual world". Big difference.
- famouswaffles 3y ago
  So ? That's not a big difference lol. The world can be sufficiently represented in text. and bolting in additional modalities to a transformer is trivial. Could just as well be images and audio also. You think you experience the "true" world ? You don't. You experience a slice of it that your brain often further fabricates at parts. To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect. No one gets to experience the real world.
  - pdonis 3y ago
    > That's not a big difference lol. You must be joking. You really think there's not a big difference between a corpus of text downloaded from the Internet and the entire actual world? Or even the sense-data that humans (or birds, for that matter) take in from the world? (And that doesn't even take into account the actions that humans, and birds, take in the world, and then compare the results with what their internal model predicts in order to update the model.) I don't even know how to respond to something that is so totally off base. > You think you experience the "true" world ? I have made no such claim. > To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect. Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything.
    - famouswaffles 3y ago
      I'm not joking. Obviously there's a difference but it's not one of kind. There is a projection of the world in text. >Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything. Who is arguing whether LLMs are humans ? That's such an irrelevant point. The question is whether they are intelligent and understand which they are by any criteria that is actually testable. Nobody cares about building an artificial human. >LLMs don't experience anything. You don't know that
ckyngcybra 3y ago
[flagged]

pdonis 3y ago

> They are quite literally trained to predict a distribution that could supply the next token given previous context. "Previous context" here means text. It does not mean "the actual world". Big difference.

famouswaffles 3y ago
So ? That's not a big difference lol. The world can be sufficiently represented in text. and bolting in additional modalities to a transformer is trivial. Could just as well be images and audio also. You think you experience the "true" world ? You don't. You experience a slice of it that your brain often further fabricates at parts. To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect. No one gets to experience the real world.
- pdonis 3y ago
  > That's not a big difference lol. You must be joking. You really think there's not a big difference between a corpus of text downloaded from the Internet and the entire actual world? Or even the sense-data that humans (or birds, for that matter) take in from the world? (And that doesn't even take into account the actions that humans, and birds, take in the world, and then compare the results with what their internal model predicts in order to update the model.) I don't even know how to respond to something that is so totally off base. > You think you experience the "true" world ? I have made no such claim. > To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect. Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything.
  - famouswaffles 3y ago
    I'm not joking. Obviously there's a difference but it's not one of kind. There is a projection of the world in text. >Yes, but an LLM isn't a bird any more than it is a human. LLMs don't experience anything. Who is arguing whether LLMs are humans ? That's such an irrelevant point. The question is whether they are intelligent and understand which they are by any criteria that is actually testable. Nobody cares about building an artificial human. >LLMs don't experience anything. You don't know that

ckyngcybra 3y ago

[flagged]