5 ms·

Lambda and these language models are not currently capable maintaining an internal train of thought that isn't expressed as token context. It has no memory of t

Lambda and these language models are not currently capable maintaining an internal train of thought that isn't expressed as token context. It has no memory of the world other than what we tell it to have. It can only "think" as long as it takes to infer a token. I think these big language models might get closer to something we would all call sentience once they can refer directly to their weights' previous activations for context instead of just tokens.

naasking 4y ago
> Lambda and these language models are not currently capable maintaining an internal train of thought that isn't expressed as token context. Why would that necessarily be insufficient for sentience? Sentience is just about the ability to "feel". Whatever "feeling" is, mechanistically, it will be some process that accepts some information and produces some output for these "feelings". It's not obvious that any kind of "memory" has to be part of the input to that process. For all we know, the transformer model is producing feelings as it computes, ie. that the next generated word is selected because it "feels right", where "feels right" simply means it's the most statistically likely word to follow the current token. Can you prove that this isn't basically what that gut feeling is for people? We don't actually know what "feelings" are fundamentally, so most of the people dismissing LaMDA's sentience don't really understand the can of worms they just stepped in.
- cscurmudgeon 4y ago
  The burden of evidence is on proving "feelings" existing in these models rather than proving "feelings" NOT existing in these models.
  - naasking 4y ago
    The burden of proof is on anyone making firm claims, whether they are claiming these models do or do not have feelings.
    - cscurmudgeon 4y ago
      No one was making claims till the guy from Google made the first claim.
- visarga 4y ago
  > We don't actually know what "feelings" are fundamentally I bet feelings are value functions - they estimate the expected future reward from a given state. As we move from self-supervised to RL, AIs will get feelings. But they are always task dependent, the system of values is built around goals and past experience, it's not something that can be separated from the specifics of the case. Humans come preloaded with reward signals that guide the development of feelings. We got our goals in-built - to survive, to thrive, to self replicate, and then all the sub-goals necessary to achieve the main directives - to move around and manipulate objects, to be social, to obtain food, etc.
  - naasking 4y ago
    A plausible abstract description, but what are the specific properties of value function that create the feeling? We need that mechanistic understanding to avoid creating feelings when we don't want a sentient system. More than likely this will make the system less effective at certain tasks as well, but the ethics around this are unclear and can only be discussed once that mechanistic understanding becomes clear.
    - visarga 4y ago
      I think feeling is an act of imagination, we imagine the future rewards, and this is necessary in order to select our actions. As we imagine scenarios we assign value to them, positive or negative, that makes them pop out as more than images.
woojoo666 4y ago
Was it sentient during training though?

naasking 4y ago

> Lambda and these language models are not currently capable maintaining an internal train of thought that isn't expressed as token context. Why would that necessarily be insufficient for sentience? Sentience is just about the ability to "feel". Whatever "feeling" is, mechanistically, it will be some process that accepts some information and produces some output for these "feelings". It's not obvious that any kind of "memory" has to be part of the input to that process. For all we know, the transformer model is producing feelings as it computes, ie. that the next generated word is selected because it "feels right", where "feels right" simply means it's the most statistically likely word to follow the current token. Can you prove that this isn't basically what that gut feeling is for people? We don't actually know what "feelings" are fundamentally, so most of the people dismissing LaMDA's sentience don't really understand the can of worms they just stepped in.

cscurmudgeon 4y ago
The burden of evidence is on proving "feelings" existing in these models rather than proving "feelings" NOT existing in these models.
- naasking 4y ago
  The burden of proof is on anyone making firm claims, whether they are claiming these models do or do not have feelings.
  - cscurmudgeon 4y ago
    No one was making claims till the guy from Google made the first claim.
visarga 4y ago
> We don't actually know what "feelings" are fundamentally I bet feelings are value functions - they estimate the expected future reward from a given state. As we move from self-supervised to RL, AIs will get feelings. But they are always task dependent, the system of values is built around goals and past experience, it's not something that can be separated from the specifics of the case. Humans come preloaded with reward signals that guide the development of feelings. We got our goals in-built - to survive, to thrive, to self replicate, and then all the sub-goals necessary to achieve the main directives - to move around and manipulate objects, to be social, to obtain food, etc.
- naasking 4y ago
  A plausible abstract description, but what are the specific properties of value function that create the feeling? We need that mechanistic understanding to avoid creating feelings when we don't want a sentient system. More than likely this will make the system less effective at certain tasks as well, but the ethics around this are unclear and can only be discussed once that mechanistic understanding becomes clear.
  - visarga 4y ago
    I think feeling is an act of imagination, we imagine the future rewards, and this is necessary in order to select our actions. As we imagine scenarios we assign value to them, positive or negative, that makes them pop out as more than images.

woojoo666 4y ago

Was it sentient during training though?